2024-03-12T16:52:33.232 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-12T16:52:33.232 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/acls} --name yuriw-2024-03-12_14:59:27-fs-wip-yuri11-testing-2024-03-11-0838-reef-distro-default-smithi --owner scheduled_yuriw@teuthology --verbose --archive /home/teuthworker/archive/yuriw-2024-03-12_14:59:27-fs-wip-yuri11-testing-2024-03-11-0838-reef-distro-default-smithi/7593724 -- /home/teuthworker/archive/yuriw-2024-03-12_14:59:27-fs-wip-yuri11-testing-2024-03-11-0838-reef-distro-default-smithi/7593724/orig.config.yaml 2024-03-12T16:52:33.278 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-12T16:52:33.374 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-03-12_14:59:27-fs-wip-yuri11-testing-2024-03-11-0838-reef-distro-default-smithi/7593724 branch: wip-yuri11-testing-2024-03-11-0838-reef description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/acls} email: yweinste@redhat.com first_in_suite: false job_id: '7593724' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2024-03-12_14:59:27-fs-wip-yuri11-testing-2024-03-11-0838-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-yuri11-testing-2024-03-11-0838-reef ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - Reduced data availability - Degraded data redundancy sha1: 49e4a6794d5502bd66908faf714fc54896c51712 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 49e4a6794d5502bd66908faf714fc54896c51712 kclient: syntax: v2 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri11-testing-2024-03-11-0838-reef sha1: 49e4a6794d5502bd66908faf714fc54896c51712 owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 9096 sha1: 49e4a6794d5502bd66908faf714fc54896c51712 sleep_before_teardown: 0 subset: 111/120000 suite: fs suite_branch: wip-yuri11-testing-2024-03-11-0838-reef suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_49e4a6794d5502bd66908faf714fc54896c51712/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 49e4a6794d5502bd66908faf714fc54896c51712 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi052.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNP9kBxoQIrVoXMCHmLIBeBNFhyJHWPOQVv9rSZGWE09a9SJHaMmnq62ps8uuT4MnufX+/MMAvqYSiGfz0/5UbM= smithi102.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEiIR1hJQrQZGb6VRJkvAr1V6lA0j7iFz+k9F4XpdzY2y1WZkNwMFGYXVspkew3cAc+Ow+rLYDlJyPFQcugo9NA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_acls teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-12_14:59:27 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-12T16:52:33.374 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_49e4a6794d5502bd66908faf714fc54896c51712/qa; will attempt to use it 2024-03-12T16:52:33.375 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_49e4a6794d5502bd66908faf714fc54896c51712/qa/tasks 2024-03-12T16:52:33.375 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-12T16:52:33.376 INFO:teuthology.task.internal:Checking packages... 2024-03-12T16:52:33.401 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '49e4a6794d5502bd66908faf714fc54896c51712' 2024-03-12T16:52:33.401 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-12T16:52:33.401 INFO:teuthology.packaging:ref: None 2024-03-12T16:52:33.401 INFO:teuthology.packaging:tag: None 2024-03-12T16:52:33.402 INFO:teuthology.packaging:branch: wip-yuri11-testing-2024-03-11-0838-reef 2024-03-12T16:52:33.402 INFO:teuthology.packaging:sha1: 49e4a6794d5502bd66908faf714fc54896c51712 2024-03-12T16:52:33.402 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-yuri11-testing-2024-03-11-0838-reef 2024-03-12T16:52:33.548 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-797.g49e4a679 2024-03-12T16:52:33.550 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-12T16:52:33.556 INFO:teuthology.task.internal:no buildpackages task found 2024-03-12T16:52:33.556 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-12T16:52:33.611 INFO:teuthology.task.internal:Saving configuration 2024-03-12T16:52:33.627 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-12T16:52:33.690 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-12T16:52:33.721 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi052.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-12_14:59:27-fs-wip-yuri11-testing-2024-03-11-0838-reef-distro-default-smithi/7593724', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-12 16:47:30.869855', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNP9kBxoQIrVoXMCHmLIBeBNFhyJHWPOQVv9rSZGWE09a9SJHaMmnq62ps8uuT4MnufX+/MMAvqYSiGfz0/5UbM='} 2024-03-12T16:52:33.741 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi102.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-12_14:59:27-fs-wip-yuri11-testing-2024-03-11-0838-reef-distro-default-smithi/7593724', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-12 16:47:30.871359', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEiIR1hJQrQZGb6VRJkvAr1V6lA0j7iFz+k9F4XpdzY2y1WZkNwMFGYXVspkew3cAc+Ow+rLYDlJyPFQcugo9NA='} 2024-03-12T16:52:33.742 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-12T16:52:33.760 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-03-12T16:52:33.760 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-03-12T16:52:33.760 INFO:teuthology.run_tasks:Running task console_log... 2024-03-12T16:52:33.842 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0bc76bf160>, signals=[15]) 2024-03-12T16:52:33.842 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-12T16:52:33.848 INFO:teuthology.task.internal:Opening connections... 2024-03-12T16:52:33.848 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi052.front.sepia.ceph.com 2024-03-12T16:52:33.850 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T16:52:33.964 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi102.front.sepia.ceph.com 2024-03-12T16:52:33.965 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T16:52:34.053 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-12T16:52:34.060 DEBUG:teuthology.orchestra.run.smithi052:> uname -m 2024-03-12T16:52:34.078 INFO:teuthology.orchestra.run.smithi052.stdout:x86_64 2024-03-12T16:52:34.078 DEBUG:teuthology.orchestra.run.smithi052:> cat /etc/os-release 2024-03-12T16:52:34.136 INFO:teuthology.orchestra.run.smithi052.stdout:NAME="CentOS Stream" 2024-03-12T16:52:34.136 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION="9" 2024-03-12T16:52:34.136 INFO:teuthology.orchestra.run.smithi052.stdout:ID="centos" 2024-03-12T16:52:34.136 INFO:teuthology.orchestra.run.smithi052.stdout:ID_LIKE="rhel fedora" 2024-03-12T16:52:34.137 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION_ID="9" 2024-03-12T16:52:34.137 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM_ID="platform:el9" 2024-03-12T16:52:34.137 INFO:teuthology.orchestra.run.smithi052.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-12T16:52:34.137 INFO:teuthology.orchestra.run.smithi052.stdout:ANSI_COLOR="0;31" 2024-03-12T16:52:34.137 INFO:teuthology.orchestra.run.smithi052.stdout:LOGO="fedora-logo-icon" 2024-03-12T16:52:34.137 INFO:teuthology.orchestra.run.smithi052.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-12T16:52:34.137 INFO:teuthology.orchestra.run.smithi052.stdout:HOME_URL="https://centos.org/" 2024-03-12T16:52:34.137 INFO:teuthology.orchestra.run.smithi052.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-12T16:52:34.137 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-12T16:52:34.137 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-12T16:52:34.138 INFO:teuthology.lock.ops:Updating smithi052.front.sepia.ceph.com on lock server 2024-03-12T16:52:34.159 DEBUG:teuthology.orchestra.run.smithi102:> uname -m 2024-03-12T16:52:34.176 INFO:teuthology.orchestra.run.smithi102.stdout:x86_64 2024-03-12T16:52:34.176 DEBUG:teuthology.orchestra.run.smithi102:> cat /etc/os-release 2024-03-12T16:52:34.236 INFO:teuthology.orchestra.run.smithi102.stdout:NAME="CentOS Stream" 2024-03-12T16:52:34.236 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION="9" 2024-03-12T16:52:34.237 INFO:teuthology.orchestra.run.smithi102.stdout:ID="centos" 2024-03-12T16:52:34.237 INFO:teuthology.orchestra.run.smithi102.stdout:ID_LIKE="rhel fedora" 2024-03-12T16:52:34.237 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION_ID="9" 2024-03-12T16:52:34.237 INFO:teuthology.orchestra.run.smithi102.stdout:PLATFORM_ID="platform:el9" 2024-03-12T16:52:34.237 INFO:teuthology.orchestra.run.smithi102.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-12T16:52:34.237 INFO:teuthology.orchestra.run.smithi102.stdout:ANSI_COLOR="0;31" 2024-03-12T16:52:34.237 INFO:teuthology.orchestra.run.smithi102.stdout:LOGO="fedora-logo-icon" 2024-03-12T16:52:34.237 INFO:teuthology.orchestra.run.smithi102.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-12T16:52:34.237 INFO:teuthology.orchestra.run.smithi102.stdout:HOME_URL="https://centos.org/" 2024-03-12T16:52:34.237 INFO:teuthology.orchestra.run.smithi102.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-12T16:52:34.237 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-12T16:52:34.237 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-12T16:52:34.238 INFO:teuthology.lock.ops:Updating smithi102.front.sepia.ceph.com on lock server 2024-03-12T16:52:34.259 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-12T16:52:34.269 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-12T16:52:34.295 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-12T16:52:34.295 DEBUG:teuthology.orchestra.run.smithi052:> test '!' -e /home/ubuntu/cephtest 2024-03-12T16:52:34.297 DEBUG:teuthology.orchestra.run.smithi102:> test '!' -e /home/ubuntu/cephtest 2024-03-12T16:52:34.313 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-12T16:52:34.318 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-12T16:52:34.318 DEBUG:teuthology.orchestra.run.smithi052:> test -z $(ls -A /var/lib/ceph) 2024-03-12T16:52:34.354 DEBUG:teuthology.orchestra.run.smithi102:> test -z $(ls -A /var/lib/ceph) 2024-03-12T16:52:34.382 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-12T16:52:34.440 INFO:teuthology.run_tasks:Running task kernel... 2024-03-12T16:52:34.455 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-03-12T16:52:34.455 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-03-12T16:52:34.455 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-03-12T16:52:34.455 DEBUG:teuthology.orchestra.run.smithi052:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-12T16:52:34.456 DEBUG:teuthology.orchestra.run.smithi102:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-12T16:52:34.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T16:52:34.471 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-03-12T16:52:34.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T16:52:34.475 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-03-12T16:52:34.684 DEBUG:teuthology.task.kernel:sha1 for client.0 is bf761b6efc7e0ff8090671a5bd7700e429806836 2024-03-12T16:52:34.684 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "bf761b6efc7e0ff8090671a5bd7700e429806836"... 2024-03-12T16:52:34.684 DEBUG:teuthology.orchestra.run.smithi052:> uname -r 2024-03-12T16:52:34.703 INFO:teuthology.orchestra.run.smithi052.stdout:5.14.0-361.el9.x86_64 2024-03-12T16:52:34.703 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs bf761b6efc7e0ff8090671a5bd7700e429806836 2024-03-12T16:52:34.704 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-12T16:52:34.704 DEBUG:teuthology.orchestra.run.smithi052:> sudo yum upgrade -y linux-firmware 2024-03-12T16:52:34.861 DEBUG:teuthology.task.kernel:sha1 for client.1 is bf761b6efc7e0ff8090671a5bd7700e429806836 2024-03-12T16:52:34.861 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "bf761b6efc7e0ff8090671a5bd7700e429806836"... 2024-03-12T16:52:34.862 DEBUG:teuthology.orchestra.run.smithi102:> uname -r 2024-03-12T16:52:34.881 INFO:teuthology.orchestra.run.smithi102.stdout:5.14.0-361.el9.x86_64 2024-03-12T16:52:34.881 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs bf761b6efc7e0ff8090671a5bd7700e429806836 2024-03-12T16:52:34.882 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-12T16:52:34.882 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum upgrade -y linux-firmware 2024-03-12T16:52:37.161 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS-9 - AppStream 8.0 MB/s | 6.7 MB 00:00 2024-03-12T16:52:37.402 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS-9 - AppStream 7.8 MB/s | 6.7 MB 00:00 2024-03-12T16:52:38.828 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS-9 - BaseOS 3.9 MB/s | 2.0 MB 00:00 2024-03-12T16:52:39.110 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS-9 - BaseOS 3.4 MB/s | 2.0 MB 00:00 2024-03-12T16:52:45.722 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - BaseOS 1.3 MB/s | 8.0 MB 00:06 2024-03-12T16:52:45.734 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - BaseOS 1.2 MB/s | 8.0 MB 00:06 2024-03-12T16:52:49.966 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - AppStream 5.9 MB/s | 19 MB 00:03 2024-03-12T16:52:52.105 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - AppStream 3.6 MB/s | 19 MB 00:05 2024-03-12T16:53:04.537 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - CRB 564 kB/s | 6.1 MB 00:11 2024-03-12T16:53:07.396 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - Extras packages 10 kB/s | 15 kB 00:01 2024-03-12T16:53:07.822 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - CRB 519 kB/s | 6.1 MB 00:12 2024-03-12T16:53:08.983 INFO:teuthology.orchestra.run.smithi102.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-03-12T16:53:09.273 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - Extras packages 62 kB/s | 15 kB 00:00 2024-03-12T16:53:11.233 INFO:teuthology.orchestra.run.smithi052.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-03-12T16:53:14.039 INFO:teuthology.orchestra.run.smithi102.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-03-12T16:53:16.357 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-03-12T16:53:16.367 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-03-12T16:53:16.367 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-03-12T16:53:16.367 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-03-12T16:53:16.367 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrading: 2024-03-12T16:53:16.367 INFO:teuthology.orchestra.run.smithi102.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-12T16:53:16.367 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T16:53:16.367 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-03-12T16:53:16.368 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-03-12T16:53:16.368 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrade 1 Package 2024-03-12T16:53:16.368 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T16:53:16.369 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 375 M 2024-03-12T16:53:16.369 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-03-12T16:53:16.441 INFO:teuthology.orchestra.run.smithi052.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-03-12T16:53:18.811 INFO:teuthology.orchestra.run.smithi052.stdout:Dependencies resolved. 2024-03-12T16:53:18.819 INFO:teuthology.orchestra.run.smithi052.stdout:================================================================================ 2024-03-12T16:53:18.819 INFO:teuthology.orchestra.run.smithi052.stdout: Package Arch Version Repository Size 2024-03-12T16:53:18.819 INFO:teuthology.orchestra.run.smithi052.stdout:================================================================================ 2024-03-12T16:53:18.819 INFO:teuthology.orchestra.run.smithi052.stdout:Upgrading: 2024-03-12T16:53:18.819 INFO:teuthology.orchestra.run.smithi052.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-12T16:53:18.820 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T16:53:18.820 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction Summary 2024-03-12T16:53:18.820 INFO:teuthology.orchestra.run.smithi052.stdout:================================================================================ 2024-03-12T16:53:18.820 INFO:teuthology.orchestra.run.smithi052.stdout:Upgrade 1 Package 2024-03-12T16:53:18.820 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T16:53:18.820 INFO:teuthology.orchestra.run.smithi052.stdout:Total download size: 375 M 2024-03-12T16:53:18.820 INFO:teuthology.orchestra.run.smithi052.stdout:Downloading Packages: 2024-03-12T16:53:29.382 INFO:teuthology.orchestra.run.smithi102.stdout:linux-firmware-20240219-143.el9.noarch.rpm 29 MB/s | 375 MB 00:13 2024-03-12T16:53:29.383 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-03-12T16:53:29.383 INFO:teuthology.orchestra.run.smithi102.stdout:Total 29 MB/s | 375 MB 00:13 2024-03-12T16:53:29.383 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-03-12T16:53:29.473 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-03-12T16:53:29.473 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-03-12T16:53:31.150 INFO:teuthology.orchestra.run.smithi052.stdout:linux-firmware-20240219-143.el9.noarch.rpm 30 MB/s | 375 MB 00:12 2024-03-12T16:53:31.151 INFO:teuthology.orchestra.run.smithi052.stdout:-------------------------------------------------------------------------------- 2024-03-12T16:53:31.152 INFO:teuthology.orchestra.run.smithi052.stdout:Total 30 MB/s | 375 MB 00:12 2024-03-12T16:53:31.152 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction check 2024-03-12T16:53:31.255 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction check succeeded. 2024-03-12T16:53:31.255 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction test 2024-03-12T16:53:32.126 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-03-12T16:53:32.126 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-03-12T16:53:33.738 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction test succeeded. 2024-03-12T16:53:33.738 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction 2024-03-12T16:53:38.452 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-03-12T16:53:38.732 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-12T16:53:38.831 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-12T16:53:39.985 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-12T16:53:39.985 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-12T16:53:40.386 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-12T16:53:40.387 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T16:53:40.387 INFO:teuthology.orchestra.run.smithi102.stdout:Upgraded: 2024-03-12T16:53:40.387 INFO:teuthology.orchestra.run.smithi102.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-12T16:53:40.387 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T16:53:40.387 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-03-12T16:53:40.410 INFO:teuthology.orchestra.run.smithi052.stdout: Preparing : 1/1 2024-03-12T16:53:40.662 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-12T16:53:40.683 INFO:teuthology.task.kernel:Downloading kernel bf761b6efc7e0ff8090671a5bd7700e429806836 on client.1... 2024-03-12T16:53:40.684 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bf761b6efc7e0ff8090671a5bd7700e429806836 2024-03-12T16:53:40.752 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-12T16:53:40.834 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/bf761b6efc7e0ff8090671a5bd7700e429806836/centos/9/flavors/default/x86_64/ 2024-03-12T16:53:40.834 DEBUG:teuthology.orchestra.run.smithi102:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_rc7_gbf761b6efc7e-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/bf761b6efc7e0ff8090671a5bd7700e429806836/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-12T16:53:40.837 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi102.front.sepia.ceph.com to complete... 2024-03-12T16:53:41.864 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-12T16:53:41.864 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-12T16:53:42.274 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-12T16:53:42.274 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T16:53:42.274 INFO:teuthology.orchestra.run.smithi052.stdout:Upgraded: 2024-03-12T16:53:42.274 INFO:teuthology.orchestra.run.smithi052.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-12T16:53:42.274 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T16:53:42.274 INFO:teuthology.orchestra.run.smithi052.stdout:Complete! 2024-03-12T16:53:42.565 INFO:teuthology.task.kernel:Downloading kernel bf761b6efc7e0ff8090671a5bd7700e429806836 on client.0... 2024-03-12T16:53:42.738 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bf761b6efc7e0ff8090671a5bd7700e429806836 2024-03-12T16:53:42.886 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/bf761b6efc7e0ff8090671a5bd7700e429806836/centos/9/flavors/default/x86_64/ 2024-03-12T16:53:42.886 DEBUG:teuthology.orchestra.run.smithi052:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_rc7_gbf761b6efc7e-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/bf761b6efc7e0ff8090671a5bd7700e429806836/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-12T16:53:42.888 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi052.front.sepia.ceph.com to complete... 2024-03-12T16:54:02.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12 16:54:02 URL:https://3.chacra.ceph.com/r/kernel/testing/bf761b6efc7e0ff8090671a5bd7700e429806836/centos/9/flavors/default/x86_64/kernel-6.8.0_rc7_gbf761b6efc7e-1.x86_64.rpm [336341774/336341774] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-12T16:54:02.159 INFO:teuthology.orchestra.run.smithi052.stderr:FINISHED --2024-03-12 16:54:02-- 2024-03-12T16:54:02.159 INFO:teuthology.orchestra.run.smithi052.stderr:Total wall clock time: 19s 2024-03-12T16:54:02.159 INFO:teuthology.orchestra.run.smithi052.stderr:Downloaded: 1 files, 321M in 19s (17.0 MB/s) 2024-03-12T16:54:02.162 INFO:teuthology.task.kernel:Installing kernel bf761b6efc7e0ff8090671a5bd7700e429806836 on client.0... 2024-03-12T16:54:02.163 DEBUG:teuthology.orchestra.run.smithi052:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-12T16:54:03.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12 16:54:03 URL:https://3.chacra.ceph.com/r/kernel/testing/bf761b6efc7e0ff8090671a5bd7700e429806836/centos/9/flavors/default/x86_64/kernel-6.8.0_rc7_gbf761b6efc7e-1.x86_64.rpm [336341774/336341774] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-12T16:54:03.250 INFO:teuthology.orchestra.run.smithi102.stderr:FINISHED --2024-03-12 16:54:03-- 2024-03-12T16:54:03.250 INFO:teuthology.orchestra.run.smithi102.stderr:Total wall clock time: 22s 2024-03-12T16:54:03.250 INFO:teuthology.orchestra.run.smithi102.stderr:Downloaded: 1 files, 321M in 22s (14.5 MB/s) 2024-03-12T16:54:03.252 INFO:teuthology.task.kernel:Installing kernel bf761b6efc7e0ff8090671a5bd7700e429806836 on client.1... 2024-03-12T16:54:03.252 DEBUG:teuthology.orchestra.run.smithi102:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-12T16:54:04.196 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying... ######################################## 2024-03-12T16:54:04.251 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing... ######################################## 2024-03-12T16:54:04.336 INFO:teuthology.orchestra.run.smithi052.stdout:Updating / installing... 2024-03-12T16:54:05.194 INFO:teuthology.orchestra.run.smithi102.stdout:Verifying... ######################################## 2024-03-12T16:54:05.243 INFO:teuthology.orchestra.run.smithi102.stdout:Preparing... ######################################## 2024-03-12T16:54:05.326 INFO:teuthology.orchestra.run.smithi102.stdout:Updating / installing... 2024-03-12T16:54:12.184 INFO:teuthology.orchestra.run.smithi052.stdout:kernel-6.8.0_rc7_gbf761b6efc7e-1 ######################################## 2024-03-12T16:54:12.828 INFO:teuthology.orchestra.run.smithi102.stdout:kernel-6.8.0_rc7_gbf761b6efc7e-1 ######################################## 2024-03-12T16:54:28.464 INFO:teuthology.orchestra.run.smithi052.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-12T16:54:28.870 INFO:teuthology.orchestra.run.smithi102.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-12T16:54:51.390 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi102.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-12T16:54:51.391 DEBUG:teuthology.orchestra.run.smithi102:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-12T16:54:51.464 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e 2024-03-12T16:54:51.464 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/System.map 2024-03-12T16:54:51.464 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/config 2024-03-12T16:54:51.464 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel 2024-03-12T16:54:51.464 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch 2024-03-12T16:54:51.465 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86 2024-03-12T16:54:51.465 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto 2024-03-12T16:54:51.465 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-12T16:54:51.465 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-12T16:54:51.465 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-12T16:54:51.465 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-12T16:54:51.465 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-12T16:54:51.465 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-12T16:54:51.465 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-12T16:54:51.465 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-12T16:54:51.465 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-12T16:54:51.465 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-12T16:54:51.466 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-12T16:54:51.466 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-12T16:54:51.466 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-12T16:54:51.466 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-12T16:54:51.466 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-12T16:54:51.466 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-12T16:54:51.466 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-12T16:54:51.466 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-12T16:54:51.466 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-12T16:54:51.466 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-12T16:54:51.466 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/kernel 2024-03-12T16:54:51.466 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/kernel/cpu 2024-03-12T16:54:51.467 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/kernel/cpu/mce 2024-03-12T16:54:51.467 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-12T16:54:51.467 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/kvm 2024-03-12T16:54:51.467 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-12T16:54:51.467 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-12T16:54:51.467 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/kvm/kvm.ko 2024-03-12T16:54:51.467 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto 2024-03-12T16:54:51.467 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/ansi_cprng.ko 2024-03-12T16:54:51.467 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/anubis.ko 2024-03-12T16:54:51.467 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/arc4.ko 2024-03-12T16:54:51.467 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/async_tx 2024-03-12T16:54:51.468 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/async_tx/async_memcpy.ko 2024-03-12T16:54:51.468 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/async_tx/async_pq.ko 2024-03-12T16:54:51.468 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-12T16:54:51.468 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/async_tx/async_tx.ko 2024-03-12T16:54:51.468 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/async_tx/async_xor.ko 2024-03-12T16:54:51.468 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/async_tx/raid6test.ko 2024-03-12T16:54:51.468 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/authenc.ko 2024-03-12T16:54:51.468 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/authencesn.ko 2024-03-12T16:54:51.468 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/blake2b_generic.ko 2024-03-12T16:54:51.468 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/blowfish_common.ko 2024-03-12T16:54:51.468 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/blowfish_generic.ko 2024-03-12T16:54:51.468 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/camellia_generic.ko 2024-03-12T16:54:51.469 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/cast5_generic.ko 2024-03-12T16:54:51.469 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/cast6_generic.ko 2024-03-12T16:54:51.469 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/cast_common.ko 2024-03-12T16:54:51.469 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/ccm.ko 2024-03-12T16:54:51.469 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/cmac.ko 2024-03-12T16:54:51.469 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/crc32_generic.ko 2024-03-12T16:54:51.469 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/crypto_engine.ko 2024-03-12T16:54:51.469 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/crypto_user.ko 2024-03-12T16:54:51.469 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/cts.ko 2024-03-12T16:54:51.469 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/des_generic.ko 2024-03-12T16:54:51.469 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/dh_generic.ko 2024-03-12T16:54:51.469 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/echainiv.ko 2024-03-12T16:54:51.470 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/essiv.ko 2024-03-12T16:54:51.470 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/fcrypt.ko 2024-03-12T16:54:51.470 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/khazad.ko 2024-03-12T16:54:51.470 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/lz4.ko 2024-03-12T16:54:51.470 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/lz4hc.ko 2024-03-12T16:54:51.470 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/md4.ko 2024-03-12T16:54:51.470 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/michael_mic.ko 2024-03-12T16:54:51.470 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/pcbc.ko 2024-03-12T16:54:51.470 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/pcrypt.ko 2024-03-12T16:54:51.470 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/rmd160.ko 2024-03-12T16:54:51.470 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/seed.ko 2024-03-12T16:54:51.470 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/serpent_generic.ko 2024-03-12T16:54:51.471 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/tcrypt.ko 2024-03-12T16:54:51.471 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/tea.ko 2024-03-12T16:54:51.471 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/twofish_common.ko 2024-03-12T16:54:51.471 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/twofish_generic.ko 2024-03-12T16:54:51.471 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/vmac.ko 2024-03-12T16:54:51.471 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/wp512.ko 2024-03-12T16:54:51.471 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/xcbc.ko 2024-03-12T16:54:51.471 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/xor.ko 2024-03-12T16:54:51.471 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/xxhash_generic.ko 2024-03-12T16:54:51.471 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers 2024-03-12T16:54:51.471 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/acpi 2024-03-12T16:54:51.472 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-12T16:54:51.472 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/acpi/acpi_pad.ko 2024-03-12T16:54:51.472 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/acpi/custom_method.ko 2024-03-12T16:54:51.472 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/acpi/ec_sys.ko 2024-03-12T16:54:51.472 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/acpi/platform_profile.ko 2024-03-12T16:54:51.472 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/acpi/sbs.ko 2024-03-12T16:54:51.472 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/acpi/sbshc.ko 2024-03-12T16:54:51.472 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/acpi/video.ko 2024-03-12T16:54:51.472 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata 2024-03-12T16:54:51.472 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/acard-ahci.ko 2024-03-12T16:54:51.472 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/ahci_platform.ko 2024-03-12T16:54:51.472 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/ata_generic.ko 2024-03-12T16:54:51.472 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/libahci_platform.ko 2024-03-12T16:54:51.473 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_acpi.ko 2024-03-12T16:54:51.473 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_ali.ko 2024-03-12T16:54:51.473 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_amd.ko 2024-03-12T16:54:51.473 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_artop.ko 2024-03-12T16:54:51.473 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_atiixp.ko 2024-03-12T16:54:51.473 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_atp867x.ko 2024-03-12T16:54:51.473 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_cmd640.ko 2024-03-12T16:54:51.473 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_cmd64x.ko 2024-03-12T16:54:51.473 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_cypress.ko 2024-03-12T16:54:51.473 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_efar.ko 2024-03-12T16:54:51.473 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_hpt366.ko 2024-03-12T16:54:51.473 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_hpt37x.ko 2024-03-12T16:54:51.473 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-12T16:54:51.474 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-12T16:54:51.474 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_it8213.ko 2024-03-12T16:54:51.474 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_it821x.ko 2024-03-12T16:54:51.474 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_jmicron.ko 2024-03-12T16:54:51.474 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_marvell.ko 2024-03-12T16:54:51.474 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_mpiix.ko 2024-03-12T16:54:51.474 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_netcell.ko 2024-03-12T16:54:51.474 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_ninja32.ko 2024-03-12T16:54:51.474 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_ns87410.ko 2024-03-12T16:54:51.474 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_ns87415.ko 2024-03-12T16:54:51.474 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_oldpiix.ko 2024-03-12T16:54:51.474 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_opti.ko 2024-03-12T16:54:51.474 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_optidma.ko 2024-03-12T16:54:51.475 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_pcmcia.ko 2024-03-12T16:54:51.475 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-12T16:54:51.475 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-12T16:54:51.475 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_piccolo.ko 2024-03-12T16:54:51.475 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_rdc.ko 2024-03-12T16:54:51.475 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_sch.ko 2024-03-12T16:54:51.475 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_serverworks.ko 2024-03-12T16:54:51.475 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_sil680.ko 2024-03-12T16:54:51.475 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_sis.ko 2024-03-12T16:54:51.475 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_sl82c105.ko 2024-03-12T16:54:51.475 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_triflex.ko 2024-03-12T16:54:51.475 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_via.ko 2024-03-12T16:54:51.475 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pdc_adma.ko 2024-03-12T16:54:51.476 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/sata_inic162x.ko 2024-03-12T16:54:51.476 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/sata_mv.ko 2024-03-12T16:54:51.476 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/sata_nv.ko 2024-03-12T16:54:51.476 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/sata_promise.ko 2024-03-12T16:54:51.476 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/sata_qstor.ko 2024-03-12T16:54:51.476 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/sata_sil.ko 2024-03-12T16:54:51.476 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/sata_sil24.ko 2024-03-12T16:54:51.476 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/sata_sis.ko 2024-03-12T16:54:51.476 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/sata_svw.ko 2024-03-12T16:54:51.476 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/sata_sx4.ko 2024-03-12T16:54:51.476 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/sata_uli.ko 2024-03-12T16:54:51.476 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/sata_via.ko 2024-03-12T16:54:51.476 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/sata_vsc.ko 2024-03-12T16:54:51.477 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/auxdisplay 2024-03-12T16:54:51.477 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/auxdisplay/ks0108.ko 2024-03-12T16:54:51.477 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/base 2024-03-12T16:54:51.477 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/base/regmap 2024-03-12T16:54:51.477 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-12T16:54:51.477 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-12T16:54:51.477 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/bcma 2024-03-12T16:54:51.477 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/bcma/bcma.ko 2024-03-12T16:54:51.477 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block 2024-03-12T16:54:51.477 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/aoe 2024-03-12T16:54:51.477 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/aoe/aoe.ko 2024-03-12T16:54:51.477 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/brd.ko 2024-03-12T16:54:51.477 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/drbd 2024-03-12T16:54:51.477 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/drbd/drbd.ko 2024-03-12T16:54:51.478 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/floppy.ko 2024-03-12T16:54:51.478 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/loop.ko 2024-03-12T16:54:51.478 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/mtip32xx 2024-03-12T16:54:51.478 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-12T16:54:51.478 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/nbd.ko 2024-03-12T16:54:51.478 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/null_blk 2024-03-12T16:54:51.478 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/null_blk/null_blk.ko 2024-03-12T16:54:51.478 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/pktcdvd.ko 2024-03-12T16:54:51.478 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/rbd.ko 2024-03-12T16:54:51.478 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/virtio_blk.ko 2024-03-12T16:54:51.478 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/xen-blkback 2024-03-12T16:54:51.478 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-12T16:54:51.478 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/xen-blkfront.ko 2024-03-12T16:54:51.478 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/zram 2024-03-12T16:54:51.479 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/zram/zram.ko 2024-03-12T16:54:51.479 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char 2024-03-12T16:54:51.479 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/hangcheck-timer.ko 2024-03-12T16:54:51.479 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/hw_random 2024-03-12T16:54:51.479 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-12T16:54:51.479 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-12T16:54:51.479 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-12T16:54:51.479 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/hw_random/via-rng.ko 2024-03-12T16:54:51.479 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-12T16:54:51.479 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/ipmi 2024-03-12T16:54:51.479 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-12T16:54:51.479 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-12T16:54:51.479 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-12T16:54:51.480 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-12T16:54:51.480 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-12T16:54:51.480 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-12T16:54:51.480 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/lp.ko 2024-03-12T16:54:51.480 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/mwave 2024-03-12T16:54:51.480 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/mwave/mwave.ko 2024-03-12T16:54:51.480 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/ppdev.ko 2024-03-12T16:54:51.480 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/tlclk.ko 2024-03-12T16:54:51.480 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/tpm 2024-03-12T16:54:51.480 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/tpm/tpm.ko 2024-03-12T16:54:51.480 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-12T16:54:51.480 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-12T16:54:51.480 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-12T16:54:51.480 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-12T16:54:51.481 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-12T16:54:51.481 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/uv_mmtimer.ko 2024-03-12T16:54:51.481 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/virtio_console.ko 2024-03-12T16:54:51.481 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/cpufreq 2024-03-12T16:54:51.481 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-12T16:54:51.481 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-12T16:54:51.481 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-12T16:54:51.481 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-12T16:54:51.481 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-12T16:54:51.481 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-12T16:54:51.481 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/crypto 2024-03-12T16:54:51.481 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/crypto/ccp 2024-03-12T16:54:51.481 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-12T16:54:51.481 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/crypto/ccp/ccp.ko 2024-03-12T16:54:51.482 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/crypto/intel 2024-03-12T16:54:51.482 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/crypto/intel/qat 2024-03-12T16:54:51.482 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/crypto/intel/qat/qat_common 2024-03-12T16:54:51.482 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-12T16:54:51.482 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-12T16:54:51.482 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-12T16:54:51.482 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/crypto/padlock-aes.ko 2024-03-12T16:54:51.482 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/crypto/padlock-sha.ko 2024-03-12T16:54:51.482 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/crypto/virtio 2024-03-12T16:54:51.482 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-12T16:54:51.482 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/dca 2024-03-12T16:54:51.482 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/dca/dca.ko 2024-03-12T16:54:51.482 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/devfreq 2024-03-12T16:54:51.482 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-12T16:54:51.483 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/dma 2024-03-12T16:54:51.483 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/dma/dw 2024-03-12T16:54:51.483 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-12T16:54:51.483 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/dma/ioat 2024-03-12T16:54:51.483 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-12T16:54:51.483 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/edac 2024-03-12T16:54:51.483 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/edac/amd64_edac.ko 2024-03-12T16:54:51.483 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/edac/e752x_edac.ko 2024-03-12T16:54:51.483 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/edac/edac_mce_amd.ko 2024-03-12T16:54:51.483 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/edac/i3000_edac.ko 2024-03-12T16:54:51.483 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/edac/i3200_edac.ko 2024-03-12T16:54:51.483 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/edac/i5100_edac.ko 2024-03-12T16:54:51.483 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/edac/i5400_edac.ko 2024-03-12T16:54:51.483 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/edac/i7300_edac.ko 2024-03-12T16:54:51.484 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/edac/i7core_edac.ko 2024-03-12T16:54:51.484 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/edac/i82975x_edac.ko 2024-03-12T16:54:51.484 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/edac/ie31200_edac.ko 2024-03-12T16:54:51.484 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/edac/sb_edac.ko 2024-03-12T16:54:51.484 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/edac/x38_edac.ko 2024-03-12T16:54:51.484 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/firmware 2024-03-12T16:54:51.484 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/firmware/edd.ko 2024-03-12T16:54:51.484 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-12T16:54:51.484 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/gpio 2024-03-12T16:54:51.484 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/gpio/gpio-ich.ko 2024-03-12T16:54:51.484 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-12T16:54:51.484 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid 2024-03-12T16:54:51.484 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-appleir.ko 2024-03-12T16:54:51.484 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-aureal.ko 2024-03-12T16:54:51.485 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-axff.ko 2024-03-12T16:54:51.485 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-dr.ko 2024-03-12T16:54:51.485 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-elecom.ko 2024-03-12T16:54:51.485 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-elo.ko 2024-03-12T16:54:51.485 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-emsff.ko 2024-03-12T16:54:51.485 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-gaff.ko 2024-03-12T16:54:51.485 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-gt683r.ko 2024-03-12T16:54:51.485 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-gyration.ko 2024-03-12T16:54:51.485 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-12T16:54:51.485 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-12T16:54:51.485 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-holtekff.ko 2024-03-12T16:54:51.485 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-hyperv.ko 2024-03-12T16:54:51.485 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-icade.ko 2024-03-12T16:54:51.485 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-keytouch.ko 2024-03-12T16:54:51.486 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-kye.ko 2024-03-12T16:54:51.486 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-lcpower.ko 2024-03-12T16:54:51.486 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-led.ko 2024-03-12T16:54:51.486 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-lenovo.ko 2024-03-12T16:54:51.486 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-12T16:54:51.486 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-12T16:54:51.486 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-multitouch.ko 2024-03-12T16:54:51.486 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-ortek.ko 2024-03-12T16:54:51.486 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-penmount.ko 2024-03-12T16:54:51.486 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-petalynx.ko 2024-03-12T16:54:51.486 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-picolcd.ko 2024-03-12T16:54:51.486 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-pl.ko 2024-03-12T16:54:51.486 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-plantronics.ko 2024-03-12T16:54:51.487 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-primax.ko 2024-03-12T16:54:51.487 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-rmi.ko 2024-03-12T16:54:51.487 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-12T16:54:51.487 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-roccat-common.ko 2024-03-12T16:54:51.487 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-12T16:54:51.487 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-12T16:54:51.487 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-12T16:54:51.487 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-12T16:54:51.487 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-12T16:54:51.487 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-12T16:54:51.487 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-12T16:54:51.487 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-12T16:54:51.487 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-12T16:54:51.487 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-roccat.ko 2024-03-12T16:54:51.488 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-saitek.ko 2024-03-12T16:54:51.488 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-samsung.ko 2024-03-12T16:54:51.488 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-12T16:54:51.488 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-sjoy.ko 2024-03-12T16:54:51.488 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-sony.ko 2024-03-12T16:54:51.488 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-speedlink.ko 2024-03-12T16:54:51.488 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-steelseries.ko 2024-03-12T16:54:51.488 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-sunplus.ko 2024-03-12T16:54:51.488 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-12T16:54:51.488 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-tivo.ko 2024-03-12T16:54:51.488 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-tmff.ko 2024-03-12T16:54:51.488 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-topseed.ko 2024-03-12T16:54:51.488 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-twinhan.ko 2024-03-12T16:54:51.488 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-uclogic.ko 2024-03-12T16:54:51.489 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-waltop.ko 2024-03-12T16:54:51.489 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-wiimote.ko 2024-03-12T16:54:51.489 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-xinmo.ko 2024-03-12T16:54:51.489 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-zpff.ko 2024-03-12T16:54:51.489 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-zydacron.ko 2024-03-12T16:54:51.489 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/uhid.ko 2024-03-12T16:54:51.489 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/wacom.ko 2024-03-12T16:54:51.489 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hv 2024-03-12T16:54:51.489 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hv/hv_balloon.ko 2024-03-12T16:54:51.489 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hv/hv_utils.ko 2024-03-12T16:54:51.489 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hv/hv_vmbus.ko 2024-03-12T16:54:51.489 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon 2024-03-12T16:54:51.489 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/abituguru.ko 2024-03-12T16:54:51.489 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/abituguru3.ko 2024-03-12T16:54:51.490 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-12T16:54:51.490 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/ad7414.ko 2024-03-12T16:54:51.490 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/ad7418.ko 2024-03-12T16:54:51.490 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/adc128d818.ko 2024-03-12T16:54:51.490 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/adm1025.ko 2024-03-12T16:54:51.490 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/adm1026.ko 2024-03-12T16:54:51.490 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/adm1029.ko 2024-03-12T16:54:51.490 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/adm1031.ko 2024-03-12T16:54:51.490 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/adm9240.ko 2024-03-12T16:54:51.490 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/ads7828.ko 2024-03-12T16:54:51.490 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/adt7410.ko 2024-03-12T16:54:51.490 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/adt7411.ko 2024-03-12T16:54:51.490 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/adt7462.ko 2024-03-12T16:54:51.490 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/adt7470.ko 2024-03-12T16:54:51.491 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/adt7475.ko 2024-03-12T16:54:51.491 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/adt7x10.ko 2024-03-12T16:54:51.491 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/amc6821.ko 2024-03-12T16:54:51.491 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/applesmc.ko 2024-03-12T16:54:51.491 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/asb100.ko 2024-03-12T16:54:51.491 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/asc7621.ko 2024-03-12T16:54:51.491 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-12T16:54:51.491 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/atxp1.ko 2024-03-12T16:54:51.491 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/coretemp.ko 2024-03-12T16:54:51.491 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-12T16:54:51.491 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/dme1737.ko 2024-03-12T16:54:51.491 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/ds1621.ko 2024-03-12T16:54:51.491 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/ds620.ko 2024-03-12T16:54:51.491 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/emc1403.ko 2024-03-12T16:54:51.492 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/emc6w201.ko 2024-03-12T16:54:51.492 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/f71805f.ko 2024-03-12T16:54:51.492 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/f71882fg.ko 2024-03-12T16:54:51.492 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/f75375s.ko 2024-03-12T16:54:51.492 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/fam15h_power.ko 2024-03-12T16:54:51.492 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/fschmd.ko 2024-03-12T16:54:51.492 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/g760a.ko 2024-03-12T16:54:51.492 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/g762.ko 2024-03-12T16:54:51.492 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/gl518sm.ko 2024-03-12T16:54:51.492 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/gl520sm.ko 2024-03-12T16:54:51.492 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-12T16:54:51.492 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/i5500_temp.ko 2024-03-12T16:54:51.492 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/i5k_amb.ko 2024-03-12T16:54:51.492 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/ibmaem.ko 2024-03-12T16:54:51.493 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/ibmpex.ko 2024-03-12T16:54:51.493 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/ina209.ko 2024-03-12T16:54:51.493 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/ina2xx.ko 2024-03-12T16:54:51.493 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/it87.ko 2024-03-12T16:54:51.493 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/k10temp.ko 2024-03-12T16:54:51.493 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/k8temp.ko 2024-03-12T16:54:51.493 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/lineage-pem.ko 2024-03-12T16:54:51.493 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/lm63.ko 2024-03-12T16:54:51.493 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/lm73.ko 2024-03-12T16:54:51.493 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/lm75.ko 2024-03-12T16:54:51.493 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/lm77.ko 2024-03-12T16:54:51.493 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/lm78.ko 2024-03-12T16:54:51.493 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/lm80.ko 2024-03-12T16:54:51.493 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/lm83.ko 2024-03-12T16:54:51.494 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/lm85.ko 2024-03-12T16:54:51.494 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/lm87.ko 2024-03-12T16:54:51.494 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/lm90.ko 2024-03-12T16:54:51.494 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/lm92.ko 2024-03-12T16:54:51.494 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/lm93.ko 2024-03-12T16:54:51.494 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/lm95234.ko 2024-03-12T16:54:51.494 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/lm95241.ko 2024-03-12T16:54:51.494 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/lm95245.ko 2024-03-12T16:54:51.494 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/ltc2945.ko 2024-03-12T16:54:51.494 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/ltc4151.ko 2024-03-12T16:54:51.494 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/ltc4215.ko 2024-03-12T16:54:51.494 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/ltc4222.ko 2024-03-12T16:54:51.494 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/ltc4245.ko 2024-03-12T16:54:51.495 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/ltc4260.ko 2024-03-12T16:54:51.495 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/ltc4261.ko 2024-03-12T16:54:51.495 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/max16065.ko 2024-03-12T16:54:51.495 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/max1619.ko 2024-03-12T16:54:51.495 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/max1668.ko 2024-03-12T16:54:51.495 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/max197.ko 2024-03-12T16:54:51.495 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/max6639.ko 2024-03-12T16:54:51.495 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/max6650.ko 2024-03-12T16:54:51.495 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/max6697.ko 2024-03-12T16:54:51.495 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/mcp3021.ko 2024-03-12T16:54:51.495 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/nct6683.ko 2024-03-12T16:54:51.495 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/nct6775-core.ko 2024-03-12T16:54:51.495 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/nct6775.ko 2024-03-12T16:54:51.495 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/nct7802.ko 2024-03-12T16:54:51.496 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/pc87360.ko 2024-03-12T16:54:51.496 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/pc87427.ko 2024-03-12T16:54:51.496 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/pcf8591.ko 2024-03-12T16:54:51.496 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/pmbus 2024-03-12T16:54:51.496 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-12T16:54:51.496 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-12T16:54:51.496 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-12T16:54:51.496 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-12T16:54:51.496 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-12T16:54:51.496 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-12T16:54:51.496 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-12T16:54:51.496 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-12T16:54:51.496 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-12T16:54:51.497 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-12T16:54:51.497 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-12T16:54:51.497 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-12T16:54:51.497 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/powr1220.ko 2024-03-12T16:54:51.497 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/sch5627.ko 2024-03-12T16:54:51.497 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/sch5636.ko 2024-03-12T16:54:51.497 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-12T16:54:51.497 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/sht15.ko 2024-03-12T16:54:51.497 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/sht21.ko 2024-03-12T16:54:51.497 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/shtc1.ko 2024-03-12T16:54:51.497 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/sis5595.ko 2024-03-12T16:54:51.497 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/smsc47b397.ko 2024-03-12T16:54:51.497 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/smsc47m1.ko 2024-03-12T16:54:51.497 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/smsc47m192.ko 2024-03-12T16:54:51.498 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/thmc50.ko 2024-03-12T16:54:51.498 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/tmp102.ko 2024-03-12T16:54:51.498 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/tmp103.ko 2024-03-12T16:54:51.498 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/tmp401.ko 2024-03-12T16:54:51.498 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/tmp421.ko 2024-03-12T16:54:51.498 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/via-cputemp.ko 2024-03-12T16:54:51.498 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/via686a.ko 2024-03-12T16:54:51.498 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/vt1211.ko 2024-03-12T16:54:51.498 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/vt8231.ko 2024-03-12T16:54:51.498 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/w83627ehf.ko 2024-03-12T16:54:51.498 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/w83627hf.ko 2024-03-12T16:54:51.498 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/w83781d.ko 2024-03-12T16:54:51.498 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/w83791d.ko 2024-03-12T16:54:51.499 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/w83792d.ko 2024-03-12T16:54:51.499 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/w83793.ko 2024-03-12T16:54:51.499 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/w83795.ko 2024-03-12T16:54:51.499 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/w83l785ts.ko 2024-03-12T16:54:51.499 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/w83l786ng.ko 2024-03-12T16:54:51.499 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c 2024-03-12T16:54:51.499 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/algos 2024-03-12T16:54:51.499 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-12T16:54:51.499 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-12T16:54:51.499 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses 2024-03-12T16:54:51.499 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-12T16:54:51.499 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-12T16:54:51.499 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-12T16:54:51.500 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-12T16:54:51.500 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-12T16:54:51.500 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-12T16:54:51.500 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-12T16:54:51.500 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-12T16:54:51.500 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-12T16:54:51.500 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-12T16:54:51.500 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-12T16:54:51.500 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-12T16:54:51.500 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-12T16:54:51.500 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-12T16:54:51.500 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-12T16:54:51.500 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-12T16:54:51.501 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-12T16:54:51.501 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-12T16:54:51.501 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-12T16:54:51.501 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-12T16:54:51.501 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-12T16:54:51.501 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-12T16:54:51.501 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-12T16:54:51.501 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/i2c-dev.ko 2024-03-12T16:54:51.501 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/i2c-mux.ko 2024-03-12T16:54:51.501 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-12T16:54:51.501 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/i2c-smbus.ko 2024-03-12T16:54:51.501 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/i2c-stub.ko 2024-03-12T16:54:51.502 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband 2024-03-12T16:54:51.502 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/core 2024-03-12T16:54:51.502 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-12T16:54:51.502 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/core/ib_core.ko 2024-03-12T16:54:51.502 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-12T16:54:51.502 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-12T16:54:51.502 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-12T16:54:51.502 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-12T16:54:51.502 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-12T16:54:51.502 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/hw 2024-03-12T16:54:51.502 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/hw/cxgb4 2024-03-12T16:54:51.502 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-12T16:54:51.503 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/hw/mlx4 2024-03-12T16:54:51.503 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-12T16:54:51.503 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/hw/mlx5 2024-03-12T16:54:51.503 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-12T16:54:51.503 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/hw/mthca 2024-03-12T16:54:51.503 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-12T16:54:51.503 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/ulp 2024-03-12T16:54:51.503 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/ulp/ipoib 2024-03-12T16:54:51.503 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-12T16:54:51.503 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/ulp/iser 2024-03-12T16:54:51.503 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-12T16:54:51.503 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/ulp/isert 2024-03-12T16:54:51.503 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-12T16:54:51.503 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/ulp/srp 2024-03-12T16:54:51.503 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-12T16:54:51.504 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/ulp/srpt 2024-03-12T16:54:51.504 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-12T16:54:51.504 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input 2024-03-12T16:54:51.504 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/gameport 2024-03-12T16:54:51.504 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-12T16:54:51.504 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-12T16:54:51.504 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/gameport/gameport.ko 2024-03-12T16:54:51.504 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/joydev.ko 2024-03-12T16:54:51.504 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/keyboard 2024-03-12T16:54:51.504 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-12T16:54:51.504 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/mouse 2024-03-12T16:54:51.504 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/mouse/appletouch.ko 2024-03-12T16:54:51.505 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/mouse/bcm5974.ko 2024-03-12T16:54:51.505 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/mouse/cyapatp.ko 2024-03-12T16:54:51.505 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-12T16:54:51.505 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/mouse/sermouse.ko 2024-03-12T16:54:51.505 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-12T16:54:51.505 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-12T16:54:51.505 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-12T16:54:51.505 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/rmi4 2024-03-12T16:54:51.505 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-12T16:54:51.505 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/serio 2024-03-12T16:54:51.505 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/serio/altera_ps2.ko 2024-03-12T16:54:51.505 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/serio/arc_ps2.ko 2024-03-12T16:54:51.506 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-12T16:54:51.506 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/serio/serio_raw.ko 2024-03-12T16:54:51.506 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/sparse-keymap.ko 2024-03-12T16:54:51.506 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/leds 2024-03-12T16:54:51.506 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/leds/leds-blinkm.ko 2024-03-12T16:54:51.506 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/leds/leds-lm3530.ko 2024-03-12T16:54:51.506 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/leds/leds-lp3944.ko 2024-03-12T16:54:51.506 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/leds/leds-ss4200.ko 2024-03-12T16:54:51.506 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/leds/trigger 2024-03-12T16:54:51.506 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-12T16:54:51.506 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-12T16:54:51.506 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-12T16:54:51.506 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-12T16:54:51.506 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-12T16:54:51.507 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-12T16:54:51.507 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-12T16:54:51.507 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-12T16:54:51.507 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-12T16:54:51.507 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md 2024-03-12T16:54:51.507 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/bcache 2024-03-12T16:54:51.507 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/bcache/bcache.ko 2024-03-12T16:54:51.507 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/dm-bio-prison.ko 2024-03-12T16:54:51.507 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/dm-cache-smq.ko 2024-03-12T16:54:51.507 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/dm-cache.ko 2024-03-12T16:54:51.507 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/dm-crypt.ko 2024-03-12T16:54:51.507 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/dm-delay.ko 2024-03-12T16:54:51.508 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/dm-flakey.ko 2024-03-12T16:54:51.508 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/dm-log-userspace.ko 2024-03-12T16:54:51.508 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/dm-multipath.ko 2024-03-12T16:54:51.508 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/dm-queue-length.ko 2024-03-12T16:54:51.508 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/dm-raid.ko 2024-03-12T16:54:51.508 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/dm-round-robin.ko 2024-03-12T16:54:51.508 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/dm-service-time.ko 2024-03-12T16:54:51.508 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/dm-switch.ko 2024-03-12T16:54:51.508 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/dm-thin-pool.ko 2024-03-12T16:54:51.508 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/dm-verity.ko 2024-03-12T16:54:51.508 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/persistent-data 2024-03-12T16:54:51.509 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-12T16:54:51.509 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/raid0.ko 2024-03-12T16:54:51.509 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/raid1.ko 2024-03-12T16:54:51.509 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/raid10.ko 2024-03-12T16:54:51.509 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/raid456.ko 2024-03-12T16:54:51.509 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/message 2024-03-12T16:54:51.509 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/message/fusion 2024-03-12T16:54:51.509 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/message/fusion/mptbase.ko 2024-03-12T16:54:51.509 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/message/fusion/mptctl.ko 2024-03-12T16:54:51.509 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/message/fusion/mptfc.ko 2024-03-12T16:54:51.509 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/message/fusion/mptlan.ko 2024-03-12T16:54:51.509 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/message/fusion/mptsas.ko 2024-03-12T16:54:51.509 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/message/fusion/mptscsih.ko 2024-03-12T16:54:51.510 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/message/fusion/mptspi.ko 2024-03-12T16:54:51.510 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/mfd 2024-03-12T16:54:51.510 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/mfd/lpc_ich.ko 2024-03-12T16:54:51.510 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/mfd/lpc_sch.ko 2024-03-12T16:54:51.510 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/mfd/mfd-core.ko 2024-03-12T16:54:51.510 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/mfd/sm501.ko 2024-03-12T16:54:51.510 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/mfd/viperboard.ko 2024-03-12T16:54:51.510 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/mfd/vx855.ko 2024-03-12T16:54:51.510 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/mfd/wl1273-core.ko 2024-03-12T16:54:51.510 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc 2024-03-12T16:54:51.510 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/altera-stapl 2024-03-12T16:54:51.511 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-12T16:54:51.511 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/apds9802als.ko 2024-03-12T16:54:51.511 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/apds990x.ko 2024-03-12T16:54:51.511 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/bh1770glc.ko 2024-03-12T16:54:51.511 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/cb710 2024-03-12T16:54:51.511 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/cb710/cb710.ko 2024-03-12T16:54:51.511 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/eeprom 2024-03-12T16:54:51.511 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/eeprom/at24.ko 2024-03-12T16:54:51.511 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-12T16:54:51.511 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/eeprom/max6875.ko 2024-03-12T16:54:51.511 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/enclosure.ko 2024-03-12T16:54:51.511 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/hpilo.ko 2024-03-12T16:54:51.512 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/isl29003.ko 2024-03-12T16:54:51.512 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/isl29020.ko 2024-03-12T16:54:51.512 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/lis3lv02d 2024-03-12T16:54:51.512 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-12T16:54:51.512 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-12T16:54:51.512 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/mei 2024-03-12T16:54:51.512 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/mei/mei-me.ko 2024-03-12T16:54:51.512 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/mei/mei-txe.ko 2024-03-12T16:54:51.512 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/mei/mei.ko 2024-03-12T16:54:51.512 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/sgi-gru 2024-03-12T16:54:51.512 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-12T16:54:51.513 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/sgi-xp 2024-03-12T16:54:51.513 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-12T16:54:51.513 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-12T16:54:51.513 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-12T16:54:51.513 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/tifm_7xx1.ko 2024-03-12T16:54:51.513 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/tifm_core.ko 2024-03-12T16:54:51.513 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/tsl2550.ko 2024-03-12T16:54:51.513 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/vmw_balloon.ko 2024-03-12T16:54:51.513 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/vmw_vmci 2024-03-12T16:54:51.513 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-12T16:54:51.513 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net 2024-03-12T16:54:51.513 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/bonding 2024-03-12T16:54:51.514 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/bonding/bonding.ko 2024-03-12T16:54:51.514 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/dsa 2024-03-12T16:54:51.514 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/dsa/b53 2024-03-12T16:54:51.514 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-12T16:54:51.514 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-12T16:54:51.514 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-12T16:54:51.514 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/dsa/mv88e6xxx 2024-03-12T16:54:51.514 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-12T16:54:51.514 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/dummy.ko 2024-03-12T16:54:51.514 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/eql.ko 2024-03-12T16:54:51.514 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet 2024-03-12T16:54:51.514 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/3com 2024-03-12T16:54:51.514 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-12T16:54:51.515 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-12T16:54:51.515 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-12T16:54:51.515 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-12T16:54:51.515 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/8390 2024-03-12T16:54:51.515 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-12T16:54:51.515 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-12T16:54:51.515 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-12T16:54:51.515 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-12T16:54:51.515 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/adaptec 2024-03-12T16:54:51.515 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-12T16:54:51.515 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/agere 2024-03-12T16:54:51.515 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-12T16:54:51.516 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/alteon 2024-03-12T16:54:51.516 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-12T16:54:51.516 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/altera 2024-03-12T16:54:51.516 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-12T16:54:51.516 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/amd 2024-03-12T16:54:51.516 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-12T16:54:51.516 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-12T16:54:51.516 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-12T16:54:51.516 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/atheros 2024-03-12T16:54:51.516 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/atheros/alx 2024-03-12T16:54:51.516 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-12T16:54:51.516 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-12T16:54:51.517 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-12T16:54:51.517 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-12T16:54:51.517 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-12T16:54:51.517 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/atheros/atlx 2024-03-12T16:54:51.517 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-12T16:54:51.517 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-12T16:54:51.517 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/broadcom 2024-03-12T16:54:51.517 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-12T16:54:51.517 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-12T16:54:51.517 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-12T16:54:51.517 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-12T16:54:51.517 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-12T16:54:51.517 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/broadcom/genet 2024-03-12T16:54:51.517 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-12T16:54:51.518 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-12T16:54:51.518 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/brocade 2024-03-12T16:54:51.518 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/brocade/bna 2024-03-12T16:54:51.518 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-12T16:54:51.518 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/chelsio 2024-03-12T16:54:51.518 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-12T16:54:51.518 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-12T16:54:51.518 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-12T16:54:51.518 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-12T16:54:51.518 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-12T16:54:51.518 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-12T16:54:51.518 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-12T16:54:51.519 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-12T16:54:51.519 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-12T16:54:51.519 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-12T16:54:51.519 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/cisco 2024-03-12T16:54:51.519 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/cisco/enic 2024-03-12T16:54:51.519 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-12T16:54:51.519 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/dec 2024-03-12T16:54:51.519 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/dec/tulip 2024-03-12T16:54:51.519 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-12T16:54:51.519 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-12T16:54:51.519 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-12T16:54:51.519 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-12T16:54:51.519 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-12T16:54:51.519 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-12T16:54:51.520 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/dlink 2024-03-12T16:54:51.520 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-12T16:54:51.520 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-12T16:54:51.520 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/dnet.ko 2024-03-12T16:54:51.520 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/emulex 2024-03-12T16:54:51.520 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/emulex/benet 2024-03-12T16:54:51.520 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-12T16:54:51.520 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/ethoc.ko 2024-03-12T16:54:51.520 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/fealnx.ko 2024-03-12T16:54:51.520 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel 2024-03-12T16:54:51.520 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-12T16:54:51.520 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/e1000 2024-03-12T16:54:51.520 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-12T16:54:51.521 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/e1000e 2024-03-12T16:54:51.521 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-12T16:54:51.521 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/fm10k 2024-03-12T16:54:51.521 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-12T16:54:51.521 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/i40e 2024-03-12T16:54:51.521 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-12T16:54:51.521 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/iavf 2024-03-12T16:54:51.521 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-12T16:54:51.521 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/igb 2024-03-12T16:54:51.521 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-12T16:54:51.521 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/igbvf 2024-03-12T16:54:51.521 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-12T16:54:51.521 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-12T16:54:51.522 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-12T16:54:51.522 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-12T16:54:51.522 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-12T16:54:51.522 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/jme.ko 2024-03-12T16:54:51.522 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/marvell 2024-03-12T16:54:51.522 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-12T16:54:51.522 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-12T16:54:51.522 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-12T16:54:51.522 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/mellanox 2024-03-12T16:54:51.522 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-12T16:54:51.522 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-12T16:54:51.522 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-12T16:54:51.522 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-12T16:54:51.523 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-12T16:54:51.523 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-12T16:54:51.523 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/micrel 2024-03-12T16:54:51.523 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-12T16:54:51.523 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/myricom 2024-03-12T16:54:51.523 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-12T16:54:51.523 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-12T16:54:51.523 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/natsemi 2024-03-12T16:54:51.523 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-12T16:54:51.523 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-12T16:54:51.523 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/neterion 2024-03-12T16:54:51.523 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-12T16:54:51.523 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/nvidia 2024-03-12T16:54:51.523 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-12T16:54:51.524 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/packetengines 2024-03-12T16:54:51.524 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-12T16:54:51.524 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-12T16:54:51.524 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/qlogic 2024-03-12T16:54:51.524 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-12T16:54:51.524 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-12T16:54:51.524 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-12T16:54:51.524 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-12T16:54:51.524 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-12T16:54:51.524 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/rdc 2024-03-12T16:54:51.524 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-12T16:54:51.524 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/realtek 2024-03-12T16:54:51.524 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-12T16:54:51.524 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-12T16:54:51.524 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-12T16:54:51.525 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-12T16:54:51.525 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/rocker 2024-03-12T16:54:51.525 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-12T16:54:51.525 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/sfc 2024-03-12T16:54:51.525 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-12T16:54:51.525 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/silan 2024-03-12T16:54:51.525 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-12T16:54:51.525 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/sis 2024-03-12T16:54:51.525 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-12T16:54:51.525 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-12T16:54:51.525 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/smsc 2024-03-12T16:54:51.525 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-12T16:54:51.525 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-12T16:54:51.525 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-12T16:54:51.525 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-12T16:54:51.526 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/stmicro 2024-03-12T16:54:51.526 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-12T16:54:51.526 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-12T16:54:51.526 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-12T16:54:51.526 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/sun 2024-03-12T16:54:51.526 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-12T16:54:51.526 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-12T16:54:51.526 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-12T16:54:51.526 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-12T16:54:51.526 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/tehuti 2024-03-12T16:54:51.526 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-12T16:54:51.526 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/ti 2024-03-12T16:54:51.526 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-12T16:54:51.526 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/via 2024-03-12T16:54:51.527 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-12T16:54:51.527 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-12T16:54:51.527 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/wiznet 2024-03-12T16:54:51.527 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-12T16:54:51.527 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-12T16:54:51.527 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/xircom 2024-03-12T16:54:51.527 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-12T16:54:51.527 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/geneve.ko 2024-03-12T16:54:51.527 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/hyperv 2024-03-12T16:54:51.527 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-12T16:54:51.527 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ieee802154 2024-03-12T16:54:51.527 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-12T16:54:51.527 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ifb.ko 2024-03-12T16:54:51.527 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ipvlan 2024-03-12T16:54:51.527 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-12T16:54:51.527 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/macvlan.ko 2024-03-12T16:54:51.528 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/macvtap.ko 2024-03-12T16:54:51.528 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/mdio 2024-03-12T16:54:51.528 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/mdio.ko 2024-03-12T16:54:51.528 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-12T16:54:51.528 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-12T16:54:51.528 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-12T16:54:51.528 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/mii.ko 2024-03-12T16:54:51.528 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/net_failover.ko 2024-03-12T16:54:51.528 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/netconsole.ko 2024-03-12T16:54:51.528 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/nlmon.ko 2024-03-12T16:54:51.528 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/pcs 2024-03-12T16:54:51.528 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-12T16:54:51.528 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-12T16:54:51.528 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy 2024-03-12T16:54:51.529 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/amd.ko 2024-03-12T16:54:51.529 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-12T16:54:51.529 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-12T16:54:51.529 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-12T16:54:51.529 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/bcm87xx.ko 2024-03-12T16:54:51.529 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/broadcom.ko 2024-03-12T16:54:51.529 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/cicada.ko 2024-03-12T16:54:51.529 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/davicom.ko 2024-03-12T16:54:51.529 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/dp83640.ko 2024-03-12T16:54:51.529 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/et1011c.ko 2024-03-12T16:54:51.530 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/icplus.ko 2024-03-12T16:54:51.530 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/lxt.ko 2024-03-12T16:54:51.530 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/marvell.ko 2024-03-12T16:54:51.530 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/micrel.ko 2024-03-12T16:54:51.530 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/national.ko 2024-03-12T16:54:51.530 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/phylink.ko 2024-03-12T16:54:51.530 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/qsemi.ko 2024-03-12T16:54:51.530 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/realtek.ko 2024-03-12T16:54:51.530 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/smsc.ko 2024-03-12T16:54:51.530 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/ste10Xp.ko 2024-03-12T16:54:51.530 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/vitesse.ko 2024-03-12T16:54:51.530 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ppp 2024-03-12T16:54:51.531 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-12T16:54:51.531 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ppp/ppp_async.ko 2024-03-12T16:54:51.531 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-12T16:54:51.531 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-12T16:54:51.531 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-12T16:54:51.531 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-12T16:54:51.531 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ppp/pppoe.ko 2024-03-12T16:54:51.531 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ppp/pppox.ko 2024-03-12T16:54:51.531 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ppp/pptp.ko 2024-03-12T16:54:51.531 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/slip 2024-03-12T16:54:51.531 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/slip/slhc.ko 2024-03-12T16:54:51.531 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/slip/slip.ko 2024-03-12T16:54:51.532 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/sungem_phy.ko 2024-03-12T16:54:51.532 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/tap.ko 2024-03-12T16:54:51.532 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/team 2024-03-12T16:54:51.532 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/team/team.ko 2024-03-12T16:54:51.532 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-12T16:54:51.532 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-12T16:54:51.532 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-12T16:54:51.532 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/team/team_mode_random.ko 2024-03-12T16:54:51.532 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-12T16:54:51.532 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/tun.ko 2024-03-12T16:54:51.532 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/veth.ko 2024-03-12T16:54:51.532 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/virtio_net.ko 2024-03-12T16:54:51.533 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/vmxnet3 2024-03-12T16:54:51.533 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-12T16:54:51.533 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/vxlan 2024-03-12T16:54:51.533 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/vxlan/vxlan.ko 2024-03-12T16:54:51.533 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/xen-netback 2024-03-12T16:54:51.533 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-12T16:54:51.533 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/xen-netfront.ko 2024-03-12T16:54:51.533 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ntb 2024-03-12T16:54:51.533 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ntb/ntb.ko 2024-03-12T16:54:51.533 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/nvme 2024-03-12T16:54:51.533 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/nvme/host 2024-03-12T16:54:51.533 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/nvme/host/nvme-core.ko 2024-03-12T16:54:51.534 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/nvme/host/nvme.ko 2024-03-12T16:54:51.534 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/parport 2024-03-12T16:54:51.534 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/parport/parport.ko 2024-03-12T16:54:51.534 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/parport/parport_cs.ko 2024-03-12T16:54:51.534 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/parport/parport_pc.ko 2024-03-12T16:54:51.534 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/parport/parport_serial.ko 2024-03-12T16:54:51.534 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pci 2024-03-12T16:54:51.534 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pci/hotplug 2024-03-12T16:54:51.534 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-12T16:54:51.534 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pci/pcie 2024-03-12T16:54:51.534 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-12T16:54:51.534 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pci/xen-pcifront.ko 2024-03-12T16:54:51.535 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pcmcia 2024-03-12T16:54:51.535 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pcmcia/i82092.ko 2024-03-12T16:54:51.535 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pcmcia/pd6729.ko 2024-03-12T16:54:51.535 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-12T16:54:51.535 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pinctrl 2024-03-12T16:54:51.535 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pinctrl/intel 2024-03-12T16:54:51.535 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-12T16:54:51.535 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform 2024-03-12T16:54:51.535 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86 2024-03-12T16:54:51.535 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-12T16:54:51.535 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/acerhdf.ko 2024-03-12T16:54:51.535 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-12T16:54:51.535 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-12T16:54:51.536 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-12T16:54:51.536 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-12T16:54:51.536 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-12T16:54:51.536 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-12T16:54:51.536 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-12T16:54:51.536 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-12T16:54:51.536 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-12T16:54:51.536 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/hdaps.ko 2024-03-12T16:54:51.536 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/intel 2024-03-12T16:54:51.536 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-12T16:54:51.536 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/intel_ips.ko 2024-03-12T16:54:51.536 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-12T16:54:51.536 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-12T16:54:51.536 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-12T16:54:51.536 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-12T16:54:51.537 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-12T16:54:51.537 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-12T16:54:51.537 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-12T16:54:51.537 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-12T16:54:51.537 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-12T16:54:51.537 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/wmi.ko 2024-03-12T16:54:51.537 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/powercap 2024-03-12T16:54:51.537 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-12T16:54:51.537 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-12T16:54:51.537 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pps 2024-03-12T16:54:51.537 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pps/clients 2024-03-12T16:54:51.537 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-12T16:54:51.537 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-12T16:54:51.537 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pps/clients/pps_parport.ko 2024-03-12T16:54:51.538 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pps/pps_core.ko 2024-03-12T16:54:51.538 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ptp 2024-03-12T16:54:51.538 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ptp/ptp.ko 2024-03-12T16:54:51.538 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ptp/ptp_kvm.ko 2024-03-12T16:54:51.538 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc 2024-03-12T16:54:51.538 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-12T16:54:51.538 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-12T16:54:51.538 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-12T16:54:51.538 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-12T16:54:51.538 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-12T16:54:51.538 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-12T16:54:51.538 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-12T16:54:51.538 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-12T16:54:51.538 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-12T16:54:51.539 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-12T16:54:51.539 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-em3027.ko 2024-03-12T16:54:51.539 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-12T16:54:51.539 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-12T16:54:51.539 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-12T16:54:51.539 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-12T16:54:51.539 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-12T16:54:51.539 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-12T16:54:51.539 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-max6900.ko 2024-03-12T16:54:51.539 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-12T16:54:51.539 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-12T16:54:51.539 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-12T16:54:51.539 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-12T16:54:51.539 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-12T16:54:51.540 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-12T16:54:51.540 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-12T16:54:51.542 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-12T16:54:51.542 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-12T16:54:51.542 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-12T16:54:51.542 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-12T16:54:51.542 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-12T16:54:51.542 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-x1205.ko 2024-03-12T16:54:51.542 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi 2024-03-12T16:54:51.542 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/3w-9xxx.ko 2024-03-12T16:54:51.542 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/3w-sas.ko 2024-03-12T16:54:51.542 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/3w-xxxx.ko 2024-03-12T16:54:51.542 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/BusLogic.ko 2024-03-12T16:54:51.543 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/a100u2w.ko 2024-03-12T16:54:51.543 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/aacraid 2024-03-12T16:54:51.543 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-12T16:54:51.543 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/advansys.ko 2024-03-12T16:54:51.543 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/aic7xxx 2024-03-12T16:54:51.543 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-12T16:54:51.543 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-12T16:54:51.543 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/am53c974.ko 2024-03-12T16:54:51.543 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/arcmsr 2024-03-12T16:54:51.543 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-12T16:54:51.543 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/atp870u.ko 2024-03-12T16:54:51.543 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/be2iscsi 2024-03-12T16:54:51.543 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-12T16:54:51.543 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/bfa 2024-03-12T16:54:51.544 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/bfa/bfa.ko 2024-03-12T16:54:51.544 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/bnx2fc 2024-03-12T16:54:51.544 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-12T16:54:51.544 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/bnx2i 2024-03-12T16:54:51.544 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-12T16:54:51.544 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/ch.ko 2024-03-12T16:54:51.544 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/csiostor 2024-03-12T16:54:51.544 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-12T16:54:51.544 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/cxgbi 2024-03-12T16:54:51.544 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-12T16:54:51.544 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-12T16:54:51.544 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-12T16:54:51.544 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-12T16:54:51.545 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-12T16:54:51.545 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/dc395x.ko 2024-03-12T16:54:51.545 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/device_handler 2024-03-12T16:54:51.545 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-12T16:54:51.545 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-12T16:54:51.545 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-12T16:54:51.545 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-12T16:54:51.545 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/dmx3191d.ko 2024-03-12T16:54:51.545 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/esas2r 2024-03-12T16:54:51.545 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-12T16:54:51.545 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/esp_scsi.ko 2024-03-12T16:54:51.545 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/fcoe 2024-03-12T16:54:51.545 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-12T16:54:51.546 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-12T16:54:51.546 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/fnic 2024-03-12T16:54:51.546 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/fnic/fnic.ko 2024-03-12T16:54:51.546 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/hpsa.ko 2024-03-12T16:54:51.546 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/hptiop.ko 2024-03-12T16:54:51.546 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/hv_storvsc.ko 2024-03-12T16:54:51.546 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/initio.ko 2024-03-12T16:54:51.546 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/ipr.ko 2024-03-12T16:54:51.546 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/ips.ko 2024-03-12T16:54:51.546 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/isci 2024-03-12T16:54:51.546 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/isci/isci.ko 2024-03-12T16:54:51.546 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-12T16:54:51.547 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-12T16:54:51.547 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/libfc 2024-03-12T16:54:51.547 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/libfc/libfc.ko 2024-03-12T16:54:51.547 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/libiscsi.ko 2024-03-12T16:54:51.547 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-12T16:54:51.547 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/libsas 2024-03-12T16:54:51.547 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/libsas/libsas.ko 2024-03-12T16:54:51.547 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/lpfc 2024-03-12T16:54:51.547 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-12T16:54:51.547 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/megaraid 2024-03-12T16:54:51.547 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/megaraid.ko 2024-03-12T16:54:51.547 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-12T16:54:51.547 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-12T16:54:51.547 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-12T16:54:51.548 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/mpt3sas 2024-03-12T16:54:51.548 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-12T16:54:51.548 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/mvsas 2024-03-12T16:54:51.548 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-12T16:54:51.548 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/mvumi.ko 2024-03-12T16:54:51.548 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/pm8001 2024-03-12T16:54:51.548 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-12T16:54:51.548 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/pmcraid.ko 2024-03-12T16:54:51.548 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/qla1280.ko 2024-03-12T16:54:51.548 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/qla2xxx 2024-03-12T16:54:51.548 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-12T16:54:51.548 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-12T16:54:51.549 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/qla4xxx 2024-03-12T16:54:51.549 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-12T16:54:51.549 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/raid_class.ko 2024-03-12T16:54:51.549 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/scsi_debug.ko 2024-03-12T16:54:51.549 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-12T16:54:51.549 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-12T16:54:51.549 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-12T16:54:51.549 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-12T16:54:51.549 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-12T16:54:51.549 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/ses.ko 2024-03-12T16:54:51.549 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/st.ko 2024-03-12T16:54:51.549 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/stex.ko 2024-03-12T16:54:51.549 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/sym53c8xx_2 2024-03-12T16:54:51.550 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-12T16:54:51.550 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/virtio_scsi.ko 2024-03-12T16:54:51.550 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-12T16:54:51.550 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/wd719x.ko 2024-03-12T16:54:51.550 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ssb 2024-03-12T16:54:51.550 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ssb/ssb.ko 2024-03-12T16:54:51.550 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/target 2024-03-12T16:54:51.550 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/target/iscsi 2024-03-12T16:54:51.550 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-12T16:54:51.550 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/target/loopback 2024-03-12T16:54:51.550 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-12T16:54:51.550 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/target/target_core_file.ko 2024-03-12T16:54:51.551 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/target/target_core_iblock.ko 2024-03-12T16:54:51.551 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/target/target_core_mod.ko 2024-03-12T16:54:51.551 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/target/target_core_pscsi.ko 2024-03-12T16:54:51.551 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/target/target_core_user.ko 2024-03-12T16:54:51.551 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/target/tcm_fc 2024-03-12T16:54:51.551 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-12T16:54:51.551 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal 2024-03-12T16:54:51.551 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel 2024-03-12T16:54:51.551 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/int340x_thermal 2024-03-12T16:54:51.551 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-12T16:54:51.551 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-12T16:54:51.551 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-12T16:54:51.551 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-12T16:54:51.551 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-12T16:54:51.551 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-12T16:54:51.552 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-12T16:54:51.552 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-12T16:54:51.552 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-12T16:54:51.552 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-12T16:54:51.552 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-12T16:54:51.552 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-12T16:54:51.552 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-12T16:54:51.552 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-12T16:54:51.552 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-12T16:54:51.552 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-12T16:54:51.552 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-12T16:54:51.552 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-12T16:54:51.552 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/tty 2024-03-12T16:54:51.552 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/tty/ipwireless 2024-03-12T16:54:51.553 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-12T16:54:51.553 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/tty/n_gsm.ko 2024-03-12T16:54:51.553 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/tty/n_hdlc.ko 2024-03-12T16:54:51.553 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/tty/nozomi.ko 2024-03-12T16:54:51.553 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/tty/serial 2024-03-12T16:54:51.553 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/tty/serial/8250 2024-03-12T16:54:51.553 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-12T16:54:51.553 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/tty/serial/arc_uart.ko 2024-03-12T16:54:51.553 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/tty/serial/jsm 2024-03-12T16:54:51.553 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-12T16:54:51.553 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ufs 2024-03-12T16:54:51.553 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ufs/core 2024-03-12T16:54:51.553 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-12T16:54:51.554 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ufs/host 2024-03-12T16:54:51.554 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-12T16:54:51.554 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/uio 2024-03-12T16:54:51.554 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/uio/uio.ko 2024-03-12T16:54:51.554 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/uio/uio_aec.ko 2024-03-12T16:54:51.554 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/uio/uio_cif.ko 2024-03-12T16:54:51.554 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/uio/uio_pci_generic.ko 2024-03-12T16:54:51.554 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/uio/uio_sercos3.ko 2024-03-12T16:54:51.554 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb 2024-03-12T16:54:51.554 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/class 2024-03-12T16:54:51.554 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/class/cdc-acm.ko 2024-03-12T16:54:51.554 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-12T16:54:51.555 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/class/usblp.ko 2024-03-12T16:54:51.555 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/class/usbtmc.ko 2024-03-12T16:54:51.555 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/host 2024-03-12T16:54:51.555 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-12T16:54:51.555 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/image 2024-03-12T16:54:51.555 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/image/mdc800.ko 2024-03-12T16:54:51.555 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/image/microtek.ko 2024-03-12T16:54:51.555 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc 2024-03-12T16:54:51.555 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/adutux.ko 2024-03-12T16:54:51.555 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/appledisplay.ko 2024-03-12T16:54:51.555 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/emi26.ko 2024-03-12T16:54:51.555 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/emi62.ko 2024-03-12T16:54:51.555 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/ezusb.ko 2024-03-12T16:54:51.556 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/idmouse.ko 2024-03-12T16:54:51.556 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/iowarrior.ko 2024-03-12T16:54:51.556 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-12T16:54:51.556 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/ldusb.ko 2024-03-12T16:54:51.556 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/legousbtower.ko 2024-03-12T16:54:51.556 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/sisusbvga 2024-03-12T16:54:51.556 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-12T16:54:51.556 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-12T16:54:51.556 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/usb3503.ko 2024-03-12T16:54:51.556 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/usblcd.ko 2024-03-12T16:54:51.556 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-12T16:54:51.556 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/uss720.ko 2024-03-12T16:54:51.556 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/yurex.ko 2024-03-12T16:54:51.557 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/phy 2024-03-12T16:54:51.557 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/phy/phy-generic.ko 2024-03-12T16:54:51.557 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial 2024-03-12T16:54:51.557 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/aircable.ko 2024-03-12T16:54:51.557 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/ark3116.ko 2024-03-12T16:54:51.557 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-12T16:54:51.557 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/ch341.ko 2024-03-12T16:54:51.557 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/cp210x.ko 2024-03-12T16:54:51.557 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/cyberjack.ko 2024-03-12T16:54:51.557 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-12T16:54:51.557 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-12T16:54:51.557 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/empeg.ko 2024-03-12T16:54:51.557 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-12T16:54:51.558 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-12T16:54:51.558 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-12T16:54:51.558 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/io_ti.ko 2024-03-12T16:54:51.558 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/ipaq.ko 2024-03-12T16:54:51.558 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/ipw.ko 2024-03-12T16:54:51.558 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/ir-usb.ko 2024-03-12T16:54:51.558 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-12T16:54:51.558 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/keyspan.ko 2024-03-12T16:54:51.558 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-12T16:54:51.558 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-12T16:54:51.558 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-12T16:54:51.558 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/mct_u232.ko 2024-03-12T16:54:51.558 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/mos7720.ko 2024-03-12T16:54:51.559 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/mos7840.ko 2024-03-12T16:54:51.559 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/navman.ko 2024-03-12T16:54:51.559 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/omninet.ko 2024-03-12T16:54:51.559 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/opticon.ko 2024-03-12T16:54:51.559 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/option.ko 2024-03-12T16:54:51.559 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/oti6858.ko 2024-03-12T16:54:51.559 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/pl2303.ko 2024-03-12T16:54:51.559 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/qcaux.ko 2024-03-12T16:54:51.559 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/qcserial.ko 2024-03-12T16:54:51.559 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/quatech2.ko 2024-03-12T16:54:51.559 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/safe_serial.ko 2024-03-12T16:54:51.559 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/sierra.ko 2024-03-12T16:54:51.559 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-12T16:54:51.559 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/ssu100.ko 2024-03-12T16:54:51.560 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/symbolserial.ko 2024-03-12T16:54:51.560 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-12T16:54:51.560 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-12T16:54:51.560 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/usb_debug.ko 2024-03-12T16:54:51.560 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-12T16:54:51.560 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/visor.ko 2024-03-12T16:54:51.560 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/whiteheat.ko 2024-03-12T16:54:51.560 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-12T16:54:51.560 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/storage 2024-03-12T16:54:51.560 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/storage/uas.ko 2024-03-12T16:54:51.560 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-12T16:54:51.560 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-12T16:54:51.560 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-12T16:54:51.560 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-12T16:54:51.561 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-12T16:54:51.561 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-12T16:54:51.561 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-12T16:54:51.561 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/storage/ums-karma.ko 2024-03-12T16:54:51.561 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-12T16:54:51.561 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-12T16:54:51.561 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-12T16:54:51.561 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-12T16:54:51.561 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-12T16:54:51.561 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/storage/usb-storage.ko 2024-03-12T16:54:51.561 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/usbip 2024-03-12T16:54:51.561 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-12T16:54:51.561 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-12T16:54:51.561 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-12T16:54:51.562 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/vfio 2024-03-12T16:54:51.562 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/vfio/pci 2024-03-12T16:54:51.562 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-12T16:54:51.562 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-12T16:54:51.562 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/vfio/vfio.ko 2024-03-12T16:54:51.562 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-12T16:54:51.562 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/vhost 2024-03-12T16:54:51.562 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/vhost/vhost.ko 2024-03-12T16:54:51.562 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-12T16:54:51.562 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/vhost/vhost_net.ko 2024-03-12T16:54:51.562 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/vhost/vhost_scsi.ko 2024-03-12T16:54:51.562 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/video 2024-03-12T16:54:51.562 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/video/backlight 2024-03-12T16:54:51.563 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/video/backlight/apple_bl.ko 2024-03-12T16:54:51.563 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/video/backlight/lcd.ko 2024-03-12T16:54:51.563 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-12T16:54:51.563 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/virtio 2024-03-12T16:54:51.563 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/virtio/virtio.ko 2024-03-12T16:54:51.563 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/virtio/virtio_balloon.ko 2024-03-12T16:54:51.563 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/virtio/virtio_mmio.ko 2024-03-12T16:54:51.563 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/virtio/virtio_pci.ko 2024-03-12T16:54:51.563 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-12T16:54:51.563 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-12T16:54:51.563 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/virtio/virtio_ring.ko 2024-03-12T16:54:51.563 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/w1 2024-03-12T16:54:51.563 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/w1/masters 2024-03-12T16:54:51.563 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/w1/masters/ds2482.ko 2024-03-12T16:54:51.564 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/w1/masters/ds2490.ko 2024-03-12T16:54:51.564 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/w1/slaves 2024-03-12T16:54:51.564 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-12T16:54:51.564 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-12T16:54:51.564 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-12T16:54:51.564 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-12T16:54:51.564 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-12T16:54:51.564 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-12T16:54:51.564 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-12T16:54:51.564 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-12T16:54:51.564 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-12T16:54:51.564 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-12T16:54:51.564 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-12T16:54:51.565 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/w1/wire.ko 2024-03-12T16:54:51.565 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog 2024-03-12T16:54:51.565 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-12T16:54:51.565 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-12T16:54:51.565 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-12T16:54:51.565 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/hpwdt.ko 2024-03-12T16:54:51.565 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/i6300esb.ko 2024-03-12T16:54:51.565 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-12T16:54:51.565 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-12T16:54:51.565 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/ib700wdt.ko 2024-03-12T16:54:51.565 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/ibmasr.ko 2024-03-12T16:54:51.565 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-12T16:54:51.565 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-12T16:54:51.566 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/it87_wdt.ko 2024-03-12T16:54:51.566 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/machzwd.ko 2024-03-12T16:54:51.566 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/nv_tco.ko 2024-03-12T16:54:51.566 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-12T16:54:51.566 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-12T16:54:51.566 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-12T16:54:51.566 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-12T16:54:51.566 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/softdog.ko 2024-03-12T16:54:51.566 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-12T16:54:51.566 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/via_wdt.ko 2024-03-12T16:54:51.566 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-12T16:54:51.566 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-12T16:54:51.566 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-12T16:54:51.566 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/wdt_pci.ko 2024-03-12T16:54:51.567 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/xen_wdt.ko 2024-03-12T16:54:51.567 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/xen 2024-03-12T16:54:51.567 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-12T16:54:51.567 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/xen/xen-evtchn.ko 2024-03-12T16:54:51.567 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/xen/xen-gntalloc.ko 2024-03-12T16:54:51.567 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/xen/xen-gntdev.ko 2024-03-12T16:54:51.567 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/xen/xen-pciback 2024-03-12T16:54:51.567 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-12T16:54:51.567 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/xen/xen-privcmd.ko 2024-03-12T16:54:51.567 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/xen/xenfs 2024-03-12T16:54:51.567 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-12T16:54:51.567 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs 2024-03-12T16:54:51.567 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/9p 2024-03-12T16:54:51.567 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/9p/9p.ko 2024-03-12T16:54:51.567 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/affs 2024-03-12T16:54:51.568 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/affs/affs.ko 2024-03-12T16:54:51.568 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/befs 2024-03-12T16:54:51.568 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/befs/befs.ko 2024-03-12T16:54:51.568 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/binfmt_misc.ko 2024-03-12T16:54:51.568 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/btrfs 2024-03-12T16:54:51.568 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/btrfs/btrfs.ko 2024-03-12T16:54:51.568 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/cachefiles 2024-03-12T16:54:51.568 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/cachefiles/cachefiles.ko 2024-03-12T16:54:51.568 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/ceph 2024-03-12T16:54:51.568 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/ceph/ceph.ko 2024-03-12T16:54:51.568 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/coda 2024-03-12T16:54:51.568 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/coda/coda.ko 2024-03-12T16:54:51.568 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/cramfs 2024-03-12T16:54:51.568 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/cramfs/cramfs.ko 2024-03-12T16:54:51.568 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/dlm 2024-03-12T16:54:51.569 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/dlm/dlm.ko 2024-03-12T16:54:51.569 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/ecryptfs 2024-03-12T16:54:51.569 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-12T16:54:51.569 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/f2fs 2024-03-12T16:54:51.569 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/f2fs/f2fs.ko 2024-03-12T16:54:51.569 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/fat 2024-03-12T16:54:51.569 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/fat/fat.ko 2024-03-12T16:54:51.569 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/fat/msdos.ko 2024-03-12T16:54:51.569 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/fat/vfat.ko 2024-03-12T16:54:51.569 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/fuse 2024-03-12T16:54:51.569 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/fuse/cuse.ko 2024-03-12T16:54:51.569 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/fuse/fuse.ko 2024-03-12T16:54:51.569 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/gfs2 2024-03-12T16:54:51.570 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/gfs2/gfs2.ko 2024-03-12T16:54:51.570 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/hfs 2024-03-12T16:54:51.570 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/hfs/hfs.ko 2024-03-12T16:54:51.570 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/hfsplus 2024-03-12T16:54:51.570 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/hfsplus/hfsplus.ko 2024-03-12T16:54:51.570 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/isofs 2024-03-12T16:54:51.570 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/isofs/isofs.ko 2024-03-12T16:54:51.570 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/jfs 2024-03-12T16:54:51.570 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/jfs/jfs.ko 2024-03-12T16:54:51.570 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/lockd 2024-03-12T16:54:51.570 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/lockd/lockd.ko 2024-03-12T16:54:51.570 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/minix 2024-03-12T16:54:51.570 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/minix/minix.ko 2024-03-12T16:54:51.570 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/netfs 2024-03-12T16:54:51.571 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/netfs/netfs.ko 2024-03-12T16:54:51.571 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nfs 2024-03-12T16:54:51.571 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nfs/blocklayout 2024-03-12T16:54:51.571 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-12T16:54:51.571 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nfs/filelayout 2024-03-12T16:54:51.571 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-12T16:54:51.571 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nfs/flexfilelayout 2024-03-12T16:54:51.571 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-12T16:54:51.571 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nfs/nfs.ko 2024-03-12T16:54:51.571 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nfs/nfsv3.ko 2024-03-12T16:54:51.571 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nfs/nfsv4.ko 2024-03-12T16:54:51.571 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nfs_common 2024-03-12T16:54:51.571 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nfs_common/grace.ko 2024-03-12T16:54:51.571 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nfs_common/nfs_acl.ko 2024-03-12T16:54:51.571 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nfsd 2024-03-12T16:54:51.572 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nfsd/nfsd.ko 2024-03-12T16:54:51.572 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nilfs2 2024-03-12T16:54:51.572 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nilfs2/nilfs2.ko 2024-03-12T16:54:51.572 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls 2024-03-12T16:54:51.572 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/mac-celtic.ko 2024-03-12T16:54:51.572 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/mac-centeuro.ko 2024-03-12T16:54:51.572 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/mac-croatian.ko 2024-03-12T16:54:51.572 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/mac-cyrillic.ko 2024-03-12T16:54:51.572 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/mac-gaelic.ko 2024-03-12T16:54:51.572 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/mac-greek.ko 2024-03-12T16:54:51.572 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/mac-iceland.ko 2024-03-12T16:54:51.572 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/mac-inuit.ko 2024-03-12T16:54:51.572 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/mac-roman.ko 2024-03-12T16:54:51.573 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/mac-romanian.ko 2024-03-12T16:54:51.573 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/mac-turkish.ko 2024-03-12T16:54:51.573 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp1250.ko 2024-03-12T16:54:51.573 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp1251.ko 2024-03-12T16:54:51.573 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp1255.ko 2024-03-12T16:54:51.573 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp737.ko 2024-03-12T16:54:51.573 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp775.ko 2024-03-12T16:54:51.573 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp850.ko 2024-03-12T16:54:51.573 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp852.ko 2024-03-12T16:54:51.573 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp855.ko 2024-03-12T16:54:51.573 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp857.ko 2024-03-12T16:54:51.573 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp860.ko 2024-03-12T16:54:51.573 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp861.ko 2024-03-12T16:54:51.573 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp862.ko 2024-03-12T16:54:51.574 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp863.ko 2024-03-12T16:54:51.574 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp864.ko 2024-03-12T16:54:51.574 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp865.ko 2024-03-12T16:54:51.574 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp866.ko 2024-03-12T16:54:51.574 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp869.ko 2024-03-12T16:54:51.574 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp874.ko 2024-03-12T16:54:51.574 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp932.ko 2024-03-12T16:54:51.574 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp936.ko 2024-03-12T16:54:51.574 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp949.ko 2024-03-12T16:54:51.574 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp950.ko 2024-03-12T16:54:51.574 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_euc-jp.ko 2024-03-12T16:54:51.574 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_iso8859-1.ko 2024-03-12T16:54:51.574 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_iso8859-13.ko 2024-03-12T16:54:51.575 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_iso8859-14.ko 2024-03-12T16:54:51.575 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_iso8859-15.ko 2024-03-12T16:54:51.575 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_iso8859-2.ko 2024-03-12T16:54:51.575 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_iso8859-3.ko 2024-03-12T16:54:51.575 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_iso8859-4.ko 2024-03-12T16:54:51.575 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_iso8859-5.ko 2024-03-12T16:54:51.575 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_iso8859-6.ko 2024-03-12T16:54:51.575 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_iso8859-7.ko 2024-03-12T16:54:51.575 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_iso8859-9.ko 2024-03-12T16:54:51.575 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_koi8-r.ko 2024-03-12T16:54:51.575 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_koi8-ru.ko 2024-03-12T16:54:51.575 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_koi8-u.ko 2024-03-12T16:54:51.575 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-12T16:54:51.576 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_utf8.ko 2024-03-12T16:54:51.576 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/ocfs2 2024-03-12T16:54:51.576 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/ocfs2/cluster 2024-03-12T16:54:51.576 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-12T16:54:51.576 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/ocfs2/dlm 2024-03-12T16:54:51.576 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-12T16:54:51.576 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/ocfs2/dlmfs 2024-03-12T16:54:51.576 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-12T16:54:51.576 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/ocfs2/ocfs2.ko 2024-03-12T16:54:51.576 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-12T16:54:51.576 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-12T16:54:51.576 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-12T16:54:51.576 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/overlayfs 2024-03-12T16:54:51.576 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/overlayfs/overlay.ko 2024-03-12T16:54:51.577 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/pstore 2024-03-12T16:54:51.577 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/pstore/ramoops.ko 2024-03-12T16:54:51.577 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/reiserfs 2024-03-12T16:54:51.577 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/reiserfs/reiserfs.ko 2024-03-12T16:54:51.577 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/romfs 2024-03-12T16:54:51.577 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/romfs/romfs.ko 2024-03-12T16:54:51.577 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/smb 2024-03-12T16:54:51.577 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/smb/client 2024-03-12T16:54:51.577 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/smb/client/cifs.ko 2024-03-12T16:54:51.577 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/smb/common 2024-03-12T16:54:51.577 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/smb/common/cifs_arc4.ko 2024-03-12T16:54:51.577 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/smb/common/cifs_md4.ko 2024-03-12T16:54:51.577 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/squashfs 2024-03-12T16:54:51.577 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/squashfs/squashfs.ko 2024-03-12T16:54:51.577 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/sysv 2024-03-12T16:54:51.578 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/sysv/sysv.ko 2024-03-12T16:54:51.578 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/udf 2024-03-12T16:54:51.578 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/udf/udf.ko 2024-03-12T16:54:51.578 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/ufs 2024-03-12T16:54:51.578 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/ufs/ufs.ko 2024-03-12T16:54:51.578 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/xfs 2024-03-12T16:54:51.578 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/xfs/xfs.ko 2024-03-12T16:54:51.578 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/kernel 2024-03-12T16:54:51.578 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/kernel/rcu 2024-03-12T16:54:51.578 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/kernel/rcu/rcutorture.ko 2024-03-12T16:54:51.578 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/kernel/torture.ko 2024-03-12T16:54:51.578 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/kernel/trace 2024-03-12T16:54:51.578 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-12T16:54:51.578 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib 2024-03-12T16:54:51.578 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/asn1_encoder.ko 2024-03-12T16:54:51.579 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/crc-itu-t.ko 2024-03-12T16:54:51.579 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/crc8.ko 2024-03-12T16:54:51.579 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/crypto 2024-03-12T16:54:51.579 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/crypto/libarc4.ko 2024-03-12T16:54:51.579 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/crypto/libdes.ko 2024-03-12T16:54:51.579 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/libcrc32c.ko 2024-03-12T16:54:51.579 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/lru_cache.ko 2024-03-12T16:54:51.579 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/lz4 2024-03-12T16:54:51.579 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/lz4/lz4_compress.ko 2024-03-12T16:54:51.579 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/lz4/lz4hc_compress.ko 2024-03-12T16:54:51.579 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/math 2024-03-12T16:54:51.579 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/math/cordic.ko 2024-03-12T16:54:51.579 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/raid6 2024-03-12T16:54:51.579 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/raid6/raid6_pq.ko 2024-03-12T16:54:51.579 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/reed_solomon 2024-03-12T16:54:51.580 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-12T16:54:51.580 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/ts_bm.ko 2024-03-12T16:54:51.580 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/ts_fsm.ko 2024-03-12T16:54:51.580 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/ts_kmp.ko 2024-03-12T16:54:51.580 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/zstd 2024-03-12T16:54:51.580 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/zstd/zstd_compress.ko 2024-03-12T16:54:51.580 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/mm 2024-03-12T16:54:51.580 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/mm/hwpoison-inject.ko 2024-03-12T16:54:51.580 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net 2024-03-12T16:54:51.580 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/6lowpan 2024-03-12T16:54:51.580 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/6lowpan/6lowpan.ko 2024-03-12T16:54:51.580 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/6lowpan/nhc_dest.ko 2024-03-12T16:54:51.580 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/6lowpan/nhc_fragment.ko 2024-03-12T16:54:51.580 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/6lowpan/nhc_hop.ko 2024-03-12T16:54:51.580 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-12T16:54:51.581 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/6lowpan/nhc_mobility.ko 2024-03-12T16:54:51.581 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/6lowpan/nhc_routing.ko 2024-03-12T16:54:51.581 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/6lowpan/nhc_udp.ko 2024-03-12T16:54:51.581 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/802 2024-03-12T16:54:51.581 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/802/garp.ko 2024-03-12T16:54:51.581 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/802/mrp.ko 2024-03-12T16:54:51.581 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/802/p8022.ko 2024-03-12T16:54:51.581 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/802/psnap.ko 2024-03-12T16:54:51.581 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/802/stp.ko 2024-03-12T16:54:51.581 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/8021q 2024-03-12T16:54:51.581 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/8021q/8021q.ko 2024-03-12T16:54:51.581 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/9p 2024-03-12T16:54:51.581 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/9p/9pnet.ko 2024-03-12T16:54:51.581 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/9p/9pnet_fd.ko 2024-03-12T16:54:51.581 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/9p/9pnet_rdma.ko 2024-03-12T16:54:51.581 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/9p/9pnet_virtio.ko 2024-03-12T16:54:51.582 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/batman-adv 2024-03-12T16:54:51.582 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/batman-adv/batman-adv.ko 2024-03-12T16:54:51.582 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge 2024-03-12T16:54:51.582 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/br_netfilter.ko 2024-03-12T16:54:51.582 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/bridge.ko 2024-03-12T16:54:51.582 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter 2024-03-12T16:54:51.582 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-12T16:54:51.582 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-12T16:54:51.582 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-12T16:54:51.582 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-12T16:54:51.582 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-12T16:54:51.582 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-12T16:54:51.582 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-12T16:54:51.583 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-12T16:54:51.583 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-12T16:54:51.583 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-12T16:54:51.583 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-12T16:54:51.583 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-12T16:54:51.583 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-12T16:54:51.583 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-12T16:54:51.583 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-12T16:54:51.583 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-12T16:54:51.583 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-12T16:54:51.583 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-12T16:54:51.583 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-12T16:54:51.583 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-12T16:54:51.583 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebtables.ko 2024-03-12T16:54:51.583 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-12T16:54:51.584 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-12T16:54:51.584 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-12T16:54:51.584 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ceph 2024-03-12T16:54:51.584 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ceph/libceph.ko 2024-03-12T16:54:51.584 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/core 2024-03-12T16:54:51.584 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/core/failover.ko 2024-03-12T16:54:51.584 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/core/pktgen.ko 2024-03-12T16:54:51.584 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/dccp 2024-03-12T16:54:51.584 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/dccp/dccp.ko 2024-03-12T16:54:51.584 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/dccp/dccp_diag.ko 2024-03-12T16:54:51.584 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/dccp/dccp_ipv4.ko 2024-03-12T16:54:51.584 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/dccp/dccp_ipv6.ko 2024-03-12T16:54:51.584 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/dns_resolver 2024-03-12T16:54:51.584 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/dns_resolver/dns_resolver.ko 2024-03-12T16:54:51.585 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/dsa 2024-03-12T16:54:51.585 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/dsa/dsa_core.ko 2024-03-12T16:54:51.585 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/dsa/tag_brcm.ko 2024-03-12T16:54:51.585 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/dsa/tag_dsa.ko 2024-03-12T16:54:51.585 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/dsa/tag_none.ko 2024-03-12T16:54:51.585 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/dsa/tag_trailer.ko 2024-03-12T16:54:51.585 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ieee802154 2024-03-12T16:54:51.585 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ieee802154/6lowpan 2024-03-12T16:54:51.585 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-12T16:54:51.585 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ieee802154/ieee802154.ko 2024-03-12T16:54:51.585 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-12T16:54:51.585 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4 2024-03-12T16:54:51.585 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/ah4.ko 2024-03-12T16:54:51.585 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/esp4.ko 2024-03-12T16:54:51.585 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/fou.ko 2024-03-12T16:54:51.586 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/gre.ko 2024-03-12T16:54:51.586 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/inet_diag.ko 2024-03-12T16:54:51.586 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/ip_gre.ko 2024-03-12T16:54:51.586 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/ip_tunnel.ko 2024-03-12T16:54:51.586 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/ip_vti.ko 2024-03-12T16:54:51.586 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/ipcomp.ko 2024-03-12T16:54:51.586 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/ipip.ko 2024-03-12T16:54:51.586 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter 2024-03-12T16:54:51.586 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-12T16:54:51.586 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-12T16:54:51.586 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-12T16:54:51.586 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-12T16:54:51.587 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-12T16:54:51.587 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-12T16:54:51.587 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-12T16:54:51.587 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-12T16:54:51.587 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-12T16:54:51.587 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-12T16:54:51.587 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-12T16:54:51.587 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-12T16:54:51.587 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-12T16:54:51.587 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-12T16:54:51.587 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-12T16:54:51.587 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-12T16:54:51.587 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-12T16:54:51.587 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-12T16:54:51.587 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-12T16:54:51.588 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-12T16:54:51.588 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-12T16:54:51.588 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/tcp_bic.ko 2024-03-12T16:54:51.588 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/tcp_dctcp.ko 2024-03-12T16:54:51.588 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/tcp_diag.ko 2024-03-12T16:54:51.588 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/tcp_highspeed.ko 2024-03-12T16:54:51.588 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/tcp_htcp.ko 2024-03-12T16:54:51.588 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/tcp_hybla.ko 2024-03-12T16:54:51.588 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/tcp_illinois.ko 2024-03-12T16:54:51.588 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/tcp_lp.ko 2024-03-12T16:54:51.588 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/tcp_scalable.ko 2024-03-12T16:54:51.588 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/tcp_vegas.ko 2024-03-12T16:54:51.588 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/tcp_veno.ko 2024-03-12T16:54:51.588 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/tcp_westwood.ko 2024-03-12T16:54:51.589 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/tcp_yeah.ko 2024-03-12T16:54:51.589 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/tunnel4.ko 2024-03-12T16:54:51.589 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/udp_diag.ko 2024-03-12T16:54:51.589 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/udp_tunnel.ko 2024-03-12T16:54:51.589 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-12T16:54:51.589 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6 2024-03-12T16:54:51.589 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/ah6.ko 2024-03-12T16:54:51.589 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/esp6.ko 2024-03-12T16:54:51.589 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/fou6.ko 2024-03-12T16:54:51.589 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/ip6_tunnel.ko 2024-03-12T16:54:51.589 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-12T16:54:51.589 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/ip6_vti.ko 2024-03-12T16:54:51.589 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/ipcomp6.ko 2024-03-12T16:54:51.589 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter 2024-03-12T16:54:51.589 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-12T16:54:51.590 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-12T16:54:51.590 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-12T16:54:51.590 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-12T16:54:51.590 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-12T16:54:51.590 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-12T16:54:51.590 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-12T16:54:51.590 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-12T16:54:51.590 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-12T16:54:51.590 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-12T16:54:51.590 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-12T16:54:51.590 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-12T16:54:51.590 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-12T16:54:51.591 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-12T16:54:51.591 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-12T16:54:51.591 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-12T16:54:51.591 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-12T16:54:51.591 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-12T16:54:51.591 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-12T16:54:51.591 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-12T16:54:51.591 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-12T16:54:51.591 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-12T16:54:51.591 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-12T16:54:51.591 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-12T16:54:51.591 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/sit.ko 2024-03-12T16:54:51.591 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/tunnel6.ko 2024-03-12T16:54:51.591 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-12T16:54:51.591 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/key 2024-03-12T16:54:51.592 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/key/af_key.ko 2024-03-12T16:54:51.592 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/l2tp 2024-03-12T16:54:51.592 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/l2tp/l2tp_core.ko 2024-03-12T16:54:51.592 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-12T16:54:51.592 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/l2tp/l2tp_eth.ko 2024-03-12T16:54:51.592 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/l2tp/l2tp_ip.ko 2024-03-12T16:54:51.592 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/l2tp/l2tp_ip6.ko 2024-03-12T16:54:51.592 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/l2tp/l2tp_netlink.ko 2024-03-12T16:54:51.592 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/l2tp/l2tp_ppp.ko 2024-03-12T16:54:51.592 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/llc 2024-03-12T16:54:51.592 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/llc/llc.ko 2024-03-12T16:54:51.592 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/mac802154 2024-03-12T16:54:51.592 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/mac802154/mac802154.ko 2024-03-12T16:54:51.593 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/mpls 2024-03-12T16:54:51.593 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/mpls/mpls_gso.ko 2024-03-12T16:54:51.593 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter 2024-03-12T16:54:51.593 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipset 2024-03-12T16:54:51.593 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipset/ip_set.ko 2024-03-12T16:54:51.593 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-12T16:54:51.593 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-12T16:54:51.593 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-12T16:54:51.593 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-12T16:54:51.593 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-12T16:54:51.593 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-12T16:54:51.593 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-12T16:54:51.593 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-12T16:54:51.593 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-12T16:54:51.594 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-12T16:54:51.594 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-12T16:54:51.594 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-12T16:54:51.594 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-12T16:54:51.594 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-12T16:54:51.594 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-12T16:54:51.594 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipvs 2024-03-12T16:54:51.594 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-12T16:54:51.594 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-12T16:54:51.594 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-12T16:54:51.594 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-12T16:54:51.594 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-12T16:54:51.594 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-12T16:54:51.595 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-12T16:54:51.595 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-12T16:54:51.595 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-12T16:54:51.595 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-12T16:54:51.595 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-12T16:54:51.595 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-12T16:54:51.595 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-12T16:54:51.595 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-12T16:54:51.595 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_conncount.ko 2024-03-12T16:54:51.595 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_conntrack.ko 2024-03-12T16:54:51.595 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-12T16:54:51.595 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-12T16:54:51.595 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-12T16:54:51.595 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-12T16:54:51.596 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-12T16:54:51.596 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-12T16:54:51.596 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-12T16:54:51.596 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-12T16:54:51.596 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-12T16:54:51.596 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-12T16:54:51.596 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-12T16:54:51.596 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-12T16:54:51.596 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-12T16:54:51.596 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_flow_table.ko 2024-03-12T16:54:51.596 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-12T16:54:51.596 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_log_syslog.ko 2024-03-12T16:54:51.596 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_nat.ko 2024-03-12T16:54:51.596 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-12T16:54:51.596 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-12T16:54:51.597 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_nat_irc.ko 2024-03-12T16:54:51.597 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_nat_sip.ko 2024-03-12T16:54:51.597 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-12T16:54:51.597 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-12T16:54:51.597 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_tables.ko 2024-03-12T16:54:51.597 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nfnetlink.ko 2024-03-12T16:54:51.597 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-12T16:54:51.597 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nfnetlink_log.ko 2024-03-12T16:54:51.597 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-12T16:54:51.597 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-12T16:54:51.597 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_chain_nat.ko 2024-03-12T16:54:51.597 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_compat.ko 2024-03-12T16:54:51.597 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_ct.ko 2024-03-12T16:54:51.597 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-12T16:54:51.597 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_fib.ko 2024-03-12T16:54:51.597 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-12T16:54:51.598 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_hash.ko 2024-03-12T16:54:51.598 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_limit.ko 2024-03-12T16:54:51.598 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_log.ko 2024-03-12T16:54:51.598 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_masq.ko 2024-03-12T16:54:51.598 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_nat.ko 2024-03-12T16:54:51.598 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_numgen.ko 2024-03-12T16:54:51.598 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_queue.ko 2024-03-12T16:54:51.598 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_quota.ko 2024-03-12T16:54:51.598 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_redir.ko 2024-03-12T16:54:51.598 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_reject.ko 2024-03-12T16:54:51.598 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_reject_inet.ko 2024-03-12T16:54:51.598 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_socket.ko 2024-03-12T16:54:51.598 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_tunnel.ko 2024-03-12T16:54:51.598 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_AUDIT.ko 2024-03-12T16:54:51.599 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-12T16:54:51.599 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-12T16:54:51.599 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-12T16:54:51.599 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_CT.ko 2024-03-12T16:54:51.599 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_DSCP.ko 2024-03-12T16:54:51.599 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_HL.ko 2024-03-12T16:54:51.599 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_HMARK.ko 2024-03-12T16:54:51.599 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-12T16:54:51.599 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_LED.ko 2024-03-12T16:54:51.599 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_LOG.ko 2024-03-12T16:54:51.599 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-12T16:54:51.599 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_NETMAP.ko 2024-03-12T16:54:51.599 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_NFLOG.ko 2024-03-12T16:54:51.599 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-12T16:54:51.599 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_RATEEST.ko 2024-03-12T16:54:51.600 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-12T16:54:51.600 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_SECMARK.ko 2024-03-12T16:54:51.600 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-12T16:54:51.600 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-12T16:54:51.600 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_TEE.ko 2024-03-12T16:54:51.600 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_TPROXY.ko 2024-03-12T16:54:51.600 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_TRACE.ko 2024-03-12T16:54:51.600 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_addrtype.ko 2024-03-12T16:54:51.600 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_bpf.ko 2024-03-12T16:54:51.600 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_cgroup.ko 2024-03-12T16:54:51.600 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_cluster.ko 2024-03-12T16:54:51.600 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_comment.ko 2024-03-12T16:54:51.600 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_connbytes.ko 2024-03-12T16:54:51.601 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_connlabel.ko 2024-03-12T16:54:51.601 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_connlimit.ko 2024-03-12T16:54:51.601 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_connmark.ko 2024-03-12T16:54:51.601 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_conntrack.ko 2024-03-12T16:54:51.601 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_cpu.ko 2024-03-12T16:54:51.601 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_dccp.ko 2024-03-12T16:54:51.601 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_devgroup.ko 2024-03-12T16:54:51.601 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_dscp.ko 2024-03-12T16:54:51.601 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_ecn.ko 2024-03-12T16:54:51.601 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_esp.ko 2024-03-12T16:54:51.601 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_hashlimit.ko 2024-03-12T16:54:51.601 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_helper.ko 2024-03-12T16:54:51.601 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_hl.ko 2024-03-12T16:54:51.601 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_ipcomp.ko 2024-03-12T16:54:51.601 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_iprange.ko 2024-03-12T16:54:51.602 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_ipvs.ko 2024-03-12T16:54:51.602 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_l2tp.ko 2024-03-12T16:54:51.602 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_length.ko 2024-03-12T16:54:51.602 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_limit.ko 2024-03-12T16:54:51.602 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_mac.ko 2024-03-12T16:54:51.602 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_mark.ko 2024-03-12T16:54:51.602 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_multiport.ko 2024-03-12T16:54:51.602 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_nat.ko 2024-03-12T16:54:51.602 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_nfacct.ko 2024-03-12T16:54:51.602 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_osf.ko 2024-03-12T16:54:51.602 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_owner.ko 2024-03-12T16:54:51.602 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_physdev.ko 2024-03-12T16:54:51.602 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_pkttype.ko 2024-03-12T16:54:51.602 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_policy.ko 2024-03-12T16:54:51.603 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_quota.ko 2024-03-12T16:54:51.603 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_rateest.ko 2024-03-12T16:54:51.603 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_realm.ko 2024-03-12T16:54:51.603 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_recent.ko 2024-03-12T16:54:51.603 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_sctp.ko 2024-03-12T16:54:51.603 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_set.ko 2024-03-12T16:54:51.603 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_socket.ko 2024-03-12T16:54:51.603 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_state.ko 2024-03-12T16:54:51.603 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_statistic.ko 2024-03-12T16:54:51.603 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_string.ko 2024-03-12T16:54:51.603 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_tcpmss.ko 2024-03-12T16:54:51.603 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_time.ko 2024-03-12T16:54:51.603 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_u32.ko 2024-03-12T16:54:51.603 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netlink 2024-03-12T16:54:51.603 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netlink/netlink_diag.ko 2024-03-12T16:54:51.604 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/nsh 2024-03-12T16:54:51.604 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/nsh/nsh.ko 2024-03-12T16:54:51.604 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/openvswitch 2024-03-12T16:54:51.604 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/openvswitch/openvswitch.ko 2024-03-12T16:54:51.604 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/openvswitch/vport-geneve.ko 2024-03-12T16:54:51.604 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/openvswitch/vport-gre.ko 2024-03-12T16:54:51.604 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/openvswitch/vport-vxlan.ko 2024-03-12T16:54:51.604 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/packet 2024-03-12T16:54:51.604 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/packet/af_packet_diag.ko 2024-03-12T16:54:51.604 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/rds 2024-03-12T16:54:51.604 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/rds/rds.ko 2024-03-12T16:54:51.604 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/rds/rds_rdma.ko 2024-03-12T16:54:51.604 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/rds/rds_tcp.ko 2024-03-12T16:54:51.604 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched 2024-03-12T16:54:51.605 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/act_csum.ko 2024-03-12T16:54:51.605 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/act_gact.ko 2024-03-12T16:54:51.605 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/act_mirred.ko 2024-03-12T16:54:51.605 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/act_nat.ko 2024-03-12T16:54:51.605 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/act_pedit.ko 2024-03-12T16:54:51.605 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/act_police.ko 2024-03-12T16:54:51.605 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/act_simple.ko 2024-03-12T16:54:51.605 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/act_skbedit.ko 2024-03-12T16:54:51.605 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/act_vlan.ko 2024-03-12T16:54:51.605 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/cls_basic.ko 2024-03-12T16:54:51.605 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/cls_bpf.ko 2024-03-12T16:54:51.605 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/cls_flow.ko 2024-03-12T16:54:51.605 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/cls_fw.ko 2024-03-12T16:54:51.605 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/cls_route.ko 2024-03-12T16:54:51.606 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/cls_u32.ko 2024-03-12T16:54:51.606 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/em_cmp.ko 2024-03-12T16:54:51.606 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/em_ipset.ko 2024-03-12T16:54:51.606 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/em_meta.ko 2024-03-12T16:54:51.606 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/em_nbyte.ko 2024-03-12T16:54:51.606 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/em_text.ko 2024-03-12T16:54:51.606 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/em_u32.ko 2024-03-12T16:54:51.606 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_choke.ko 2024-03-12T16:54:51.606 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_codel.ko 2024-03-12T16:54:51.606 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_drr.ko 2024-03-12T16:54:51.606 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_fq.ko 2024-03-12T16:54:51.606 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_gred.ko 2024-03-12T16:54:51.606 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_hfsc.ko 2024-03-12T16:54:51.607 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_hhf.ko 2024-03-12T16:54:51.607 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_htb.ko 2024-03-12T16:54:51.607 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_ingress.ko 2024-03-12T16:54:51.607 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_mqprio.ko 2024-03-12T16:54:51.607 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_mqprio_lib.ko 2024-03-12T16:54:51.607 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_multiq.ko 2024-03-12T16:54:51.607 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_netem.ko 2024-03-12T16:54:51.607 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_pie.ko 2024-03-12T16:54:51.607 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_plug.ko 2024-03-12T16:54:51.607 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_prio.ko 2024-03-12T16:54:51.607 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_qfq.ko 2024-03-12T16:54:51.607 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_red.ko 2024-03-12T16:54:51.607 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_sfb.ko 2024-03-12T16:54:51.607 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_sfq.ko 2024-03-12T16:54:51.607 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_tbf.ko 2024-03-12T16:54:51.608 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_teql.ko 2024-03-12T16:54:51.608 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sctp 2024-03-12T16:54:51.608 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sctp/sctp.ko 2024-03-12T16:54:51.608 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sctp/sctp_diag.ko 2024-03-12T16:54:51.608 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sunrpc 2024-03-12T16:54:51.608 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sunrpc/auth_gss 2024-03-12T16:54:51.608 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-12T16:54:51.608 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-12T16:54:51.608 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sunrpc/sunrpc.ko 2024-03-12T16:54:51.608 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sunrpc/xprtrdma 2024-03-12T16:54:51.608 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-12T16:54:51.608 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/tipc 2024-03-12T16:54:51.608 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/tipc/diag.ko 2024-03-12T16:54:51.608 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/tipc/tipc.ko 2024-03-12T16:54:51.608 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/unix 2024-03-12T16:54:51.609 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/unix/unix_diag.ko 2024-03-12T16:54:51.609 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/vmw_vsock 2024-03-12T16:54:51.609 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-12T16:54:51.609 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-12T16:54:51.609 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/vmw_vsock/vsock.ko 2024-03-12T16:54:51.609 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-12T16:54:51.609 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-12T16:54:51.609 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/xfrm 2024-03-12T16:54:51.609 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-12T16:54:51.609 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/security 2024-03-12T16:54:51.609 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/security/keys 2024-03-12T16:54:51.609 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/security/keys/encrypted-keys 2024-03-12T16:54:51.609 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-12T16:54:51.609 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/security/keys/trusted-keys 2024-03-12T16:54:51.609 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/security/keys/trusted-keys/trusted.ko 2024-03-12T16:54:51.609 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/virt 2024-03-12T16:54:51.610 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/virt/lib 2024-03-12T16:54:51.610 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/virt/lib/irqbypass.ko 2024-03-12T16:54:51.610 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/modules.alias 2024-03-12T16:54:51.610 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/modules.alias.bin 2024-03-12T16:54:51.610 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/modules.builtin 2024-03-12T16:54:51.610 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/modules.builtin.alias.bin 2024-03-12T16:54:51.610 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/modules.builtin.bin 2024-03-12T16:54:51.610 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/modules.builtin.modinfo 2024-03-12T16:54:51.610 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/modules.dep 2024-03-12T16:54:51.610 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/modules.dep.bin 2024-03-12T16:54:51.610 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/modules.devname 2024-03-12T16:54:51.610 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/modules.order 2024-03-12T16:54:51.610 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/modules.softdep 2024-03-12T16:54:51.610 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/modules.symbols 2024-03-12T16:54:51.611 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/modules.symbols.bin 2024-03-12T16:54:51.611 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/vmlinuz 2024-03-12T16:54:51.615 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-rc7-gbf761b6efc7e 2024-03-12T16:54:51.615 DEBUG:teuthology.orchestra.run.smithi102:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-12T16:54:51.639 INFO:teuthology.orchestra.run.smithi102.stdout:postinstall scriptlet (using /bin/sh): 2024-03-12T16:54:51.639 INFO:teuthology.orchestra.run.smithi102.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-12T16:54:51.639 INFO:teuthology.orchestra.run.smithi102.stdout: /usr/bin/kernel-install add 6.8.0-rc7-gbf761b6efc7e /lib/modules/6.8.0-rc7-gbf761b6efc7e/vmlinuz 2024-03-12T16:54:51.639 INFO:teuthology.orchestra.run.smithi102.stdout:fi 2024-03-12T16:54:51.639 INFO:teuthology.orchestra.run.smithi102.stdout:for file in vmlinuz System.map config; do 2024-03-12T16:54:51.639 INFO:teuthology.orchestra.run.smithi102.stdout: if ! cmp --silent "/lib/modules/6.8.0-rc7-gbf761b6efc7e/${file}" "/boot/${file}-6.8.0-rc7-gbf761b6efc7e"; then 2024-03-12T16:54:51.639 INFO:teuthology.orchestra.run.smithi102.stdout: cp "/lib/modules/6.8.0-rc7-gbf761b6efc7e/${file}" "/boot/${file}-6.8.0-rc7-gbf761b6efc7e" 2024-03-12T16:54:51.639 INFO:teuthology.orchestra.run.smithi102.stdout: fi 2024-03-12T16:54:51.639 INFO:teuthology.orchestra.run.smithi102.stdout:done 2024-03-12T16:54:51.639 INFO:teuthology.orchestra.run.smithi102.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-12T16:54:51.639 INFO:teuthology.orchestra.run.smithi102.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-12T16:54:51.639 INFO:teuthology.orchestra.run.smithi102.stdout:new-kernel-pkg --remove 6.8.0-rc7-gbf761b6efc7e --rminitrd --initrdfile=/boot/initramfs-6.8.0-rc7-gbf761b6efc7e.img 2024-03-12T16:54:51.640 INFO:teuthology.orchestra.run.smithi102.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-12T16:54:51.640 INFO:teuthology.orchestra.run.smithi102.stdout:kernel-install remove 6.8.0-rc7-gbf761b6efc7e 2024-03-12T16:54:51.640 INFO:teuthology.orchestra.run.smithi102.stdout:fi 2024-03-12T16:54:51.640 INFO:teuthology.orchestra.run.smithi102.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-12T16:54:51.640 INFO:teuthology.orchestra.run.smithi102.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-12T16:54:51.640 INFO:teuthology.orchestra.run.smithi102.stdout:/sbin/update-bootloader --remove 6.8.0-rc7-gbf761b6efc7e 2024-03-12T16:54:51.640 INFO:teuthology.orchestra.run.smithi102.stdout:fi 2024-03-12T16:54:51.640 DEBUG:teuthology.orchestra.run.smithi102:> sudo rpm -qi grub2-tools 2024-03-12T16:54:51.714 INFO:teuthology.orchestra.run.smithi102.stdout:Name : grub2-tools 2024-03-12T16:54:51.714 INFO:teuthology.orchestra.run.smithi102.stdout:Epoch : 1 2024-03-12T16:54:51.714 INFO:teuthology.orchestra.run.smithi102.stdout:Version : 2.06 2024-03-12T16:54:51.714 INFO:teuthology.orchestra.run.smithi102.stdout:Release : 68.el9 2024-03-12T16:54:51.714 INFO:teuthology.orchestra.run.smithi102.stdout:Architecture: x86_64 2024-03-12T16:54:51.714 INFO:teuthology.orchestra.run.smithi102.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-12T16:54:51.715 INFO:teuthology.orchestra.run.smithi102.stdout:Group : Unspecified 2024-03-12T16:54:51.715 INFO:teuthology.orchestra.run.smithi102.stdout:Size : 8264254 2024-03-12T16:54:51.715 INFO:teuthology.orchestra.run.smithi102.stdout:License : GPLv3+ 2024-03-12T16:54:51.715 INFO:teuthology.orchestra.run.smithi102.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-12T16:54:51.715 INFO:teuthology.orchestra.run.smithi102.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-12T16:54:51.715 INFO:teuthology.orchestra.run.smithi102.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-12T16:54:51.715 INFO:teuthology.orchestra.run.smithi102.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-12T16:54:51.715 INFO:teuthology.orchestra.run.smithi102.stdout:Packager : builder@centos.org 2024-03-12T16:54:51.715 INFO:teuthology.orchestra.run.smithi102.stdout:Vendor : CentOS 2024-03-12T16:54:51.715 INFO:teuthology.orchestra.run.smithi102.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-12T16:54:51.715 INFO:teuthology.orchestra.run.smithi102.stdout:Summary : Support tools for GRUB. 2024-03-12T16:54:51.715 INFO:teuthology.orchestra.run.smithi102.stdout:Description : 2024-03-12T16:54:51.715 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T16:54:51.715 INFO:teuthology.orchestra.run.smithi102.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-12T16:54:51.716 INFO:teuthology.orchestra.run.smithi102.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-12T16:54:51.716 INFO:teuthology.orchestra.run.smithi102.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-12T16:54:51.716 INFO:teuthology.orchestra.run.smithi102.stdout:hardware devices. 2024-03-12T16:54:51.716 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T16:54:51.716 INFO:teuthology.orchestra.run.smithi102.stdout:This subpackage provides tools for support of all platforms. 2024-03-12T16:54:51.717 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-12T16:54:51.717 INFO:teuthology.task.kernel:Updating grub on smithi102 to boot 6.8.0-rc7-gbf761b6efc7e 2024-03-12T16:54:51.717 DEBUG:teuthology.orchestra.run.smithi102:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-12T16:54:51.787 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi052.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-12T16:54:51.787 DEBUG:teuthology.orchestra.run.smithi052:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-12T16:54:51.854 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e 2024-03-12T16:54:51.854 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/System.map 2024-03-12T16:54:51.854 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/config 2024-03-12T16:54:51.854 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel 2024-03-12T16:54:51.854 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch 2024-03-12T16:54:51.854 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86 2024-03-12T16:54:51.854 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto 2024-03-12T16:54:51.854 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-12T16:54:51.855 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-12T16:54:51.855 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-12T16:54:51.855 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-12T16:54:51.855 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-12T16:54:51.855 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-12T16:54:51.855 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-12T16:54:51.855 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-12T16:54:51.855 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-12T16:54:51.855 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-12T16:54:51.855 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-12T16:54:51.855 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-12T16:54:51.855 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-12T16:54:51.855 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-12T16:54:51.855 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-12T16:54:51.856 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-12T16:54:51.856 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-12T16:54:51.856 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-12T16:54:51.856 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-12T16:54:51.856 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-12T16:54:51.856 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/kernel 2024-03-12T16:54:51.856 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/kernel/cpu 2024-03-12T16:54:51.856 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/kernel/cpu/mce 2024-03-12T16:54:51.856 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-12T16:54:51.856 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/kvm 2024-03-12T16:54:51.856 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-12T16:54:51.856 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-12T16:54:51.856 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/arch/x86/kvm/kvm.ko 2024-03-12T16:54:51.856 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto 2024-03-12T16:54:51.856 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/ansi_cprng.ko 2024-03-12T16:54:51.857 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/anubis.ko 2024-03-12T16:54:51.857 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/arc4.ko 2024-03-12T16:54:51.857 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/async_tx 2024-03-12T16:54:51.857 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/async_tx/async_memcpy.ko 2024-03-12T16:54:51.857 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/async_tx/async_pq.ko 2024-03-12T16:54:51.857 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-12T16:54:51.857 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/async_tx/async_tx.ko 2024-03-12T16:54:51.857 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/async_tx/async_xor.ko 2024-03-12T16:54:51.857 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/async_tx/raid6test.ko 2024-03-12T16:54:51.857 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/authenc.ko 2024-03-12T16:54:51.857 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/authencesn.ko 2024-03-12T16:54:51.857 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/blake2b_generic.ko 2024-03-12T16:54:51.857 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/blowfish_common.ko 2024-03-12T16:54:51.857 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/blowfish_generic.ko 2024-03-12T16:54:51.857 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/camellia_generic.ko 2024-03-12T16:54:51.858 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/cast5_generic.ko 2024-03-12T16:54:51.858 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/cast6_generic.ko 2024-03-12T16:54:51.858 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/cast_common.ko 2024-03-12T16:54:51.858 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/ccm.ko 2024-03-12T16:54:51.858 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/cmac.ko 2024-03-12T16:54:51.858 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/crc32_generic.ko 2024-03-12T16:54:51.858 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/crypto_engine.ko 2024-03-12T16:54:51.858 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/crypto_user.ko 2024-03-12T16:54:51.858 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/cts.ko 2024-03-12T16:54:51.858 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/des_generic.ko 2024-03-12T16:54:51.858 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/dh_generic.ko 2024-03-12T16:54:51.858 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/echainiv.ko 2024-03-12T16:54:51.858 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/essiv.ko 2024-03-12T16:54:51.858 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/fcrypt.ko 2024-03-12T16:54:51.859 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/khazad.ko 2024-03-12T16:54:51.859 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/lz4.ko 2024-03-12T16:54:51.859 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/lz4hc.ko 2024-03-12T16:54:51.859 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/md4.ko 2024-03-12T16:54:51.859 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/michael_mic.ko 2024-03-12T16:54:51.859 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/pcbc.ko 2024-03-12T16:54:51.859 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/pcrypt.ko 2024-03-12T16:54:51.859 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/rmd160.ko 2024-03-12T16:54:51.859 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/seed.ko 2024-03-12T16:54:51.859 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/serpent_generic.ko 2024-03-12T16:54:51.859 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/tcrypt.ko 2024-03-12T16:54:51.859 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/tea.ko 2024-03-12T16:54:51.859 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/twofish_common.ko 2024-03-12T16:54:51.859 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/twofish_generic.ko 2024-03-12T16:54:51.859 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/vmac.ko 2024-03-12T16:54:51.860 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/wp512.ko 2024-03-12T16:54:51.860 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/xcbc.ko 2024-03-12T16:54:51.860 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/xor.ko 2024-03-12T16:54:51.860 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/crypto/xxhash_generic.ko 2024-03-12T16:54:51.860 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers 2024-03-12T16:54:51.860 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/acpi 2024-03-12T16:54:51.860 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-12T16:54:51.860 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/acpi/acpi_pad.ko 2024-03-12T16:54:51.860 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/acpi/custom_method.ko 2024-03-12T16:54:51.860 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/acpi/ec_sys.ko 2024-03-12T16:54:51.860 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/acpi/platform_profile.ko 2024-03-12T16:54:51.860 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/acpi/sbs.ko 2024-03-12T16:54:51.860 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/acpi/sbshc.ko 2024-03-12T16:54:51.860 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/acpi/video.ko 2024-03-12T16:54:51.860 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata 2024-03-12T16:54:51.861 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/acard-ahci.ko 2024-03-12T16:54:51.861 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/ahci_platform.ko 2024-03-12T16:54:51.861 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/ata_generic.ko 2024-03-12T16:54:51.861 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/libahci_platform.ko 2024-03-12T16:54:51.861 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_acpi.ko 2024-03-12T16:54:51.861 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_ali.ko 2024-03-12T16:54:51.861 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_amd.ko 2024-03-12T16:54:51.861 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_artop.ko 2024-03-12T16:54:51.861 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_atiixp.ko 2024-03-12T16:54:51.861 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_atp867x.ko 2024-03-12T16:54:51.861 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_cmd640.ko 2024-03-12T16:54:51.861 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_cmd64x.ko 2024-03-12T16:54:51.861 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_cypress.ko 2024-03-12T16:54:51.861 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_efar.ko 2024-03-12T16:54:51.861 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_hpt366.ko 2024-03-12T16:54:51.861 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_hpt37x.ko 2024-03-12T16:54:51.862 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-12T16:54:51.862 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-12T16:54:51.862 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_it8213.ko 2024-03-12T16:54:51.862 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_it821x.ko 2024-03-12T16:54:51.862 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_jmicron.ko 2024-03-12T16:54:51.862 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_marvell.ko 2024-03-12T16:54:51.862 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_mpiix.ko 2024-03-12T16:54:51.862 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_netcell.ko 2024-03-12T16:54:51.862 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_ninja32.ko 2024-03-12T16:54:51.862 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_ns87410.ko 2024-03-12T16:54:51.862 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_ns87415.ko 2024-03-12T16:54:51.862 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_oldpiix.ko 2024-03-12T16:54:51.862 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_opti.ko 2024-03-12T16:54:51.862 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_optidma.ko 2024-03-12T16:54:51.863 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_pcmcia.ko 2024-03-12T16:54:51.863 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-12T16:54:51.863 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-12T16:54:51.863 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_piccolo.ko 2024-03-12T16:54:51.863 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_rdc.ko 2024-03-12T16:54:51.863 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_sch.ko 2024-03-12T16:54:51.863 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_serverworks.ko 2024-03-12T16:54:51.863 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_sil680.ko 2024-03-12T16:54:51.863 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_sis.ko 2024-03-12T16:54:51.863 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_sl82c105.ko 2024-03-12T16:54:51.863 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_triflex.ko 2024-03-12T16:54:51.863 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pata_via.ko 2024-03-12T16:54:51.863 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/pdc_adma.ko 2024-03-12T16:54:51.863 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/sata_inic162x.ko 2024-03-12T16:54:51.863 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/sata_mv.ko 2024-03-12T16:54:51.864 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/sata_nv.ko 2024-03-12T16:54:51.864 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/sata_promise.ko 2024-03-12T16:54:51.864 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/sata_qstor.ko 2024-03-12T16:54:51.864 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/sata_sil.ko 2024-03-12T16:54:51.864 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/sata_sil24.ko 2024-03-12T16:54:51.864 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/sata_sis.ko 2024-03-12T16:54:51.864 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/sata_svw.ko 2024-03-12T16:54:51.864 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/sata_sx4.ko 2024-03-12T16:54:51.864 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/sata_uli.ko 2024-03-12T16:54:51.864 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/sata_via.ko 2024-03-12T16:54:51.864 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ata/sata_vsc.ko 2024-03-12T16:54:51.864 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/auxdisplay 2024-03-12T16:54:51.864 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/auxdisplay/ks0108.ko 2024-03-12T16:54:51.864 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/base 2024-03-12T16:54:51.864 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/base/regmap 2024-03-12T16:54:51.865 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-12T16:54:51.865 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-12T16:54:51.865 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/bcma 2024-03-12T16:54:51.865 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/bcma/bcma.ko 2024-03-12T16:54:51.865 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block 2024-03-12T16:54:51.865 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/aoe 2024-03-12T16:54:51.865 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/aoe/aoe.ko 2024-03-12T16:54:51.865 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/brd.ko 2024-03-12T16:54:51.865 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/drbd 2024-03-12T16:54:51.865 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/drbd/drbd.ko 2024-03-12T16:54:51.865 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/floppy.ko 2024-03-12T16:54:51.865 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/loop.ko 2024-03-12T16:54:51.865 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/mtip32xx 2024-03-12T16:54:51.865 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-12T16:54:51.865 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/nbd.ko 2024-03-12T16:54:51.865 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/null_blk 2024-03-12T16:54:51.866 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/null_blk/null_blk.ko 2024-03-12T16:54:51.866 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/pktcdvd.ko 2024-03-12T16:54:51.866 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/rbd.ko 2024-03-12T16:54:51.866 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/virtio_blk.ko 2024-03-12T16:54:51.866 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/xen-blkback 2024-03-12T16:54:51.866 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-12T16:54:51.866 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/xen-blkfront.ko 2024-03-12T16:54:51.866 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/zram 2024-03-12T16:54:51.866 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/block/zram/zram.ko 2024-03-12T16:54:51.866 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char 2024-03-12T16:54:51.866 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/hangcheck-timer.ko 2024-03-12T16:54:51.866 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/hw_random 2024-03-12T16:54:51.866 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-12T16:54:51.866 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-12T16:54:51.867 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-12T16:54:51.867 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/hw_random/via-rng.ko 2024-03-12T16:54:51.867 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-12T16:54:51.867 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/ipmi 2024-03-12T16:54:51.867 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-12T16:54:51.867 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-12T16:54:51.867 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-12T16:54:51.867 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-12T16:54:51.867 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-12T16:54:51.867 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-12T16:54:51.867 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/lp.ko 2024-03-12T16:54:51.867 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/mwave 2024-03-12T16:54:51.867 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/mwave/mwave.ko 2024-03-12T16:54:51.867 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/ppdev.ko 2024-03-12T16:54:51.867 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/tlclk.ko 2024-03-12T16:54:51.868 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/tpm 2024-03-12T16:54:51.868 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/tpm/tpm.ko 2024-03-12T16:54:51.868 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-12T16:54:51.868 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-12T16:54:51.868 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-12T16:54:51.868 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-12T16:54:51.868 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-12T16:54:51.868 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/uv_mmtimer.ko 2024-03-12T16:54:51.868 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/char/virtio_console.ko 2024-03-12T16:54:51.868 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/cpufreq 2024-03-12T16:54:51.868 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-12T16:54:51.868 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-12T16:54:51.868 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-12T16:54:51.868 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-12T16:54:51.868 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-12T16:54:51.868 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-12T16:54:51.869 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/crypto 2024-03-12T16:54:51.869 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/crypto/ccp 2024-03-12T16:54:51.869 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-12T16:54:51.869 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/crypto/ccp/ccp.ko 2024-03-12T16:54:51.869 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/crypto/intel 2024-03-12T16:54:51.869 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/crypto/intel/qat 2024-03-12T16:54:51.869 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/crypto/intel/qat/qat_common 2024-03-12T16:54:51.869 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-12T16:54:51.869 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-12T16:54:51.869 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-12T16:54:51.869 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/crypto/padlock-aes.ko 2024-03-12T16:54:51.869 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/crypto/padlock-sha.ko 2024-03-12T16:54:51.869 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/crypto/virtio 2024-03-12T16:54:51.869 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-12T16:54:51.869 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/dca 2024-03-12T16:54:51.869 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/dca/dca.ko 2024-03-12T16:54:51.870 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/devfreq 2024-03-12T16:54:51.870 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-12T16:54:51.870 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/dma 2024-03-12T16:54:51.870 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/dma/dw 2024-03-12T16:54:51.870 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-12T16:54:51.870 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/dma/ioat 2024-03-12T16:54:51.870 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-12T16:54:51.870 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/edac 2024-03-12T16:54:51.870 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/edac/amd64_edac.ko 2024-03-12T16:54:51.870 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/edac/e752x_edac.ko 2024-03-12T16:54:51.870 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/edac/edac_mce_amd.ko 2024-03-12T16:54:51.870 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/edac/i3000_edac.ko 2024-03-12T16:54:51.870 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/edac/i3200_edac.ko 2024-03-12T16:54:51.871 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/edac/i5100_edac.ko 2024-03-12T16:54:51.871 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/edac/i5400_edac.ko 2024-03-12T16:54:51.871 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/edac/i7300_edac.ko 2024-03-12T16:54:51.871 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/edac/i7core_edac.ko 2024-03-12T16:54:51.871 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/edac/i82975x_edac.ko 2024-03-12T16:54:51.871 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/edac/ie31200_edac.ko 2024-03-12T16:54:51.871 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/edac/sb_edac.ko 2024-03-12T16:54:51.871 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/edac/x38_edac.ko 2024-03-12T16:54:51.871 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/firmware 2024-03-12T16:54:51.871 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/firmware/edd.ko 2024-03-12T16:54:51.871 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-12T16:54:51.871 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/gpio 2024-03-12T16:54:51.871 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/gpio/gpio-ich.ko 2024-03-12T16:54:51.871 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-12T16:54:51.872 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid 2024-03-12T16:54:51.872 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-appleir.ko 2024-03-12T16:54:51.872 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-aureal.ko 2024-03-12T16:54:51.872 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-axff.ko 2024-03-12T16:54:51.872 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-dr.ko 2024-03-12T16:54:51.872 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-elecom.ko 2024-03-12T16:54:51.872 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-elo.ko 2024-03-12T16:54:51.872 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-emsff.ko 2024-03-12T16:54:51.872 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-gaff.ko 2024-03-12T16:54:51.872 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-gt683r.ko 2024-03-12T16:54:51.872 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-gyration.ko 2024-03-12T16:54:51.872 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-12T16:54:51.872 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-12T16:54:51.872 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-holtekff.ko 2024-03-12T16:54:51.872 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-hyperv.ko 2024-03-12T16:54:51.873 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-icade.ko 2024-03-12T16:54:51.873 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-keytouch.ko 2024-03-12T16:54:51.873 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-kye.ko 2024-03-12T16:54:51.873 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-lcpower.ko 2024-03-12T16:54:51.873 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-led.ko 2024-03-12T16:54:51.873 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-lenovo.ko 2024-03-12T16:54:51.873 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-12T16:54:51.873 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-12T16:54:51.873 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-multitouch.ko 2024-03-12T16:54:51.873 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-ortek.ko 2024-03-12T16:54:51.873 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-penmount.ko 2024-03-12T16:54:51.873 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-petalynx.ko 2024-03-12T16:54:51.873 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-picolcd.ko 2024-03-12T16:54:51.873 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-pl.ko 2024-03-12T16:54:51.873 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-plantronics.ko 2024-03-12T16:54:51.874 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-primax.ko 2024-03-12T16:54:51.874 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-rmi.ko 2024-03-12T16:54:51.874 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-12T16:54:51.874 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-roccat-common.ko 2024-03-12T16:54:51.874 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-12T16:54:51.874 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-12T16:54:51.874 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-12T16:54:51.874 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-12T16:54:51.874 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-12T16:54:51.874 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-12T16:54:51.874 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-12T16:54:51.874 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-12T16:54:51.875 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-12T16:54:51.875 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-roccat.ko 2024-03-12T16:54:51.875 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-saitek.ko 2024-03-12T16:54:51.875 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-samsung.ko 2024-03-12T16:54:51.875 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-12T16:54:51.875 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-sjoy.ko 2024-03-12T16:54:51.875 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-sony.ko 2024-03-12T16:54:51.875 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-speedlink.ko 2024-03-12T16:54:51.875 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-steelseries.ko 2024-03-12T16:54:51.875 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-sunplus.ko 2024-03-12T16:54:51.875 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-12T16:54:51.875 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-tivo.ko 2024-03-12T16:54:51.875 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-tmff.ko 2024-03-12T16:54:51.876 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-topseed.ko 2024-03-12T16:54:51.876 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-twinhan.ko 2024-03-12T16:54:51.876 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-uclogic.ko 2024-03-12T16:54:51.876 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-waltop.ko 2024-03-12T16:54:51.876 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-wiimote.ko 2024-03-12T16:54:51.876 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-xinmo.ko 2024-03-12T16:54:51.876 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-zpff.ko 2024-03-12T16:54:51.876 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/hid-zydacron.ko 2024-03-12T16:54:51.876 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/uhid.ko 2024-03-12T16:54:51.876 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hid/wacom.ko 2024-03-12T16:54:51.876 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hv 2024-03-12T16:54:51.876 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hv/hv_balloon.ko 2024-03-12T16:54:51.876 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hv/hv_utils.ko 2024-03-12T16:54:51.876 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hv/hv_vmbus.ko 2024-03-12T16:54:51.876 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon 2024-03-12T16:54:51.877 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/abituguru.ko 2024-03-12T16:54:51.877 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/abituguru3.ko 2024-03-12T16:54:51.877 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-12T16:54:51.877 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/ad7414.ko 2024-03-12T16:54:51.877 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/ad7418.ko 2024-03-12T16:54:51.877 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/adc128d818.ko 2024-03-12T16:54:51.877 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/adm1025.ko 2024-03-12T16:54:51.877 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/adm1026.ko 2024-03-12T16:54:51.877 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/adm1029.ko 2024-03-12T16:54:51.877 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/adm1031.ko 2024-03-12T16:54:51.877 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/adm9240.ko 2024-03-12T16:54:51.877 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/ads7828.ko 2024-03-12T16:54:51.877 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/adt7410.ko 2024-03-12T16:54:51.877 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/adt7411.ko 2024-03-12T16:54:51.877 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/adt7462.ko 2024-03-12T16:54:51.878 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/adt7470.ko 2024-03-12T16:54:51.878 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/adt7475.ko 2024-03-12T16:54:51.878 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/adt7x10.ko 2024-03-12T16:54:51.878 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/amc6821.ko 2024-03-12T16:54:51.878 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/applesmc.ko 2024-03-12T16:54:51.878 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/asb100.ko 2024-03-12T16:54:51.878 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/asc7621.ko 2024-03-12T16:54:51.878 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-12T16:54:51.878 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/atxp1.ko 2024-03-12T16:54:51.878 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/coretemp.ko 2024-03-12T16:54:51.878 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-12T16:54:51.878 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/dme1737.ko 2024-03-12T16:54:51.878 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/ds1621.ko 2024-03-12T16:54:51.878 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/ds620.ko 2024-03-12T16:54:51.879 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/emc1403.ko 2024-03-12T16:54:51.879 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/emc6w201.ko 2024-03-12T16:54:51.879 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/f71805f.ko 2024-03-12T16:54:51.879 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/f71882fg.ko 2024-03-12T16:54:51.879 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/f75375s.ko 2024-03-12T16:54:51.879 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/fam15h_power.ko 2024-03-12T16:54:51.879 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/fschmd.ko 2024-03-12T16:54:51.879 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/g760a.ko 2024-03-12T16:54:51.879 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/g762.ko 2024-03-12T16:54:51.879 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/gl518sm.ko 2024-03-12T16:54:51.879 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/gl520sm.ko 2024-03-12T16:54:51.879 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-12T16:54:51.879 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/i5500_temp.ko 2024-03-12T16:54:51.880 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/i5k_amb.ko 2024-03-12T16:54:51.880 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/ibmaem.ko 2024-03-12T16:54:51.880 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/ibmpex.ko 2024-03-12T16:54:51.880 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/ina209.ko 2024-03-12T16:54:51.880 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/ina2xx.ko 2024-03-12T16:54:51.880 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/it87.ko 2024-03-12T16:54:51.880 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/k10temp.ko 2024-03-12T16:54:51.880 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/k8temp.ko 2024-03-12T16:54:51.880 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/lineage-pem.ko 2024-03-12T16:54:51.880 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/lm63.ko 2024-03-12T16:54:51.880 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/lm73.ko 2024-03-12T16:54:51.880 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/lm75.ko 2024-03-12T16:54:51.880 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/lm77.ko 2024-03-12T16:54:51.880 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/lm78.ko 2024-03-12T16:54:51.880 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/lm80.ko 2024-03-12T16:54:51.881 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/lm83.ko 2024-03-12T16:54:51.881 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/lm85.ko 2024-03-12T16:54:51.881 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/lm87.ko 2024-03-12T16:54:51.881 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/lm90.ko 2024-03-12T16:54:51.881 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/lm92.ko 2024-03-12T16:54:51.881 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/lm93.ko 2024-03-12T16:54:51.881 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/lm95234.ko 2024-03-12T16:54:51.881 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/lm95241.ko 2024-03-12T16:54:51.881 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/lm95245.ko 2024-03-12T16:54:51.881 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/ltc2945.ko 2024-03-12T16:54:51.881 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/ltc4151.ko 2024-03-12T16:54:51.881 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/ltc4215.ko 2024-03-12T16:54:51.881 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/ltc4222.ko 2024-03-12T16:54:51.881 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/ltc4245.ko 2024-03-12T16:54:51.881 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/ltc4260.ko 2024-03-12T16:54:51.881 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/ltc4261.ko 2024-03-12T16:54:51.882 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/max16065.ko 2024-03-12T16:54:51.882 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/max1619.ko 2024-03-12T16:54:51.882 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/max1668.ko 2024-03-12T16:54:51.882 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/max197.ko 2024-03-12T16:54:51.882 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/max6639.ko 2024-03-12T16:54:51.882 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/max6650.ko 2024-03-12T16:54:51.882 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/max6697.ko 2024-03-12T16:54:51.882 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/mcp3021.ko 2024-03-12T16:54:51.882 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/nct6683.ko 2024-03-12T16:54:51.882 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/nct6775-core.ko 2024-03-12T16:54:51.882 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/nct6775.ko 2024-03-12T16:54:51.882 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/nct7802.ko 2024-03-12T16:54:51.882 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/pc87360.ko 2024-03-12T16:54:51.883 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/pc87427.ko 2024-03-12T16:54:51.883 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/pcf8591.ko 2024-03-12T16:54:51.883 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/pmbus 2024-03-12T16:54:51.883 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-12T16:54:51.883 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-12T16:54:51.883 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-12T16:54:51.883 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-12T16:54:51.883 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-12T16:54:51.883 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-12T16:54:51.883 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-12T16:54:51.883 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-12T16:54:51.883 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-12T16:54:51.883 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-12T16:54:51.884 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-12T16:54:51.884 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-12T16:54:51.884 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/powr1220.ko 2024-03-12T16:54:51.884 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/sch5627.ko 2024-03-12T16:54:51.884 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/sch5636.ko 2024-03-12T16:54:51.884 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-12T16:54:51.884 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/sht15.ko 2024-03-12T16:54:51.884 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/sht21.ko 2024-03-12T16:54:51.884 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/shtc1.ko 2024-03-12T16:54:51.884 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/sis5595.ko 2024-03-12T16:54:51.884 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/smsc47b397.ko 2024-03-12T16:54:51.884 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/smsc47m1.ko 2024-03-12T16:54:51.884 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/smsc47m192.ko 2024-03-12T16:54:51.884 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/thmc50.ko 2024-03-12T16:54:51.885 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/tmp102.ko 2024-03-12T16:54:51.885 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/tmp103.ko 2024-03-12T16:54:51.885 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/tmp401.ko 2024-03-12T16:54:51.885 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/tmp421.ko 2024-03-12T16:54:51.885 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/via-cputemp.ko 2024-03-12T16:54:51.885 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/via686a.ko 2024-03-12T16:54:51.885 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/vt1211.ko 2024-03-12T16:54:51.885 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/vt8231.ko 2024-03-12T16:54:51.885 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/w83627ehf.ko 2024-03-12T16:54:51.885 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/w83627hf.ko 2024-03-12T16:54:51.885 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/w83781d.ko 2024-03-12T16:54:51.885 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/w83791d.ko 2024-03-12T16:54:51.885 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/w83792d.ko 2024-03-12T16:54:51.886 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/w83793.ko 2024-03-12T16:54:51.886 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/w83795.ko 2024-03-12T16:54:51.886 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/w83l785ts.ko 2024-03-12T16:54:51.886 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/hwmon/w83l786ng.ko 2024-03-12T16:54:51.886 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c 2024-03-12T16:54:51.886 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/algos 2024-03-12T16:54:51.886 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-12T16:54:51.886 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-12T16:54:51.886 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses 2024-03-12T16:54:51.886 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-12T16:54:51.886 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-12T16:54:51.886 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-12T16:54:51.886 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-12T16:54:51.886 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-12T16:54:51.887 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-12T16:54:51.887 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-12T16:54:51.887 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-12T16:54:51.887 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-12T16:54:51.887 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-12T16:54:51.887 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-12T16:54:51.887 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-12T16:54:51.887 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-12T16:54:51.887 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-12T16:54:51.887 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-12T16:54:51.887 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-12T16:54:51.887 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-12T16:54:51.887 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-12T16:54:51.888 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-12T16:54:51.888 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-12T16:54:51.888 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-12T16:54:51.888 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-12T16:54:51.888 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-12T16:54:51.888 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/i2c-dev.ko 2024-03-12T16:54:51.888 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/i2c-mux.ko 2024-03-12T16:54:51.888 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-12T16:54:51.888 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/i2c-smbus.ko 2024-03-12T16:54:51.888 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/i2c/i2c-stub.ko 2024-03-12T16:54:51.888 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband 2024-03-12T16:54:51.888 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/core 2024-03-12T16:54:51.888 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-12T16:54:51.888 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/core/ib_core.ko 2024-03-12T16:54:51.889 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-12T16:54:51.889 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-12T16:54:51.889 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-12T16:54:51.889 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-12T16:54:51.889 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-12T16:54:51.889 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/hw 2024-03-12T16:54:51.889 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/hw/cxgb4 2024-03-12T16:54:51.889 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-12T16:54:51.889 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/hw/mlx4 2024-03-12T16:54:51.889 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-12T16:54:51.889 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/hw/mlx5 2024-03-12T16:54:51.889 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-12T16:54:51.889 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/hw/mthca 2024-03-12T16:54:51.890 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-12T16:54:51.890 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/ulp 2024-03-12T16:54:51.890 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/ulp/ipoib 2024-03-12T16:54:51.890 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-12T16:54:51.890 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/ulp/iser 2024-03-12T16:54:51.890 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-12T16:54:51.890 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/ulp/isert 2024-03-12T16:54:51.890 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-12T16:54:51.890 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/ulp/srp 2024-03-12T16:54:51.890 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-12T16:54:51.890 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/ulp/srpt 2024-03-12T16:54:51.890 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-12T16:54:51.890 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input 2024-03-12T16:54:51.890 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/gameport 2024-03-12T16:54:51.891 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-12T16:54:51.891 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-12T16:54:51.891 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/gameport/gameport.ko 2024-03-12T16:54:51.891 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/joydev.ko 2024-03-12T16:54:51.891 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/keyboard 2024-03-12T16:54:51.891 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-12T16:54:51.891 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/mouse 2024-03-12T16:54:51.891 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/mouse/appletouch.ko 2024-03-12T16:54:51.891 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/mouse/bcm5974.ko 2024-03-12T16:54:51.891 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/mouse/cyapatp.ko 2024-03-12T16:54:51.891 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-12T16:54:51.891 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/mouse/sermouse.ko 2024-03-12T16:54:51.891 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-12T16:54:51.891 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-12T16:54:51.892 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-12T16:54:51.892 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/rmi4 2024-03-12T16:54:51.892 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-12T16:54:51.892 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/serio 2024-03-12T16:54:51.892 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/serio/altera_ps2.ko 2024-03-12T16:54:51.892 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/serio/arc_ps2.ko 2024-03-12T16:54:51.892 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-12T16:54:51.892 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/serio/serio_raw.ko 2024-03-12T16:54:51.892 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/input/sparse-keymap.ko 2024-03-12T16:54:51.892 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/leds 2024-03-12T16:54:51.892 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/leds/leds-blinkm.ko 2024-03-12T16:54:51.892 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/leds/leds-lm3530.ko 2024-03-12T16:54:51.892 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/leds/leds-lp3944.ko 2024-03-12T16:54:51.892 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/leds/leds-ss4200.ko 2024-03-12T16:54:51.893 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/leds/trigger 2024-03-12T16:54:51.893 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-12T16:54:51.893 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-12T16:54:51.893 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-12T16:54:51.893 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-12T16:54:51.893 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-12T16:54:51.893 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-12T16:54:51.893 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-12T16:54:51.893 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-12T16:54:51.893 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-12T16:54:51.893 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md 2024-03-12T16:54:51.893 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/bcache 2024-03-12T16:54:51.893 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/bcache/bcache.ko 2024-03-12T16:54:51.893 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/dm-bio-prison.ko 2024-03-12T16:54:51.893 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/dm-cache-smq.ko 2024-03-12T16:54:51.894 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/dm-cache.ko 2024-03-12T16:54:51.894 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/dm-crypt.ko 2024-03-12T16:54:51.894 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/dm-delay.ko 2024-03-12T16:54:51.894 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/dm-flakey.ko 2024-03-12T16:54:51.894 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/dm-log-userspace.ko 2024-03-12T16:54:51.894 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/dm-multipath.ko 2024-03-12T16:54:51.894 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/dm-queue-length.ko 2024-03-12T16:54:51.894 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/dm-raid.ko 2024-03-12T16:54:51.894 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/dm-round-robin.ko 2024-03-12T16:54:51.894 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/dm-service-time.ko 2024-03-12T16:54:51.894 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/dm-switch.ko 2024-03-12T16:54:51.894 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/dm-thin-pool.ko 2024-03-12T16:54:51.894 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/dm-verity.ko 2024-03-12T16:54:51.895 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/persistent-data 2024-03-12T16:54:51.895 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-12T16:54:51.895 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/raid0.ko 2024-03-12T16:54:51.895 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/raid1.ko 2024-03-12T16:54:51.895 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/raid10.ko 2024-03-12T16:54:51.895 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/md/raid456.ko 2024-03-12T16:54:51.895 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/message 2024-03-12T16:54:51.895 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/message/fusion 2024-03-12T16:54:51.895 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/message/fusion/mptbase.ko 2024-03-12T16:54:51.895 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/message/fusion/mptctl.ko 2024-03-12T16:54:51.895 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/message/fusion/mptfc.ko 2024-03-12T16:54:51.895 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/message/fusion/mptlan.ko 2024-03-12T16:54:51.895 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/message/fusion/mptsas.ko 2024-03-12T16:54:51.895 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/message/fusion/mptscsih.ko 2024-03-12T16:54:51.895 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/message/fusion/mptspi.ko 2024-03-12T16:54:51.895 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/mfd 2024-03-12T16:54:51.896 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/mfd/lpc_ich.ko 2024-03-12T16:54:51.896 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/mfd/lpc_sch.ko 2024-03-12T16:54:51.896 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/mfd/mfd-core.ko 2024-03-12T16:54:51.896 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/mfd/sm501.ko 2024-03-12T16:54:51.896 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/mfd/viperboard.ko 2024-03-12T16:54:51.896 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/mfd/vx855.ko 2024-03-12T16:54:51.896 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/mfd/wl1273-core.ko 2024-03-12T16:54:51.896 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc 2024-03-12T16:54:51.896 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/altera-stapl 2024-03-12T16:54:51.896 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-12T16:54:51.896 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/apds9802als.ko 2024-03-12T16:54:51.896 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/apds990x.ko 2024-03-12T16:54:51.896 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/bh1770glc.ko 2024-03-12T16:54:51.896 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/cb710 2024-03-12T16:54:51.896 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/cb710/cb710.ko 2024-03-12T16:54:51.896 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/eeprom 2024-03-12T16:54:51.897 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/eeprom/at24.ko 2024-03-12T16:54:51.897 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-12T16:54:51.897 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/eeprom/max6875.ko 2024-03-12T16:54:51.897 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/enclosure.ko 2024-03-12T16:54:51.897 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/hpilo.ko 2024-03-12T16:54:51.897 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/isl29003.ko 2024-03-12T16:54:51.897 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/isl29020.ko 2024-03-12T16:54:51.897 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/lis3lv02d 2024-03-12T16:54:51.897 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-12T16:54:51.897 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-12T16:54:51.897 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/mei 2024-03-12T16:54:51.898 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/mei/mei-me.ko 2024-03-12T16:54:51.898 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/mei/mei-txe.ko 2024-03-12T16:54:51.898 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/mei/mei.ko 2024-03-12T16:54:51.898 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/sgi-gru 2024-03-12T16:54:51.898 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-12T16:54:51.898 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/sgi-xp 2024-03-12T16:54:51.898 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-12T16:54:51.898 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-12T16:54:51.898 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-12T16:54:51.898 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/tifm_7xx1.ko 2024-03-12T16:54:51.898 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/tifm_core.ko 2024-03-12T16:54:51.898 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/tsl2550.ko 2024-03-12T16:54:51.898 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/vmw_balloon.ko 2024-03-12T16:54:51.898 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/vmw_vmci 2024-03-12T16:54:51.898 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-12T16:54:51.899 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net 2024-03-12T16:54:51.899 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/bonding 2024-03-12T16:54:51.899 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/bonding/bonding.ko 2024-03-12T16:54:51.899 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/dsa 2024-03-12T16:54:51.899 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/dsa/b53 2024-03-12T16:54:51.899 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-12T16:54:51.899 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-12T16:54:51.899 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-12T16:54:51.899 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/dsa/mv88e6xxx 2024-03-12T16:54:51.899 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-12T16:54:51.899 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/dummy.ko 2024-03-12T16:54:51.899 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/eql.ko 2024-03-12T16:54:51.899 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet 2024-03-12T16:54:51.899 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/3com 2024-03-12T16:54:51.899 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-12T16:54:51.900 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-12T16:54:51.900 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-12T16:54:51.900 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-12T16:54:51.900 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/8390 2024-03-12T16:54:51.900 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-12T16:54:51.900 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-12T16:54:51.900 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-12T16:54:51.900 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-12T16:54:51.900 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/adaptec 2024-03-12T16:54:51.900 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-12T16:54:51.900 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/agere 2024-03-12T16:54:51.900 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-12T16:54:51.900 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/alteon 2024-03-12T16:54:51.900 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-12T16:54:51.901 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/altera 2024-03-12T16:54:51.901 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-12T16:54:51.901 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/amd 2024-03-12T16:54:51.901 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-12T16:54:51.901 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-12T16:54:51.901 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-12T16:54:51.901 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/atheros 2024-03-12T16:54:51.901 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/atheros/alx 2024-03-12T16:54:51.901 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-12T16:54:51.901 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-12T16:54:51.901 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-12T16:54:51.901 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-12T16:54:51.901 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-12T16:54:51.901 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/atheros/atlx 2024-03-12T16:54:51.901 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-12T16:54:51.902 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-12T16:54:51.902 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/broadcom 2024-03-12T16:54:51.902 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-12T16:54:51.902 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-12T16:54:51.902 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-12T16:54:51.902 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-12T16:54:51.902 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-12T16:54:51.902 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/broadcom/genet 2024-03-12T16:54:51.902 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-12T16:54:51.902 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-12T16:54:51.902 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/brocade 2024-03-12T16:54:51.902 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/brocade/bna 2024-03-12T16:54:51.902 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-12T16:54:51.902 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/chelsio 2024-03-12T16:54:51.903 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-12T16:54:51.903 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-12T16:54:51.903 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-12T16:54:51.903 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-12T16:54:51.903 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-12T16:54:51.903 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-12T16:54:51.903 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-12T16:54:51.903 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-12T16:54:51.903 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-12T16:54:51.903 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-12T16:54:51.903 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/cisco 2024-03-12T16:54:51.903 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/cisco/enic 2024-03-12T16:54:51.903 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-12T16:54:51.903 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/dec 2024-03-12T16:54:51.904 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/dec/tulip 2024-03-12T16:54:51.904 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-12T16:54:51.904 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-12T16:54:51.904 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-12T16:54:51.904 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-12T16:54:51.904 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-12T16:54:51.904 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-12T16:54:51.904 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/dlink 2024-03-12T16:54:51.904 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-12T16:54:51.904 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-12T16:54:51.904 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/dnet.ko 2024-03-12T16:54:51.904 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/emulex 2024-03-12T16:54:51.904 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/emulex/benet 2024-03-12T16:54:51.904 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-12T16:54:51.904 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/ethoc.ko 2024-03-12T16:54:51.905 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/fealnx.ko 2024-03-12T16:54:51.905 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel 2024-03-12T16:54:51.905 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-12T16:54:51.905 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/e1000 2024-03-12T16:54:51.905 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-12T16:54:51.905 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/e1000e 2024-03-12T16:54:51.905 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-12T16:54:51.905 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/fm10k 2024-03-12T16:54:51.905 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-12T16:54:51.905 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/i40e 2024-03-12T16:54:51.905 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-12T16:54:51.905 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/iavf 2024-03-12T16:54:51.905 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-12T16:54:51.905 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/igb 2024-03-12T16:54:51.906 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-12T16:54:51.906 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/igbvf 2024-03-12T16:54:51.906 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-12T16:54:51.906 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-12T16:54:51.906 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-12T16:54:51.906 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-12T16:54:51.906 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-12T16:54:51.906 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/jme.ko 2024-03-12T16:54:51.906 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/marvell 2024-03-12T16:54:51.906 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-12T16:54:51.906 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-12T16:54:51.906 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-12T16:54:51.906 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/mellanox 2024-03-12T16:54:51.906 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-12T16:54:51.907 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-12T16:54:51.907 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-12T16:54:51.907 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-12T16:54:51.907 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-12T16:54:51.907 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-12T16:54:51.907 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/micrel 2024-03-12T16:54:51.907 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-12T16:54:51.907 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/myricom 2024-03-12T16:54:51.907 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-12T16:54:51.907 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-12T16:54:51.907 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/natsemi 2024-03-12T16:54:51.907 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-12T16:54:51.907 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-12T16:54:51.907 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/neterion 2024-03-12T16:54:51.907 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-12T16:54:51.908 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/nvidia 2024-03-12T16:54:51.908 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-12T16:54:51.908 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/packetengines 2024-03-12T16:54:51.908 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-12T16:54:51.908 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-12T16:54:51.908 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/qlogic 2024-03-12T16:54:51.908 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-12T16:54:51.908 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-12T16:54:51.908 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-12T16:54:51.908 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-12T16:54:51.908 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-12T16:54:51.908 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/rdc 2024-03-12T16:54:51.908 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-12T16:54:51.908 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/realtek 2024-03-12T16:54:51.908 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-12T16:54:51.908 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-12T16:54:51.909 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-12T16:54:51.909 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-12T16:54:51.909 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/rocker 2024-03-12T16:54:51.909 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-12T16:54:51.909 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/sfc 2024-03-12T16:54:51.909 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-12T16:54:51.909 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/silan 2024-03-12T16:54:51.909 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-12T16:54:51.909 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/sis 2024-03-12T16:54:51.909 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-12T16:54:51.909 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-12T16:54:51.909 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/smsc 2024-03-12T16:54:51.909 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-12T16:54:51.909 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-12T16:54:51.909 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-12T16:54:51.910 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-12T16:54:51.910 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/stmicro 2024-03-12T16:54:51.910 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-12T16:54:51.910 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-12T16:54:51.910 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-12T16:54:51.910 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/sun 2024-03-12T16:54:51.910 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-12T16:54:51.910 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-12T16:54:51.910 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-12T16:54:51.910 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-12T16:54:51.910 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/tehuti 2024-03-12T16:54:51.910 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-12T16:54:51.910 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/ti 2024-03-12T16:54:51.910 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-12T16:54:51.910 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/via 2024-03-12T16:54:51.911 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-12T16:54:51.911 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-12T16:54:51.911 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/wiznet 2024-03-12T16:54:51.911 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-12T16:54:51.911 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-12T16:54:51.911 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/xircom 2024-03-12T16:54:51.911 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-12T16:54:51.911 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/geneve.ko 2024-03-12T16:54:51.911 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/hyperv 2024-03-12T16:54:51.911 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-12T16:54:51.911 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ieee802154 2024-03-12T16:54:51.911 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-12T16:54:51.911 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ifb.ko 2024-03-12T16:54:51.911 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ipvlan 2024-03-12T16:54:51.912 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-12T16:54:51.912 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/macvlan.ko 2024-03-12T16:54:51.912 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/macvtap.ko 2024-03-12T16:54:51.912 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/mdio 2024-03-12T16:54:51.912 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/mdio.ko 2024-03-12T16:54:51.912 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-12T16:54:51.912 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-12T16:54:51.912 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-12T16:54:51.912 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/mii.ko 2024-03-12T16:54:51.912 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/net_failover.ko 2024-03-12T16:54:51.912 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/netconsole.ko 2024-03-12T16:54:51.912 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/nlmon.ko 2024-03-12T16:54:51.912 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/pcs 2024-03-12T16:54:51.912 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-12T16:54:51.913 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-12T16:54:51.913 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy 2024-03-12T16:54:51.913 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/amd.ko 2024-03-12T16:54:51.913 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-12T16:54:51.913 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-12T16:54:51.913 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-12T16:54:51.913 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/bcm87xx.ko 2024-03-12T16:54:51.913 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/broadcom.ko 2024-03-12T16:54:51.913 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/cicada.ko 2024-03-12T16:54:51.913 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/davicom.ko 2024-03-12T16:54:51.913 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/dp83640.ko 2024-03-12T16:54:51.914 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/et1011c.ko 2024-03-12T16:54:51.914 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/icplus.ko 2024-03-12T16:54:51.914 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/lxt.ko 2024-03-12T16:54:51.914 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/marvell.ko 2024-03-12T16:54:51.914 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/micrel.ko 2024-03-12T16:54:51.914 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/national.ko 2024-03-12T16:54:51.914 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/phylink.ko 2024-03-12T16:54:51.914 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/qsemi.ko 2024-03-12T16:54:51.914 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/realtek.ko 2024-03-12T16:54:51.914 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/smsc.ko 2024-03-12T16:54:51.914 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/ste10Xp.ko 2024-03-12T16:54:51.914 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/phy/vitesse.ko 2024-03-12T16:54:51.914 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ppp 2024-03-12T16:54:51.915 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-12T16:54:51.915 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ppp/ppp_async.ko 2024-03-12T16:54:51.915 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-12T16:54:51.915 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-12T16:54:51.915 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-12T16:54:51.915 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-12T16:54:51.915 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ppp/pppoe.ko 2024-03-12T16:54:51.915 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ppp/pppox.ko 2024-03-12T16:54:51.915 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/ppp/pptp.ko 2024-03-12T16:54:51.915 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/slip 2024-03-12T16:54:51.915 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/slip/slhc.ko 2024-03-12T16:54:51.915 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/slip/slip.ko 2024-03-12T16:54:51.915 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/sungem_phy.ko 2024-03-12T16:54:51.915 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/tap.ko 2024-03-12T16:54:51.916 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/team 2024-03-12T16:54:51.916 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/team/team.ko 2024-03-12T16:54:51.916 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-12T16:54:51.916 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-12T16:54:51.916 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-12T16:54:51.916 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/team/team_mode_random.ko 2024-03-12T16:54:51.916 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-12T16:54:51.916 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/tun.ko 2024-03-12T16:54:51.916 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/veth.ko 2024-03-12T16:54:51.916 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/virtio_net.ko 2024-03-12T16:54:51.916 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/vmxnet3 2024-03-12T16:54:51.916 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-12T16:54:51.916 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/vxlan 2024-03-12T16:54:51.916 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/vxlan/vxlan.ko 2024-03-12T16:54:51.916 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/xen-netback 2024-03-12T16:54:51.917 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-12T16:54:51.917 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/net/xen-netfront.ko 2024-03-12T16:54:51.917 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ntb 2024-03-12T16:54:51.917 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ntb/ntb.ko 2024-03-12T16:54:51.917 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/nvme 2024-03-12T16:54:51.917 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/nvme/host 2024-03-12T16:54:51.917 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/nvme/host/nvme-core.ko 2024-03-12T16:54:51.917 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/nvme/host/nvme.ko 2024-03-12T16:54:51.917 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/parport 2024-03-12T16:54:51.917 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/parport/parport.ko 2024-03-12T16:54:51.917 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/parport/parport_cs.ko 2024-03-12T16:54:51.917 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/parport/parport_pc.ko 2024-03-12T16:54:51.918 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/parport/parport_serial.ko 2024-03-12T16:54:51.918 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pci 2024-03-12T16:54:51.918 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pci/hotplug 2024-03-12T16:54:51.918 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-12T16:54:51.918 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pci/pcie 2024-03-12T16:54:51.918 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-12T16:54:51.918 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pci/xen-pcifront.ko 2024-03-12T16:54:51.918 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pcmcia 2024-03-12T16:54:51.918 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pcmcia/i82092.ko 2024-03-12T16:54:51.918 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pcmcia/pd6729.ko 2024-03-12T16:54:51.918 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-12T16:54:51.918 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pinctrl 2024-03-12T16:54:51.918 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pinctrl/intel 2024-03-12T16:54:51.918 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-12T16:54:51.919 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform 2024-03-12T16:54:51.919 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86 2024-03-12T16:54:51.919 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-12T16:54:51.919 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/acerhdf.ko 2024-03-12T16:54:51.919 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-12T16:54:51.919 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-12T16:54:51.919 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-12T16:54:51.919 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-12T16:54:51.919 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-12T16:54:51.919 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-12T16:54:51.919 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-12T16:54:51.919 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-12T16:54:51.919 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-12T16:54:51.919 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/hdaps.ko 2024-03-12T16:54:51.919 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/intel 2024-03-12T16:54:51.920 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-12T16:54:51.920 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/intel_ips.ko 2024-03-12T16:54:51.920 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-12T16:54:51.920 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-12T16:54:51.920 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-12T16:54:51.920 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-12T16:54:51.920 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-12T16:54:51.920 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-12T16:54:51.920 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-12T16:54:51.920 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-12T16:54:51.920 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-12T16:54:51.920 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/platform/x86/wmi.ko 2024-03-12T16:54:51.920 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/powercap 2024-03-12T16:54:51.920 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-12T16:54:51.920 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-12T16:54:51.921 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pps 2024-03-12T16:54:51.921 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pps/clients 2024-03-12T16:54:51.921 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-12T16:54:51.921 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-12T16:54:51.921 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pps/clients/pps_parport.ko 2024-03-12T16:54:51.921 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/pps/pps_core.ko 2024-03-12T16:54:51.921 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ptp 2024-03-12T16:54:51.921 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ptp/ptp.ko 2024-03-12T16:54:51.921 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ptp/ptp_kvm.ko 2024-03-12T16:54:51.921 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc 2024-03-12T16:54:51.921 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-12T16:54:51.921 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-12T16:54:51.921 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-12T16:54:51.921 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-12T16:54:51.921 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-12T16:54:51.922 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-12T16:54:51.922 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-12T16:54:51.922 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-12T16:54:51.922 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-12T16:54:51.922 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-12T16:54:51.922 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-em3027.ko 2024-03-12T16:54:51.922 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-12T16:54:51.922 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-12T16:54:51.922 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-12T16:54:51.922 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-12T16:54:51.922 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-12T16:54:51.922 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-12T16:54:51.922 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-max6900.ko 2024-03-12T16:54:51.922 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-12T16:54:51.923 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-12T16:54:51.923 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-12T16:54:51.923 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-12T16:54:51.923 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-12T16:54:51.923 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-12T16:54:51.923 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-12T16:54:51.923 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-12T16:54:51.923 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-12T16:54:51.923 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-12T16:54:51.923 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-12T16:54:51.923 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-12T16:54:51.923 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/rtc/rtc-x1205.ko 2024-03-12T16:54:51.923 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi 2024-03-12T16:54:51.923 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/3w-9xxx.ko 2024-03-12T16:54:51.924 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/3w-sas.ko 2024-03-12T16:54:51.924 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/3w-xxxx.ko 2024-03-12T16:54:51.924 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/BusLogic.ko 2024-03-12T16:54:51.924 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/a100u2w.ko 2024-03-12T16:54:51.924 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/aacraid 2024-03-12T16:54:51.924 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-12T16:54:51.924 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/advansys.ko 2024-03-12T16:54:51.924 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/aic7xxx 2024-03-12T16:54:51.924 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-12T16:54:51.924 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-12T16:54:51.924 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/am53c974.ko 2024-03-12T16:54:51.924 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/arcmsr 2024-03-12T16:54:51.924 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-12T16:54:51.924 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/atp870u.ko 2024-03-12T16:54:51.924 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/be2iscsi 2024-03-12T16:54:51.924 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-12T16:54:51.925 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/bfa 2024-03-12T16:54:51.925 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/bfa/bfa.ko 2024-03-12T16:54:51.925 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/bnx2fc 2024-03-12T16:54:51.925 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-12T16:54:51.925 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/bnx2i 2024-03-12T16:54:51.925 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-12T16:54:51.925 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/ch.ko 2024-03-12T16:54:51.925 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/csiostor 2024-03-12T16:54:51.925 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-12T16:54:51.925 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/cxgbi 2024-03-12T16:54:51.925 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-12T16:54:51.925 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-12T16:54:51.925 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-12T16:54:51.925 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-12T16:54:51.926 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-12T16:54:51.926 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/dc395x.ko 2024-03-12T16:54:51.926 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/device_handler 2024-03-12T16:54:51.926 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-12T16:54:51.926 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-12T16:54:51.926 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-12T16:54:51.926 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-12T16:54:51.926 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/dmx3191d.ko 2024-03-12T16:54:51.926 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/esas2r 2024-03-12T16:54:51.926 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-12T16:54:51.926 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/esp_scsi.ko 2024-03-12T16:54:51.926 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/fcoe 2024-03-12T16:54:51.926 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-12T16:54:51.926 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-12T16:54:51.926 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/fnic 2024-03-12T16:54:51.927 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/fnic/fnic.ko 2024-03-12T16:54:51.927 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/hpsa.ko 2024-03-12T16:54:51.927 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/hptiop.ko 2024-03-12T16:54:51.927 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/hv_storvsc.ko 2024-03-12T16:54:51.927 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/initio.ko 2024-03-12T16:54:51.927 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/ipr.ko 2024-03-12T16:54:51.927 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/ips.ko 2024-03-12T16:54:51.927 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/isci 2024-03-12T16:54:51.927 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/isci/isci.ko 2024-03-12T16:54:51.927 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-12T16:54:51.927 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-12T16:54:51.927 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/libfc 2024-03-12T16:54:51.927 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/libfc/libfc.ko 2024-03-12T16:54:51.927 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/libiscsi.ko 2024-03-12T16:54:51.927 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-12T16:54:51.928 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/libsas 2024-03-12T16:54:51.928 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/libsas/libsas.ko 2024-03-12T16:54:51.928 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/lpfc 2024-03-12T16:54:51.928 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-12T16:54:51.928 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/megaraid 2024-03-12T16:54:51.928 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/megaraid.ko 2024-03-12T16:54:51.928 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-12T16:54:51.928 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-12T16:54:51.928 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-12T16:54:51.928 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/mpt3sas 2024-03-12T16:54:51.928 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-12T16:54:51.928 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/mvsas 2024-03-12T16:54:51.928 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-12T16:54:51.928 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/mvumi.ko 2024-03-12T16:54:51.928 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/pm8001 2024-03-12T16:54:51.928 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-12T16:54:51.929 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/pmcraid.ko 2024-03-12T16:54:51.929 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/qla1280.ko 2024-03-12T16:54:51.929 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/qla2xxx 2024-03-12T16:54:51.929 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-12T16:54:51.929 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-12T16:54:51.929 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/qla4xxx 2024-03-12T16:54:51.929 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-12T16:54:51.929 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/raid_class.ko 2024-03-12T16:54:51.929 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/scsi_debug.ko 2024-03-12T16:54:51.929 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-12T16:54:51.929 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-12T16:54:51.929 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-12T16:54:51.929 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-12T16:54:51.929 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-12T16:54:51.930 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/ses.ko 2024-03-12T16:54:51.930 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/st.ko 2024-03-12T16:54:51.930 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/stex.ko 2024-03-12T16:54:51.930 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/sym53c8xx_2 2024-03-12T16:54:51.930 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-12T16:54:51.930 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/virtio_scsi.ko 2024-03-12T16:54:51.930 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-12T16:54:51.930 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/scsi/wd719x.ko 2024-03-12T16:54:51.930 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ssb 2024-03-12T16:54:51.930 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ssb/ssb.ko 2024-03-12T16:54:51.930 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/target 2024-03-12T16:54:51.930 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/target/iscsi 2024-03-12T16:54:51.930 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-12T16:54:51.930 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/target/loopback 2024-03-12T16:54:51.931 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-12T16:54:51.931 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/target/target_core_file.ko 2024-03-12T16:54:51.931 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/target/target_core_iblock.ko 2024-03-12T16:54:51.931 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/target/target_core_mod.ko 2024-03-12T16:54:51.931 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/target/target_core_pscsi.ko 2024-03-12T16:54:51.931 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/target/target_core_user.ko 2024-03-12T16:54:51.931 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/target/tcm_fc 2024-03-12T16:54:51.931 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-12T16:54:51.931 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal 2024-03-12T16:54:51.931 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel 2024-03-12T16:54:51.931 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/int340x_thermal 2024-03-12T16:54:51.931 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-12T16:54:51.931 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-12T16:54:51.931 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-12T16:54:51.932 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-12T16:54:51.932 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-12T16:54:51.932 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-12T16:54:51.932 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-12T16:54:51.932 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-12T16:54:51.932 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-12T16:54:51.932 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-12T16:54:51.932 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-12T16:54:51.932 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-12T16:54:51.932 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-12T16:54:51.932 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-12T16:54:51.932 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-12T16:54:51.932 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-12T16:54:51.932 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-12T16:54:51.932 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-12T16:54:51.933 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/tty 2024-03-12T16:54:51.933 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/tty/ipwireless 2024-03-12T16:54:51.933 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-12T16:54:51.933 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/tty/n_gsm.ko 2024-03-12T16:54:51.933 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/tty/n_hdlc.ko 2024-03-12T16:54:51.933 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/tty/nozomi.ko 2024-03-12T16:54:51.933 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/tty/serial 2024-03-12T16:54:51.933 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/tty/serial/8250 2024-03-12T16:54:51.933 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-12T16:54:51.933 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/tty/serial/arc_uart.ko 2024-03-12T16:54:51.933 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/tty/serial/jsm 2024-03-12T16:54:51.933 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-12T16:54:51.933 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ufs 2024-03-12T16:54:51.934 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ufs/core 2024-03-12T16:54:51.934 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-12T16:54:51.934 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ufs/host 2024-03-12T16:54:51.934 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-12T16:54:51.934 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/uio 2024-03-12T16:54:51.934 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/uio/uio.ko 2024-03-12T16:54:51.934 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/uio/uio_aec.ko 2024-03-12T16:54:51.934 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/uio/uio_cif.ko 2024-03-12T16:54:51.934 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/uio/uio_pci_generic.ko 2024-03-12T16:54:51.934 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/uio/uio_sercos3.ko 2024-03-12T16:54:51.934 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb 2024-03-12T16:54:51.934 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/class 2024-03-12T16:54:51.934 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/class/cdc-acm.ko 2024-03-12T16:54:51.934 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-12T16:54:51.935 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/class/usblp.ko 2024-03-12T16:54:51.935 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/class/usbtmc.ko 2024-03-12T16:54:51.935 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/host 2024-03-12T16:54:51.935 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-12T16:54:51.935 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/image 2024-03-12T16:54:51.935 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/image/mdc800.ko 2024-03-12T16:54:51.935 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/image/microtek.ko 2024-03-12T16:54:51.935 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc 2024-03-12T16:54:51.935 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/adutux.ko 2024-03-12T16:54:51.935 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/appledisplay.ko 2024-03-12T16:54:51.935 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/emi26.ko 2024-03-12T16:54:51.935 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/emi62.ko 2024-03-12T16:54:51.935 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/ezusb.ko 2024-03-12T16:54:51.936 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/idmouse.ko 2024-03-12T16:54:51.936 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/iowarrior.ko 2024-03-12T16:54:51.936 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-12T16:54:51.936 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/ldusb.ko 2024-03-12T16:54:51.936 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/legousbtower.ko 2024-03-12T16:54:51.936 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/sisusbvga 2024-03-12T16:54:51.936 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-12T16:54:51.936 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-12T16:54:51.936 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/usb3503.ko 2024-03-12T16:54:51.936 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/usblcd.ko 2024-03-12T16:54:51.936 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-12T16:54:51.936 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/uss720.ko 2024-03-12T16:54:51.937 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/misc/yurex.ko 2024-03-12T16:54:51.937 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/phy 2024-03-12T16:54:51.937 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/phy/phy-generic.ko 2024-03-12T16:54:51.937 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial 2024-03-12T16:54:51.937 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/aircable.ko 2024-03-12T16:54:51.937 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/ark3116.ko 2024-03-12T16:54:51.937 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-12T16:54:51.937 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/ch341.ko 2024-03-12T16:54:51.937 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/cp210x.ko 2024-03-12T16:54:51.937 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/cyberjack.ko 2024-03-12T16:54:51.937 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-12T16:54:51.937 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-12T16:54:51.938 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/empeg.ko 2024-03-12T16:54:51.938 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-12T16:54:51.938 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-12T16:54:51.938 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-12T16:54:51.938 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/io_ti.ko 2024-03-12T16:54:51.938 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/ipaq.ko 2024-03-12T16:54:51.938 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/ipw.ko 2024-03-12T16:54:51.938 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/ir-usb.ko 2024-03-12T16:54:51.938 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-12T16:54:51.938 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/keyspan.ko 2024-03-12T16:54:51.938 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-12T16:54:51.938 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-12T16:54:51.939 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-12T16:54:51.939 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/mct_u232.ko 2024-03-12T16:54:51.939 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/mos7720.ko 2024-03-12T16:54:51.939 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/mos7840.ko 2024-03-12T16:54:51.939 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/navman.ko 2024-03-12T16:54:51.939 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/omninet.ko 2024-03-12T16:54:51.939 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/opticon.ko 2024-03-12T16:54:51.939 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/option.ko 2024-03-12T16:54:51.939 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/oti6858.ko 2024-03-12T16:54:51.939 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/pl2303.ko 2024-03-12T16:54:51.939 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/qcaux.ko 2024-03-12T16:54:51.939 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/qcserial.ko 2024-03-12T16:54:51.939 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/quatech2.ko 2024-03-12T16:54:51.940 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/safe_serial.ko 2024-03-12T16:54:51.940 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/sierra.ko 2024-03-12T16:54:51.940 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-12T16:54:51.940 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/ssu100.ko 2024-03-12T16:54:51.940 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/symbolserial.ko 2024-03-12T16:54:51.940 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-12T16:54:51.940 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-12T16:54:51.940 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/usb_debug.ko 2024-03-12T16:54:51.940 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-12T16:54:51.940 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/visor.ko 2024-03-12T16:54:51.940 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/whiteheat.ko 2024-03-12T16:54:51.940 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-12T16:54:51.940 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/storage 2024-03-12T16:54:51.941 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/storage/uas.ko 2024-03-12T16:54:51.941 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-12T16:54:51.941 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-12T16:54:51.941 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-12T16:54:51.941 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-12T16:54:51.941 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-12T16:54:51.941 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-12T16:54:51.941 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-12T16:54:51.941 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/storage/ums-karma.ko 2024-03-12T16:54:51.941 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-12T16:54:51.941 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-12T16:54:51.941 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-12T16:54:51.941 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-12T16:54:51.941 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-12T16:54:51.942 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/storage/usb-storage.ko 2024-03-12T16:54:51.942 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/usbip 2024-03-12T16:54:51.942 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-12T16:54:51.942 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-12T16:54:51.942 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-12T16:54:51.942 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/vfio 2024-03-12T16:54:51.942 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/vfio/pci 2024-03-12T16:54:51.942 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-12T16:54:51.942 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-12T16:54:51.942 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/vfio/vfio.ko 2024-03-12T16:54:51.942 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-12T16:54:51.942 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/vhost 2024-03-12T16:54:51.942 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/vhost/vhost.ko 2024-03-12T16:54:51.942 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-12T16:54:51.943 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/vhost/vhost_net.ko 2024-03-12T16:54:51.943 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/vhost/vhost_scsi.ko 2024-03-12T16:54:51.943 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/video 2024-03-12T16:54:51.943 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/video/backlight 2024-03-12T16:54:51.943 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/video/backlight/apple_bl.ko 2024-03-12T16:54:51.943 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/video/backlight/lcd.ko 2024-03-12T16:54:51.943 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-12T16:54:51.943 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/virtio 2024-03-12T16:54:51.943 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/virtio/virtio.ko 2024-03-12T16:54:51.943 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/virtio/virtio_balloon.ko 2024-03-12T16:54:51.943 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/virtio/virtio_mmio.ko 2024-03-12T16:54:51.943 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/virtio/virtio_pci.ko 2024-03-12T16:54:51.943 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-12T16:54:51.943 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-12T16:54:51.943 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/virtio/virtio_ring.ko 2024-03-12T16:54:51.944 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/w1 2024-03-12T16:54:51.944 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/w1/masters 2024-03-12T16:54:51.944 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/w1/masters/ds2482.ko 2024-03-12T16:54:51.944 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/w1/masters/ds2490.ko 2024-03-12T16:54:51.944 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/w1/slaves 2024-03-12T16:54:51.944 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-12T16:54:51.944 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-12T16:54:51.944 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-12T16:54:51.944 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-12T16:54:51.944 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-12T16:54:51.944 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-12T16:54:51.944 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-12T16:54:51.944 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-12T16:54:51.944 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-12T16:54:51.944 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-12T16:54:51.945 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-12T16:54:51.945 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/w1/wire.ko 2024-03-12T16:54:51.945 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog 2024-03-12T16:54:51.945 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-12T16:54:51.945 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-12T16:54:51.945 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-12T16:54:51.945 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/hpwdt.ko 2024-03-12T16:54:51.945 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/i6300esb.ko 2024-03-12T16:54:51.945 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-12T16:54:51.945 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-12T16:54:51.945 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/ib700wdt.ko 2024-03-12T16:54:51.945 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/ibmasr.ko 2024-03-12T16:54:51.945 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-12T16:54:51.945 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-12T16:54:51.945 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/it87_wdt.ko 2024-03-12T16:54:51.945 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/machzwd.ko 2024-03-12T16:54:51.946 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/nv_tco.ko 2024-03-12T16:54:51.946 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-12T16:54:51.946 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-12T16:54:51.946 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-12T16:54:51.946 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-12T16:54:51.946 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/softdog.ko 2024-03-12T16:54:51.946 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-12T16:54:51.946 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/via_wdt.ko 2024-03-12T16:54:51.946 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-12T16:54:51.946 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-12T16:54:51.946 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-12T16:54:51.946 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/wdt_pci.ko 2024-03-12T16:54:51.946 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/watchdog/xen_wdt.ko 2024-03-12T16:54:51.946 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/xen 2024-03-12T16:54:51.949 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-12T16:54:51.949 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/xen/xen-evtchn.ko 2024-03-12T16:54:51.950 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/xen/xen-gntalloc.ko 2024-03-12T16:54:51.950 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/xen/xen-gntdev.ko 2024-03-12T16:54:51.950 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/xen/xen-pciback 2024-03-12T16:54:51.950 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-12T16:54:51.950 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/xen/xen-privcmd.ko 2024-03-12T16:54:51.950 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/xen/xenfs 2024-03-12T16:54:51.950 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-12T16:54:51.950 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs 2024-03-12T16:54:51.950 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/9p 2024-03-12T16:54:51.950 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/9p/9p.ko 2024-03-12T16:54:51.950 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/affs 2024-03-12T16:54:51.950 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/affs/affs.ko 2024-03-12T16:54:51.951 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/befs 2024-03-12T16:54:51.951 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/befs/befs.ko 2024-03-12T16:54:51.951 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/binfmt_misc.ko 2024-03-12T16:54:51.951 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/btrfs 2024-03-12T16:54:51.951 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/btrfs/btrfs.ko 2024-03-12T16:54:51.951 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/cachefiles 2024-03-12T16:54:51.951 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/cachefiles/cachefiles.ko 2024-03-12T16:54:51.951 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/ceph 2024-03-12T16:54:51.951 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/ceph/ceph.ko 2024-03-12T16:54:51.951 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/coda 2024-03-12T16:54:51.951 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/coda/coda.ko 2024-03-12T16:54:51.951 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/cramfs 2024-03-12T16:54:51.951 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/cramfs/cramfs.ko 2024-03-12T16:54:51.951 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/dlm 2024-03-12T16:54:51.952 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/dlm/dlm.ko 2024-03-12T16:54:51.952 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/ecryptfs 2024-03-12T16:54:51.952 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-12T16:54:51.952 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/f2fs 2024-03-12T16:54:51.952 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/f2fs/f2fs.ko 2024-03-12T16:54:51.952 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/fat 2024-03-12T16:54:51.952 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/fat/fat.ko 2024-03-12T16:54:51.952 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/fat/msdos.ko 2024-03-12T16:54:51.952 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/fat/vfat.ko 2024-03-12T16:54:51.952 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/fuse 2024-03-12T16:54:51.952 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/fuse/cuse.ko 2024-03-12T16:54:51.952 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/fuse/fuse.ko 2024-03-12T16:54:51.952 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/gfs2 2024-03-12T16:54:51.952 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/gfs2/gfs2.ko 2024-03-12T16:54:51.952 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/hfs 2024-03-12T16:54:51.953 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/hfs/hfs.ko 2024-03-12T16:54:51.953 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/hfsplus 2024-03-12T16:54:51.953 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/hfsplus/hfsplus.ko 2024-03-12T16:54:51.953 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/isofs 2024-03-12T16:54:51.953 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/isofs/isofs.ko 2024-03-12T16:54:51.953 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/jfs 2024-03-12T16:54:51.953 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/jfs/jfs.ko 2024-03-12T16:54:51.953 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/lockd 2024-03-12T16:54:51.953 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/lockd/lockd.ko 2024-03-12T16:54:51.953 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/minix 2024-03-12T16:54:51.953 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/minix/minix.ko 2024-03-12T16:54:51.953 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/netfs 2024-03-12T16:54:51.953 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/netfs/netfs.ko 2024-03-12T16:54:51.953 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nfs 2024-03-12T16:54:51.954 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nfs/blocklayout 2024-03-12T16:54:51.954 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-12T16:54:51.954 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nfs/filelayout 2024-03-12T16:54:51.954 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-12T16:54:51.954 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nfs/flexfilelayout 2024-03-12T16:54:51.954 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-12T16:54:51.954 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nfs/nfs.ko 2024-03-12T16:54:51.954 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nfs/nfsv3.ko 2024-03-12T16:54:51.954 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nfs/nfsv4.ko 2024-03-12T16:54:51.954 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nfs_common 2024-03-12T16:54:51.954 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nfs_common/grace.ko 2024-03-12T16:54:51.954 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nfs_common/nfs_acl.ko 2024-03-12T16:54:51.954 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nfsd 2024-03-12T16:54:51.954 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nfsd/nfsd.ko 2024-03-12T16:54:51.955 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nilfs2 2024-03-12T16:54:51.955 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nilfs2/nilfs2.ko 2024-03-12T16:54:51.955 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls 2024-03-12T16:54:51.955 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/mac-celtic.ko 2024-03-12T16:54:51.955 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/mac-centeuro.ko 2024-03-12T16:54:51.955 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/mac-croatian.ko 2024-03-12T16:54:51.955 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/mac-cyrillic.ko 2024-03-12T16:54:51.955 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/mac-gaelic.ko 2024-03-12T16:54:51.955 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/mac-greek.ko 2024-03-12T16:54:51.955 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/mac-iceland.ko 2024-03-12T16:54:51.955 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/mac-inuit.ko 2024-03-12T16:54:51.955 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/mac-roman.ko 2024-03-12T16:54:51.955 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/mac-romanian.ko 2024-03-12T16:54:51.955 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/mac-turkish.ko 2024-03-12T16:54:51.955 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp1250.ko 2024-03-12T16:54:51.956 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp1251.ko 2024-03-12T16:54:51.956 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp1255.ko 2024-03-12T16:54:51.956 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp737.ko 2024-03-12T16:54:51.956 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp775.ko 2024-03-12T16:54:51.956 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp850.ko 2024-03-12T16:54:51.956 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp852.ko 2024-03-12T16:54:51.956 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp855.ko 2024-03-12T16:54:51.956 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp857.ko 2024-03-12T16:54:51.956 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp860.ko 2024-03-12T16:54:51.956 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp861.ko 2024-03-12T16:54:51.956 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp862.ko 2024-03-12T16:54:51.956 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp863.ko 2024-03-12T16:54:51.956 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp864.ko 2024-03-12T16:54:51.957 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp865.ko 2024-03-12T16:54:51.957 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp866.ko 2024-03-12T16:54:51.957 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp869.ko 2024-03-12T16:54:51.957 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp874.ko 2024-03-12T16:54:51.957 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp932.ko 2024-03-12T16:54:51.957 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp936.ko 2024-03-12T16:54:51.957 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp949.ko 2024-03-12T16:54:51.957 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_cp950.ko 2024-03-12T16:54:51.957 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_euc-jp.ko 2024-03-12T16:54:51.957 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_iso8859-1.ko 2024-03-12T16:54:51.957 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_iso8859-13.ko 2024-03-12T16:54:51.957 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_iso8859-14.ko 2024-03-12T16:54:51.957 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_iso8859-15.ko 2024-03-12T16:54:51.957 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_iso8859-2.ko 2024-03-12T16:54:51.957 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_iso8859-3.ko 2024-03-12T16:54:51.958 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_iso8859-4.ko 2024-03-12T16:54:51.958 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_iso8859-5.ko 2024-03-12T16:54:51.958 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_iso8859-6.ko 2024-03-12T16:54:51.958 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_iso8859-7.ko 2024-03-12T16:54:51.958 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_iso8859-9.ko 2024-03-12T16:54:51.958 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_koi8-r.ko 2024-03-12T16:54:51.958 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_koi8-ru.ko 2024-03-12T16:54:51.958 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_koi8-u.ko 2024-03-12T16:54:51.958 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-12T16:54:51.958 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/nls/nls_utf8.ko 2024-03-12T16:54:51.958 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/ocfs2 2024-03-12T16:54:51.958 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/ocfs2/cluster 2024-03-12T16:54:51.959 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-12T16:54:51.959 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/ocfs2/dlm 2024-03-12T16:54:51.959 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-12T16:54:51.959 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/ocfs2/dlmfs 2024-03-12T16:54:51.959 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-12T16:54:51.959 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/ocfs2/ocfs2.ko 2024-03-12T16:54:51.959 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-12T16:54:51.959 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-12T16:54:51.959 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-12T16:54:51.959 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/overlayfs 2024-03-12T16:54:51.959 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/overlayfs/overlay.ko 2024-03-12T16:54:51.959 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/pstore 2024-03-12T16:54:51.959 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/pstore/ramoops.ko 2024-03-12T16:54:51.959 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/reiserfs 2024-03-12T16:54:51.960 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/reiserfs/reiserfs.ko 2024-03-12T16:54:51.960 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/romfs 2024-03-12T16:54:51.960 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/romfs/romfs.ko 2024-03-12T16:54:51.960 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/smb 2024-03-12T16:54:51.960 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/smb/client 2024-03-12T16:54:51.960 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/smb/client/cifs.ko 2024-03-12T16:54:51.960 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/smb/common 2024-03-12T16:54:51.960 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/smb/common/cifs_arc4.ko 2024-03-12T16:54:51.960 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/smb/common/cifs_md4.ko 2024-03-12T16:54:51.960 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/squashfs 2024-03-12T16:54:51.960 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/squashfs/squashfs.ko 2024-03-12T16:54:51.960 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/sysv 2024-03-12T16:54:51.960 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/sysv/sysv.ko 2024-03-12T16:54:51.961 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/udf 2024-03-12T16:54:51.961 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/udf/udf.ko 2024-03-12T16:54:51.961 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/ufs 2024-03-12T16:54:51.961 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/ufs/ufs.ko 2024-03-12T16:54:51.961 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/xfs 2024-03-12T16:54:51.961 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/fs/xfs/xfs.ko 2024-03-12T16:54:51.961 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/kernel 2024-03-12T16:54:51.961 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/kernel/rcu 2024-03-12T16:54:51.961 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/kernel/rcu/rcutorture.ko 2024-03-12T16:54:51.961 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/kernel/torture.ko 2024-03-12T16:54:51.961 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/kernel/trace 2024-03-12T16:54:51.961 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-12T16:54:51.961 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib 2024-03-12T16:54:51.961 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/asn1_encoder.ko 2024-03-12T16:54:51.962 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/crc-itu-t.ko 2024-03-12T16:54:51.962 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/crc8.ko 2024-03-12T16:54:51.962 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/crypto 2024-03-12T16:54:51.962 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/crypto/libarc4.ko 2024-03-12T16:54:51.962 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/crypto/libdes.ko 2024-03-12T16:54:51.962 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/libcrc32c.ko 2024-03-12T16:54:51.962 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/lru_cache.ko 2024-03-12T16:54:51.962 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/lz4 2024-03-12T16:54:51.962 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/lz4/lz4_compress.ko 2024-03-12T16:54:51.962 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/lz4/lz4hc_compress.ko 2024-03-12T16:54:51.962 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/math 2024-03-12T16:54:51.962 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/math/cordic.ko 2024-03-12T16:54:51.963 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/raid6 2024-03-12T16:54:51.963 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/raid6/raid6_pq.ko 2024-03-12T16:54:51.963 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/reed_solomon 2024-03-12T16:54:51.963 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-12T16:54:51.963 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/ts_bm.ko 2024-03-12T16:54:51.963 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/ts_fsm.ko 2024-03-12T16:54:51.963 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/ts_kmp.ko 2024-03-12T16:54:51.963 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/zstd 2024-03-12T16:54:51.963 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/lib/zstd/zstd_compress.ko 2024-03-12T16:54:51.963 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/mm 2024-03-12T16:54:51.963 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/mm/hwpoison-inject.ko 2024-03-12T16:54:51.963 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net 2024-03-12T16:54:51.963 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/6lowpan 2024-03-12T16:54:51.963 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/6lowpan/6lowpan.ko 2024-03-12T16:54:51.964 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/6lowpan/nhc_dest.ko 2024-03-12T16:54:51.964 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/6lowpan/nhc_fragment.ko 2024-03-12T16:54:51.964 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/6lowpan/nhc_hop.ko 2024-03-12T16:54:51.964 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-12T16:54:51.964 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/6lowpan/nhc_mobility.ko 2024-03-12T16:54:51.964 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/6lowpan/nhc_routing.ko 2024-03-12T16:54:51.964 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/6lowpan/nhc_udp.ko 2024-03-12T16:54:51.964 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/802 2024-03-12T16:54:51.964 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/802/garp.ko 2024-03-12T16:54:51.964 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/802/mrp.ko 2024-03-12T16:54:51.964 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/802/p8022.ko 2024-03-12T16:54:51.964 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/802/psnap.ko 2024-03-12T16:54:51.964 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/802/stp.ko 2024-03-12T16:54:51.964 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/8021q 2024-03-12T16:54:51.965 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/8021q/8021q.ko 2024-03-12T16:54:51.965 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/9p 2024-03-12T16:54:51.965 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/9p/9pnet.ko 2024-03-12T16:54:51.965 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/9p/9pnet_fd.ko 2024-03-12T16:54:51.965 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/9p/9pnet_rdma.ko 2024-03-12T16:54:51.965 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/9p/9pnet_virtio.ko 2024-03-12T16:54:51.965 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/batman-adv 2024-03-12T16:54:51.965 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/batman-adv/batman-adv.ko 2024-03-12T16:54:51.965 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge 2024-03-12T16:54:51.965 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/br_netfilter.ko 2024-03-12T16:54:51.965 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/bridge.ko 2024-03-12T16:54:51.965 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter 2024-03-12T16:54:51.965 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-12T16:54:51.965 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-12T16:54:51.966 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-12T16:54:51.966 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-12T16:54:51.966 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-12T16:54:51.966 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-12T16:54:51.966 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-12T16:54:51.966 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-12T16:54:51.966 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-12T16:54:51.966 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-12T16:54:51.966 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-12T16:54:51.966 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-12T16:54:51.966 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-12T16:54:51.966 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-12T16:54:51.967 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-12T16:54:51.967 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-12T16:54:51.967 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-12T16:54:51.967 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-12T16:54:51.967 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-12T16:54:51.967 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-12T16:54:51.967 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/ebtables.ko 2024-03-12T16:54:51.967 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-12T16:54:51.967 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-12T16:54:51.967 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-12T16:54:51.967 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ceph 2024-03-12T16:54:51.967 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ceph/libceph.ko 2024-03-12T16:54:51.967 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/core 2024-03-12T16:54:51.967 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/core/failover.ko 2024-03-12T16:54:51.968 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/core/pktgen.ko 2024-03-12T16:54:51.968 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/dccp 2024-03-12T16:54:51.968 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/dccp/dccp.ko 2024-03-12T16:54:51.968 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/dccp/dccp_diag.ko 2024-03-12T16:54:51.968 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/dccp/dccp_ipv4.ko 2024-03-12T16:54:51.968 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/dccp/dccp_ipv6.ko 2024-03-12T16:54:51.968 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/dns_resolver 2024-03-12T16:54:51.968 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/dns_resolver/dns_resolver.ko 2024-03-12T16:54:51.968 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/dsa 2024-03-12T16:54:51.968 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/dsa/dsa_core.ko 2024-03-12T16:54:51.968 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/dsa/tag_brcm.ko 2024-03-12T16:54:51.968 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/dsa/tag_dsa.ko 2024-03-12T16:54:51.968 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/dsa/tag_none.ko 2024-03-12T16:54:51.968 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/dsa/tag_trailer.ko 2024-03-12T16:54:51.969 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ieee802154 2024-03-12T16:54:51.969 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ieee802154/6lowpan 2024-03-12T16:54:51.969 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-12T16:54:51.969 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ieee802154/ieee802154.ko 2024-03-12T16:54:51.969 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-12T16:54:51.969 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4 2024-03-12T16:54:51.969 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/ah4.ko 2024-03-12T16:54:51.969 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/esp4.ko 2024-03-12T16:54:51.969 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/fou.ko 2024-03-12T16:54:51.969 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/gre.ko 2024-03-12T16:54:51.969 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/inet_diag.ko 2024-03-12T16:54:51.969 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/ip_gre.ko 2024-03-12T16:54:51.969 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/ip_tunnel.ko 2024-03-12T16:54:51.969 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/ip_vti.ko 2024-03-12T16:54:51.969 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/ipcomp.ko 2024-03-12T16:54:51.970 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/ipip.ko 2024-03-12T16:54:51.970 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter 2024-03-12T16:54:51.970 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-12T16:54:51.970 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-12T16:54:51.970 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-12T16:54:51.970 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-12T16:54:51.970 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-12T16:54:51.970 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-12T16:54:51.970 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-12T16:54:51.970 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-12T16:54:51.970 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-12T16:54:51.970 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-12T16:54:51.970 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-12T16:54:51.971 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-12T16:54:51.971 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-12T16:54:51.971 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-12T16:54:51.971 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-12T16:54:51.971 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-12T16:54:51.971 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-12T16:54:51.971 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-12T16:54:51.971 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-12T16:54:51.971 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-12T16:54:51.971 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-12T16:54:51.971 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/tcp_bic.ko 2024-03-12T16:54:51.971 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/tcp_dctcp.ko 2024-03-12T16:54:51.971 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/tcp_diag.ko 2024-03-12T16:54:51.972 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/tcp_highspeed.ko 2024-03-12T16:54:51.972 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/tcp_htcp.ko 2024-03-12T16:54:51.972 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/tcp_hybla.ko 2024-03-12T16:54:51.972 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/tcp_illinois.ko 2024-03-12T16:54:51.972 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/tcp_lp.ko 2024-03-12T16:54:51.972 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/tcp_scalable.ko 2024-03-12T16:54:51.972 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/tcp_vegas.ko 2024-03-12T16:54:51.972 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/tcp_veno.ko 2024-03-12T16:54:51.972 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/tcp_westwood.ko 2024-03-12T16:54:51.972 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/tcp_yeah.ko 2024-03-12T16:54:51.972 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/tunnel4.ko 2024-03-12T16:54:51.972 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/udp_diag.ko 2024-03-12T16:54:51.972 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/udp_tunnel.ko 2024-03-12T16:54:51.972 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-12T16:54:51.973 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6 2024-03-12T16:54:51.973 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/ah6.ko 2024-03-12T16:54:51.973 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/esp6.ko 2024-03-12T16:54:51.973 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/fou6.ko 2024-03-12T16:54:51.973 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/ip6_tunnel.ko 2024-03-12T16:54:51.973 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-12T16:54:51.973 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/ip6_vti.ko 2024-03-12T16:54:51.973 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/ipcomp6.ko 2024-03-12T16:54:51.973 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter 2024-03-12T16:54:51.973 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-12T16:54:51.973 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-12T16:54:51.973 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-12T16:54:51.973 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-12T16:54:51.974 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-12T16:54:51.974 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-12T16:54:51.974 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-12T16:54:51.974 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-12T16:54:51.974 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-12T16:54:51.974 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-12T16:54:51.974 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-12T16:54:51.974 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-12T16:54:51.974 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-12T16:54:51.974 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-12T16:54:51.974 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-12T16:54:51.974 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-12T16:54:51.974 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-12T16:54:51.974 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-12T16:54:51.975 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-12T16:54:51.975 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-12T16:54:51.975 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-12T16:54:51.975 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-12T16:54:51.975 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-12T16:54:51.975 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-12T16:54:51.975 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/sit.ko 2024-03-12T16:54:51.975 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/tunnel6.ko 2024-03-12T16:54:51.975 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-12T16:54:51.975 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/key 2024-03-12T16:54:51.975 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/key/af_key.ko 2024-03-12T16:54:51.975 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/l2tp 2024-03-12T16:54:51.975 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/l2tp/l2tp_core.ko 2024-03-12T16:54:51.976 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-12T16:54:51.976 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/l2tp/l2tp_eth.ko 2024-03-12T16:54:51.976 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/l2tp/l2tp_ip.ko 2024-03-12T16:54:51.976 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/l2tp/l2tp_ip6.ko 2024-03-12T16:54:51.976 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/l2tp/l2tp_netlink.ko 2024-03-12T16:54:51.976 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/l2tp/l2tp_ppp.ko 2024-03-12T16:54:51.976 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/llc 2024-03-12T16:54:51.976 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/llc/llc.ko 2024-03-12T16:54:51.976 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/mac802154 2024-03-12T16:54:51.976 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/mac802154/mac802154.ko 2024-03-12T16:54:51.976 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/mpls 2024-03-12T16:54:51.976 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/mpls/mpls_gso.ko 2024-03-12T16:54:51.976 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter 2024-03-12T16:54:51.976 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipset 2024-03-12T16:54:51.977 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipset/ip_set.ko 2024-03-12T16:54:51.977 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-12T16:54:51.977 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-12T16:54:51.977 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-12T16:54:51.977 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-12T16:54:51.977 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-12T16:54:51.977 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-12T16:54:51.977 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-12T16:54:51.977 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-12T16:54:51.977 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-12T16:54:51.977 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-12T16:54:51.977 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-12T16:54:51.977 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-12T16:54:51.977 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-12T16:54:51.977 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-12T16:54:51.978 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-12T16:54:51.978 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipvs 2024-03-12T16:54:51.978 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-12T16:54:51.978 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-12T16:54:51.978 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-12T16:54:51.978 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-12T16:54:51.978 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-12T16:54:51.978 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-12T16:54:51.978 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-12T16:54:51.978 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-12T16:54:51.978 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-12T16:54:51.978 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-12T16:54:51.978 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-12T16:54:51.978 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-12T16:54:51.979 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-12T16:54:51.979 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-12T16:54:51.979 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_conncount.ko 2024-03-12T16:54:51.979 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_conntrack.ko 2024-03-12T16:54:51.979 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-12T16:54:51.979 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-12T16:54:51.979 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-12T16:54:51.979 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-12T16:54:51.979 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-12T16:54:51.979 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-12T16:54:51.979 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-12T16:54:51.979 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-12T16:54:51.979 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-12T16:54:51.980 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-12T16:54:51.980 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-12T16:54:51.980 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-12T16:54:51.980 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-12T16:54:51.980 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_flow_table.ko 2024-03-12T16:54:51.980 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-12T16:54:51.980 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_log_syslog.ko 2024-03-12T16:54:51.980 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_nat.ko 2024-03-12T16:54:51.980 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-12T16:54:51.980 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-12T16:54:51.980 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_nat_irc.ko 2024-03-12T16:54:51.980 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_nat_sip.ko 2024-03-12T16:54:51.980 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-12T16:54:51.980 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-12T16:54:51.981 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nf_tables.ko 2024-03-12T16:54:51.981 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nfnetlink.ko 2024-03-12T16:54:51.981 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-12T16:54:51.981 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nfnetlink_log.ko 2024-03-12T16:54:51.981 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-12T16:54:51.981 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-12T16:54:51.981 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_chain_nat.ko 2024-03-12T16:54:51.981 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_compat.ko 2024-03-12T16:54:51.981 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_ct.ko 2024-03-12T16:54:51.981 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-12T16:54:51.981 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_fib.ko 2024-03-12T16:54:51.981 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-12T16:54:51.981 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_hash.ko 2024-03-12T16:54:51.981 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_limit.ko 2024-03-12T16:54:51.982 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_log.ko 2024-03-12T16:54:51.982 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_masq.ko 2024-03-12T16:54:51.982 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_nat.ko 2024-03-12T16:54:51.982 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_numgen.ko 2024-03-12T16:54:51.982 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_queue.ko 2024-03-12T16:54:51.982 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_quota.ko 2024-03-12T16:54:51.982 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_redir.ko 2024-03-12T16:54:51.982 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_reject.ko 2024-03-12T16:54:51.982 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_reject_inet.ko 2024-03-12T16:54:51.982 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_socket.ko 2024-03-12T16:54:51.982 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/nft_tunnel.ko 2024-03-12T16:54:51.982 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_AUDIT.ko 2024-03-12T16:54:51.982 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-12T16:54:51.982 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-12T16:54:51.983 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-12T16:54:51.983 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_CT.ko 2024-03-12T16:54:51.983 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_DSCP.ko 2024-03-12T16:54:51.983 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_HL.ko 2024-03-12T16:54:51.983 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_HMARK.ko 2024-03-12T16:54:51.983 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-12T16:54:51.983 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_LED.ko 2024-03-12T16:54:51.983 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_LOG.ko 2024-03-12T16:54:51.983 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-12T16:54:51.983 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_NETMAP.ko 2024-03-12T16:54:51.983 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_NFLOG.ko 2024-03-12T16:54:51.983 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-12T16:54:51.983 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_RATEEST.ko 2024-03-12T16:54:51.984 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-12T16:54:51.984 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_SECMARK.ko 2024-03-12T16:54:51.984 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-12T16:54:51.984 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-12T16:54:51.984 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_TEE.ko 2024-03-12T16:54:51.984 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_TPROXY.ko 2024-03-12T16:54:51.984 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_TRACE.ko 2024-03-12T16:54:51.984 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_addrtype.ko 2024-03-12T16:54:51.984 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_bpf.ko 2024-03-12T16:54:51.984 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_cgroup.ko 2024-03-12T16:54:51.984 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_cluster.ko 2024-03-12T16:54:51.984 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_comment.ko 2024-03-12T16:54:51.984 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_connbytes.ko 2024-03-12T16:54:51.984 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_connlabel.ko 2024-03-12T16:54:51.985 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_connlimit.ko 2024-03-12T16:54:51.985 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_connmark.ko 2024-03-12T16:54:51.985 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_conntrack.ko 2024-03-12T16:54:51.985 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_cpu.ko 2024-03-12T16:54:51.985 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_dccp.ko 2024-03-12T16:54:51.985 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_devgroup.ko 2024-03-12T16:54:51.985 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_dscp.ko 2024-03-12T16:54:51.985 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_ecn.ko 2024-03-12T16:54:51.985 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_esp.ko 2024-03-12T16:54:51.985 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_hashlimit.ko 2024-03-12T16:54:51.985 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_helper.ko 2024-03-12T16:54:51.985 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_hl.ko 2024-03-12T16:54:51.985 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_ipcomp.ko 2024-03-12T16:54:51.985 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_iprange.ko 2024-03-12T16:54:51.986 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_ipvs.ko 2024-03-12T16:54:51.986 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_l2tp.ko 2024-03-12T16:54:51.986 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_length.ko 2024-03-12T16:54:51.986 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_limit.ko 2024-03-12T16:54:51.986 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_mac.ko 2024-03-12T16:54:51.986 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_mark.ko 2024-03-12T16:54:51.986 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_multiport.ko 2024-03-12T16:54:51.986 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_nat.ko 2024-03-12T16:54:51.986 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_nfacct.ko 2024-03-12T16:54:51.986 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_osf.ko 2024-03-12T16:54:51.986 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_owner.ko 2024-03-12T16:54:51.986 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_physdev.ko 2024-03-12T16:54:51.986 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_pkttype.ko 2024-03-12T16:54:51.986 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_policy.ko 2024-03-12T16:54:51.987 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_quota.ko 2024-03-12T16:54:51.987 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_rateest.ko 2024-03-12T16:54:51.987 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_realm.ko 2024-03-12T16:54:51.987 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_recent.ko 2024-03-12T16:54:51.987 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_sctp.ko 2024-03-12T16:54:51.987 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_set.ko 2024-03-12T16:54:51.987 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_socket.ko 2024-03-12T16:54:51.987 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_state.ko 2024-03-12T16:54:51.987 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_statistic.ko 2024-03-12T16:54:51.987 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_string.ko 2024-03-12T16:54:51.987 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_tcpmss.ko 2024-03-12T16:54:51.987 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_time.ko 2024-03-12T16:54:51.987 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netfilter/xt_u32.ko 2024-03-12T16:54:51.987 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netlink 2024-03-12T16:54:51.987 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/netlink/netlink_diag.ko 2024-03-12T16:54:51.988 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/nsh 2024-03-12T16:54:51.988 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/nsh/nsh.ko 2024-03-12T16:54:51.988 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/openvswitch 2024-03-12T16:54:51.988 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/openvswitch/openvswitch.ko 2024-03-12T16:54:51.988 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/openvswitch/vport-geneve.ko 2024-03-12T16:54:51.988 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/openvswitch/vport-gre.ko 2024-03-12T16:54:51.988 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/openvswitch/vport-vxlan.ko 2024-03-12T16:54:51.988 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/packet 2024-03-12T16:54:51.988 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/packet/af_packet_diag.ko 2024-03-12T16:54:51.988 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/rds 2024-03-12T16:54:51.988 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/rds/rds.ko 2024-03-12T16:54:51.988 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/rds/rds_rdma.ko 2024-03-12T16:54:51.988 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/rds/rds_tcp.ko 2024-03-12T16:54:51.988 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched 2024-03-12T16:54:51.988 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/act_csum.ko 2024-03-12T16:54:51.989 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/act_gact.ko 2024-03-12T16:54:51.989 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/act_mirred.ko 2024-03-12T16:54:51.989 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/act_nat.ko 2024-03-12T16:54:51.989 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/act_pedit.ko 2024-03-12T16:54:51.989 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/act_police.ko 2024-03-12T16:54:51.989 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/act_simple.ko 2024-03-12T16:54:51.989 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/act_skbedit.ko 2024-03-12T16:54:51.989 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/act_vlan.ko 2024-03-12T16:54:51.989 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/cls_basic.ko 2024-03-12T16:54:51.989 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/cls_bpf.ko 2024-03-12T16:54:51.989 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/cls_flow.ko 2024-03-12T16:54:51.989 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/cls_fw.ko 2024-03-12T16:54:51.989 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/cls_route.ko 2024-03-12T16:54:51.989 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/cls_u32.ko 2024-03-12T16:54:51.989 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/em_cmp.ko 2024-03-12T16:54:51.990 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/em_ipset.ko 2024-03-12T16:54:51.990 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/em_meta.ko 2024-03-12T16:54:51.990 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/em_nbyte.ko 2024-03-12T16:54:51.990 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/em_text.ko 2024-03-12T16:54:51.990 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/em_u32.ko 2024-03-12T16:54:51.990 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_choke.ko 2024-03-12T16:54:51.990 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_codel.ko 2024-03-12T16:54:51.990 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_drr.ko 2024-03-12T16:54:51.990 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_fq.ko 2024-03-12T16:54:51.990 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_gred.ko 2024-03-12T16:54:51.990 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_hfsc.ko 2024-03-12T16:54:51.990 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_hhf.ko 2024-03-12T16:54:51.991 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_htb.ko 2024-03-12T16:54:51.991 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_ingress.ko 2024-03-12T16:54:51.991 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_mqprio.ko 2024-03-12T16:54:51.991 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_mqprio_lib.ko 2024-03-12T16:54:51.991 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_multiq.ko 2024-03-12T16:54:51.991 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_netem.ko 2024-03-12T16:54:51.991 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_pie.ko 2024-03-12T16:54:51.991 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_plug.ko 2024-03-12T16:54:51.991 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_prio.ko 2024-03-12T16:54:51.991 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_qfq.ko 2024-03-12T16:54:51.991 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_red.ko 2024-03-12T16:54:51.991 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_sfb.ko 2024-03-12T16:54:51.991 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_sfq.ko 2024-03-12T16:54:51.992 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_tbf.ko 2024-03-12T16:54:51.992 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sched/sch_teql.ko 2024-03-12T16:54:51.992 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sctp 2024-03-12T16:54:51.992 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sctp/sctp.ko 2024-03-12T16:54:51.992 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sctp/sctp_diag.ko 2024-03-12T16:54:51.992 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sunrpc 2024-03-12T16:54:51.992 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sunrpc/auth_gss 2024-03-12T16:54:51.992 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-12T16:54:51.992 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-12T16:54:51.992 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sunrpc/sunrpc.ko 2024-03-12T16:54:51.992 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sunrpc/xprtrdma 2024-03-12T16:54:51.992 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-12T16:54:51.992 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/tipc 2024-03-12T16:54:51.992 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/tipc/diag.ko 2024-03-12T16:54:51.993 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/tipc/tipc.ko 2024-03-12T16:54:51.993 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/unix 2024-03-12T16:54:51.993 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/unix/unix_diag.ko 2024-03-12T16:54:51.993 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/vmw_vsock 2024-03-12T16:54:51.993 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-12T16:54:51.993 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-12T16:54:51.993 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/vmw_vsock/vsock.ko 2024-03-12T16:54:51.993 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-12T16:54:51.993 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-12T16:54:51.993 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/xfrm 2024-03-12T16:54:51.993 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-12T16:54:51.993 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/security 2024-03-12T16:54:51.993 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/security/keys 2024-03-12T16:54:51.993 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/security/keys/encrypted-keys 2024-03-12T16:54:51.994 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-12T16:54:51.994 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/security/keys/trusted-keys 2024-03-12T16:54:51.994 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/security/keys/trusted-keys/trusted.ko 2024-03-12T16:54:51.994 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/virt 2024-03-12T16:54:51.994 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/virt/lib 2024-03-12T16:54:51.994 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/kernel/virt/lib/irqbypass.ko 2024-03-12T16:54:51.994 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/modules.alias 2024-03-12T16:54:51.994 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/modules.alias.bin 2024-03-12T16:54:51.994 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/modules.builtin 2024-03-12T16:54:51.994 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/modules.builtin.alias.bin 2024-03-12T16:54:51.994 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/modules.builtin.bin 2024-03-12T16:54:51.994 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/modules.builtin.modinfo 2024-03-12T16:54:51.994 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/modules.dep 2024-03-12T16:54:51.995 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/modules.dep.bin 2024-03-12T16:54:51.995 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/modules.devname 2024-03-12T16:54:51.995 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/modules.order 2024-03-12T16:54:51.995 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/modules.softdep 2024-03-12T16:54:51.995 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/modules.symbols 2024-03-12T16:54:51.995 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/modules.symbols.bin 2024-03-12T16:54:51.995 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-rc7-gbf761b6efc7e/vmlinuz 2024-03-12T16:54:51.998 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-rc7-gbf761b6efc7e 2024-03-12T16:54:51.999 DEBUG:teuthology.orchestra.run.smithi052:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-12T16:54:52.023 INFO:teuthology.orchestra.run.smithi052.stdout:postinstall scriptlet (using /bin/sh): 2024-03-12T16:54:52.024 INFO:teuthology.orchestra.run.smithi052.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-12T16:54:52.024 INFO:teuthology.orchestra.run.smithi052.stdout: /usr/bin/kernel-install add 6.8.0-rc7-gbf761b6efc7e /lib/modules/6.8.0-rc7-gbf761b6efc7e/vmlinuz 2024-03-12T16:54:52.024 INFO:teuthology.orchestra.run.smithi052.stdout:fi 2024-03-12T16:54:52.024 INFO:teuthology.orchestra.run.smithi052.stdout:for file in vmlinuz System.map config; do 2024-03-12T16:54:52.024 INFO:teuthology.orchestra.run.smithi052.stdout: if ! cmp --silent "/lib/modules/6.8.0-rc7-gbf761b6efc7e/${file}" "/boot/${file}-6.8.0-rc7-gbf761b6efc7e"; then 2024-03-12T16:54:52.024 INFO:teuthology.orchestra.run.smithi052.stdout: cp "/lib/modules/6.8.0-rc7-gbf761b6efc7e/${file}" "/boot/${file}-6.8.0-rc7-gbf761b6efc7e" 2024-03-12T16:54:52.024 INFO:teuthology.orchestra.run.smithi052.stdout: fi 2024-03-12T16:54:52.024 INFO:teuthology.orchestra.run.smithi052.stdout:done 2024-03-12T16:54:52.024 INFO:teuthology.orchestra.run.smithi052.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-12T16:54:52.024 INFO:teuthology.orchestra.run.smithi052.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-12T16:54:52.024 INFO:teuthology.orchestra.run.smithi052.stdout:new-kernel-pkg --remove 6.8.0-rc7-gbf761b6efc7e --rminitrd --initrdfile=/boot/initramfs-6.8.0-rc7-gbf761b6efc7e.img 2024-03-12T16:54:52.024 INFO:teuthology.orchestra.run.smithi052.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-12T16:54:52.025 INFO:teuthology.orchestra.run.smithi052.stdout:kernel-install remove 6.8.0-rc7-gbf761b6efc7e 2024-03-12T16:54:52.025 INFO:teuthology.orchestra.run.smithi052.stdout:fi 2024-03-12T16:54:52.025 INFO:teuthology.orchestra.run.smithi052.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-12T16:54:52.025 INFO:teuthology.orchestra.run.smithi052.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-12T16:54:52.025 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/update-bootloader --remove 6.8.0-rc7-gbf761b6efc7e 2024-03-12T16:54:52.025 INFO:teuthology.orchestra.run.smithi052.stdout:fi 2024-03-12T16:54:52.026 DEBUG:teuthology.orchestra.run.smithi052:> sudo rpm -qi grub2-tools 2024-03-12T16:54:52.123 INFO:teuthology.orchestra.run.smithi052.stdout:Name : grub2-tools 2024-03-12T16:54:52.123 INFO:teuthology.orchestra.run.smithi052.stdout:Epoch : 1 2024-03-12T16:54:52.123 INFO:teuthology.orchestra.run.smithi052.stdout:Version : 2.06 2024-03-12T16:54:52.124 INFO:teuthology.orchestra.run.smithi052.stdout:Release : 68.el9 2024-03-12T16:54:52.124 INFO:teuthology.orchestra.run.smithi052.stdout:Architecture: x86_64 2024-03-12T16:54:52.124 INFO:teuthology.orchestra.run.smithi052.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-12T16:54:52.124 INFO:teuthology.orchestra.run.smithi052.stdout:Group : Unspecified 2024-03-12T16:54:52.124 INFO:teuthology.orchestra.run.smithi052.stdout:Size : 8264254 2024-03-12T16:54:52.124 INFO:teuthology.orchestra.run.smithi052.stdout:License : GPLv3+ 2024-03-12T16:54:52.124 INFO:teuthology.orchestra.run.smithi052.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-12T16:54:52.124 INFO:teuthology.orchestra.run.smithi052.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-12T16:54:52.124 INFO:teuthology.orchestra.run.smithi052.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-12T16:54:52.124 INFO:teuthology.orchestra.run.smithi052.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-12T16:54:52.124 INFO:teuthology.orchestra.run.smithi052.stdout:Packager : builder@centos.org 2024-03-12T16:54:52.125 INFO:teuthology.orchestra.run.smithi052.stdout:Vendor : CentOS 2024-03-12T16:54:52.125 INFO:teuthology.orchestra.run.smithi052.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-12T16:54:52.125 INFO:teuthology.orchestra.run.smithi052.stdout:Summary : Support tools for GRUB. 2024-03-12T16:54:52.125 INFO:teuthology.orchestra.run.smithi052.stdout:Description : 2024-03-12T16:54:52.125 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T16:54:52.125 INFO:teuthology.orchestra.run.smithi052.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-12T16:54:52.125 INFO:teuthology.orchestra.run.smithi052.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-12T16:54:52.125 INFO:teuthology.orchestra.run.smithi052.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-12T16:54:52.125 INFO:teuthology.orchestra.run.smithi052.stdout:hardware devices. 2024-03-12T16:54:52.125 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T16:54:52.125 INFO:teuthology.orchestra.run.smithi052.stdout:This subpackage provides tools for support of all platforms. 2024-03-12T16:54:52.126 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-12T16:54:52.126 INFO:teuthology.task.kernel:Updating grub on smithi052 to boot 6.8.0-rc7-gbf761b6efc7e 2024-03-12T16:54:52.126 DEBUG:teuthology.orchestra.run.smithi052:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-12T16:54:52.333 INFO:teuthology.orchestra.run.smithi102.stderr:Generating grub configuration file ... 2024-03-12T16:54:52.746 INFO:teuthology.orchestra.run.smithi052.stderr:Generating grub configuration file ... 2024-03-12T16:54:53.698 INFO:teuthology.orchestra.run.smithi102.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-12T16:54:53.741 INFO:teuthology.orchestra.run.smithi102.stderr:done 2024-03-12T16:54:53.744 DEBUG:teuthology.orchestra.run.smithi102:> mktemp 2024-03-12T16:54:53.758 INFO:teuthology.orchestra.run.smithi102.stdout:/tmp/tmp.dB5pgiDC29 2024-03-12T16:54:53.759 DEBUG:teuthology.orchestra.run.smithi102:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.dB5pgiDC29 2024-03-12T16:54:53.838 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 0666 /tmp/tmp.dB5pgiDC29 2024-03-12T16:54:53.979 DEBUG:teuthology.orchestra.remote:smithi102:/tmp/tmp.dB5pgiDC29 is 6KB 2024-03-12T16:54:54.028 DEBUG:teuthology.orchestra.run.smithi102:> rm -fr /tmp/tmp.dB5pgiDC29 2024-03-12T16:54:54.042 DEBUG:teuthology.orchestra.run.smithi102:> sudo /bin/ls /boot/loader/entries || true 2024-03-12T16:54:54.111 INFO:teuthology.orchestra.run.smithi102.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-12T16:54:54.112 INFO:teuthology.orchestra.run.smithi102.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-12T16:54:54.112 INFO:teuthology.orchestra.run.smithi102.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-12T16:54:54.112 INFO:teuthology.orchestra.run.smithi102.stdout:15949e3c88704300882482eddfd7baae-6.8.0-rc7-gbf761b6efc7e.conf 2024-03-12T16:54:54.113 DEBUG:teuthology.orchestra.run.smithi102:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-rc7-gbf761b6efc7e 2024-03-12T16:54:54.229 INFO:teuthology.orchestra.run.smithi052.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-12T16:54:54.256 INFO:teuthology.orchestra.run.smithi052.stderr:done 2024-03-12T16:54:54.268 DEBUG:teuthology.orchestra.run.smithi102:> sudo shutdown -r now 2024-03-12T16:54:54.271 DEBUG:teuthology.orchestra.run.smithi052:> mktemp 2024-03-12T16:54:54.287 INFO:teuthology.orchestra.run.smithi052.stdout:/tmp/tmp.TV9atddhtG 2024-03-12T16:54:54.287 DEBUG:teuthology.orchestra.run.smithi052:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.TV9atddhtG 2024-03-12T16:54:54.359 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 0666 /tmp/tmp.TV9atddhtG 2024-03-12T16:54:54.502 DEBUG:teuthology.orchestra.remote:smithi052:/tmp/tmp.TV9atddhtG is 6KB 2024-03-12T16:54:54.522 DEBUG:teuthology.orchestra.run.smithi052:> rm -fr /tmp/tmp.TV9atddhtG 2024-03-12T16:54:54.537 DEBUG:teuthology.orchestra.run.smithi052:> sudo /bin/ls /boot/loader/entries || true 2024-03-12T16:54:54.606 INFO:teuthology.orchestra.run.smithi052.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-12T16:54:54.606 INFO:teuthology.orchestra.run.smithi052.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-12T16:54:54.606 INFO:teuthology.orchestra.run.smithi052.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-12T16:54:54.606 INFO:teuthology.orchestra.run.smithi052.stdout:15949e3c88704300882482eddfd7baae-6.8.0-rc7-gbf761b6efc7e.conf 2024-03-12T16:54:54.611 DEBUG:teuthology.orchestra.run.smithi052:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-rc7-gbf761b6efc7e 2024-03-12T16:54:54.776 DEBUG:teuthology.orchestra.run.smithi052:> sudo shutdown -r now 2024-03-12T16:55:24.302 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-03-12T16:55:24.302 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-03-12T16:55:24.303 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T16:55:24.780 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-12T16:55:24.780 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-03-12T16:55:24.780 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T16:55:42.627 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.102 2024-03-12T16:55:43.134 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.52 2024-03-12T16:55:51.634 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-03-12T16:55:51.635 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T16:55:52.136 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-03-12T16:55:52.137 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T16:55:54.686 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.102 2024-03-12T16:55:55.198 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.52 2024-03-12T16:56:06.699 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-03-12T16:56:06.700 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T16:56:07.200 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-03-12T16:56:07.201 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T16:56:38.689 DEBUG:teuthology.orchestra.run.smithi102:> true 2024-03-12T16:56:39.134 DEBUG:teuthology.orchestra.run.smithi052:> true 2024-03-12T16:56:39.350 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-03-12T16:56:39.351 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.8.0-rc7-gbf761b6efc7e"... 2024-03-12T16:56:39.351 DEBUG:teuthology.orchestra.run.smithi102:> uname -r 2024-03-12T16:56:39.387 INFO:teuthology.orchestra.run.smithi102.stdout:6.8.0-rc7-gbf761b6efc7e 2024-03-12T16:56:39.387 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-rc7-gbf761b6efc7e vs 6.8.0-rc7-gbf761b6efc7e 2024-03-12T16:56:39.387 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-12T16:56:39.387 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-12T16:56:39.898 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-03-12T16:56:39.898 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.8.0-rc7-gbf761b6efc7e"... 2024-03-12T16:56:39.898 DEBUG:teuthology.orchestra.run.smithi052:> uname -r 2024-03-12T16:56:39.938 INFO:teuthology.orchestra.run.smithi052.stdout:6.8.0-rc7-gbf761b6efc7e 2024-03-12T16:56:39.938 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-rc7-gbf761b6efc7e vs 6.8.0-rc7-gbf761b6efc7e 2024-03-12T16:56:39.938 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-12T16:56:39.938 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-12T16:56:40.388 DEBUG:teuthology.parallel:result is None 2024-03-12T16:56:40.939 DEBUG:teuthology.parallel:result is None 2024-03-12T16:56:40.939 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-12T16:56:40.947 INFO:teuthology.task.internal:Creating test directory... 2024-03-12T16:56:40.947 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-12T16:56:40.955 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-12T16:56:41.004 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-12T16:56:41.011 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-12T16:56:41.018 INFO:teuthology.task.internal:Creating archive directory... 2024-03-12T16:56:41.018 DEBUG:teuthology.orchestra.run.smithi052:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-12T16:56:41.022 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-12T16:56:41.079 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-12T16:56:41.086 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-12T16:56:41.086 DEBUG:teuthology.orchestra.run.smithi052:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-12T16:56:41.108 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-12T16:56:41.221 INFO:teuthology.orchestra.run.smithi052.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-12T16:56:41.256 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-12T16:56:41.290 INFO:teuthology.orchestra.run.smithi052.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-12T16:56:41.335 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-12T16:56:41.339 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-12T16:56:41.347 INFO:teuthology.task.internal:Configuring sudo... 2024-03-12T16:56:41.347 DEBUG:teuthology.orchestra.run.smithi052:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-12T16:56:41.353 DEBUG:teuthology.orchestra.run.smithi102:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-12T16:56:41.426 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-12T16:56:41.435 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-12T16:56:41.435 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-12T16:56:41.439 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-12T16:56:41.480 DEBUG:teuthology.orchestra.run.smithi052:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-12T16:56:41.583 DEBUG:teuthology.orchestra.run.smithi052:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-12T16:56:41.700 DEBUG:teuthology.orchestra.run.smithi052:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-12T16:56:41.780 DEBUG:teuthology.orchestra.run.smithi052:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-12T16:56:41.858 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T16:56:41.858 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-12T16:56:41.953 DEBUG:teuthology.orchestra.run.smithi102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-12T16:56:42.025 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-12T16:56:42.106 DEBUG:teuthology.orchestra.run.smithi102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-12T16:56:42.174 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-12T16:56:42.271 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T16:56:42.271 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-12T16:56:42.353 DEBUG:teuthology.orchestra.run.smithi052:> sudo service rsyslog restart 2024-03-12T16:56:42.361 DEBUG:teuthology.orchestra.run.smithi102:> sudo service rsyslog restart 2024-03-12T16:56:42.465 INFO:teuthology.orchestra.run.smithi052.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-12T16:56:42.507 INFO:teuthology.orchestra.run.smithi102.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-12T16:56:42.744 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-12T16:56:42.752 INFO:teuthology.task.internal:Starting timer... 2024-03-12T16:56:42.752 INFO:teuthology.run_tasks:Running task pcp... 2024-03-12T16:56:42.823 INFO:teuthology.run_tasks:Running task selinux... 2024-03-12T16:56:42.866 DEBUG:teuthology.orchestra.run.smithi052:> sudo service auditd rotate 2024-03-12T16:56:42.981 INFO:teuthology.orchestra.run.smithi052.stdout:Rotating logs: 2024-03-12T16:56:42.986 DEBUG:teuthology.orchestra.run.smithi102:> sudo service auditd rotate 2024-03-12T16:56:43.104 INFO:teuthology.orchestra.run.smithi102.stdout:Rotating logs: 2024-03-12T16:56:43.108 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-12T16:56:43.108 DEBUG:teuthology.orchestra.run.smithi052:> /usr/sbin/getenforce 2024-03-12T16:56:43.154 INFO:teuthology.orchestra.run.smithi052.stdout:Permissive 2024-03-12T16:56:43.154 DEBUG:teuthology.orchestra.run.smithi102:> /usr/sbin/getenforce 2024-03-12T16:56:43.201 INFO:teuthology.orchestra.run.smithi102.stdout:Permissive 2024-03-12T16:56:43.201 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi052.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi102.front.sepia.ceph.com': 'permissive'} 2024-03-12T16:56:43.201 DEBUG:teuthology.orchestra.run.smithi052:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-12T16:56:43.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T16:56:43.277 DEBUG:teuthology.orchestra.run.smithi102:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-12T16:56:43.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T16:56:43.358 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-12T16:56:43.358 DEBUG:teuthology.orchestra.run.smithi052:> sudo /usr/sbin/setenforce permissive 2024-03-12T16:56:43.433 DEBUG:teuthology.orchestra.run.smithi102:> sudo /usr/sbin/setenforce permissive 2024-03-12T16:56:43.521 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-12T16:56:43.531 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-12T16:56:43.537 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-12T16:56:43.663 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-12T16:56:43.679 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-12T16:56:43.681 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi052.front.sepia.ceph.com,smithi102.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-12T17:02:49.242 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi052.front.sepia.ceph.com'), Remote(name='ubuntu@smithi102.front.sepia.ceph.com')] 2024-03-12T17:02:49.244 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-03-12T17:02:49.245 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T17:02:49.397 DEBUG:teuthology.orchestra.run.smithi052:> true 2024-03-12T17:02:49.555 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-03-12T17:02:49.556 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-03-12T17:02:49.556 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T17:02:49.696 DEBUG:teuthology.orchestra.run.smithi102:> true 2024-03-12T17:02:49.854 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-03-12T17:02:49.854 INFO:teuthology.run_tasks:Running task clock... 2024-03-12T17:02:49.868 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-12T17:02:49.868 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-12T17:02:49.869 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-12T17:02:49.876 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-12T17:02:49.876 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-12T17:02:49.976 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-12T17:02:49.977 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-12T17:02:50.028 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-12T17:02:50.035 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-12T17:02:50.119 INFO:teuthology.orchestra.run.smithi102.stderr:sudo: ntpd: command not found 2024-03-12T17:02:50.137 INFO:teuthology.orchestra.run.smithi052.stderr:sudo: ntpd: command not found 2024-03-12T17:02:50.158 INFO:teuthology.orchestra.run.smithi102.stdout:506 Cannot talk to daemon 2024-03-12T17:02:50.185 INFO:teuthology.orchestra.run.smithi052.stdout:506 Cannot talk to daemon 2024-03-12T17:02:50.208 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-12T17:02:50.240 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-12T17:02:50.261 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-12T17:02:50.297 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-12T17:02:50.387 INFO:teuthology.orchestra.run.smithi102.stderr:bash: line 1: ntpq: command not found 2024-03-12T17:02:50.394 INFO:teuthology.orchestra.run.smithi102.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-12T17:02:50.394 INFO:teuthology.orchestra.run.smithi102.stdout:=============================================================================== 2024-03-12T17:02:50.395 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T17:02:50.395 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T17:02:50.395 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T17:02:50.395 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T17:02:50.433 INFO:teuthology.orchestra.run.smithi052.stderr:bash: line 1: ntpq: command not found 2024-03-12T17:02:50.441 INFO:teuthology.orchestra.run.smithi052.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-12T17:02:50.441 INFO:teuthology.orchestra.run.smithi052.stdout:=============================================================================== 2024-03-12T17:02:50.441 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T17:02:50.441 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T17:02:50.441 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T17:02:50.442 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T17:02:50.443 INFO:teuthology.run_tasks:Running task install... 2024-03-12T17:02:50.451 DEBUG:teuthology.task.install:project ceph 2024-03-12T17:02:50.451 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '49e4a6794d5502bd66908faf714fc54896c51712'}} 2024-03-12T17:02:50.451 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '49e4a6794d5502bd66908faf714fc54896c51712'} 2024-03-12T17:02:50.452 INFO:teuthology.task.install:Using flavor: default 2024-03-12T17:02:50.458 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-12T17:02:50.458 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-03-12T17:02:50.458 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '49e4a6794d5502bd66908faf714fc54896c51712', 'tag': None, 'wait_for_package': False} 2024-03-12T17:02:50.459 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=49e4a6794d5502bd66908faf714fc54896c51712 2024-03-12T17:02:50.461 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '49e4a6794d5502bd66908faf714fc54896c51712', 'tag': None, 'wait_for_package': False} 2024-03-12T17:02:50.461 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=49e4a6794d5502bd66908faf714fc54896c51712 2024-03-12T17:02:50.615 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-11-0838-reef/49e4a6794d5502bd66908faf714fc54896c51712/centos/9/flavors/default/ 2024-03-12T17:02:50.615 INFO:teuthology.task.install.rpm:Package version is 18.2.2-797.g49e4a679 2024-03-12T17:02:50.728 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-11-0838-reef/49e4a6794d5502bd66908faf714fc54896c51712/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-11-0838-reef/49e4a6794d5502bd66908faf714fc54896c51712/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-11-0838-reef/49e4a6794d5502bd66908faf714fc54896c51712/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-12T17:02:50.728 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T17:02:50.728 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-12T17:02:50.744 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-11-0838-reef/49e4a6794d5502bd66908faf714fc54896c51712/centos/9/flavors/default/ 2024-03-12T17:02:50.744 INFO:teuthology.task.install.rpm:Package version is 18.2.2-797.g49e4a679 2024-03-12T17:02:50.829 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-12T17:02:50.829 DEBUG:teuthology.orchestra.run.smithi102:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/49e4a6794d5502bd66908faf714fc54896c51712/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-12T17:02:50.857 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-11-0838-reef/49e4a6794d5502bd66908faf714fc54896c51712/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-11-0838-reef/49e4a6794d5502bd66908faf714fc54896c51712/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-11-0838-reef/49e4a6794d5502bd66908faf714fc54896c51712/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-12T17:02:50.857 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:02:50.857 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-12T17:02:50.966 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-12T17:02:50.966 DEBUG:teuthology.orchestra.run.smithi052:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/49e4a6794d5502bd66908faf714fc54896c51712/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-12T17:02:50.967 DEBUG:teuthology.orchestra.run.smithi102:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-12T17:02:51.102 DEBUG:teuthology.orchestra.run.smithi102:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-12T17:02:51.110 DEBUG:teuthology.orchestra.run.smithi052:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-12T17:02:51.191 INFO:teuthology.orchestra.run.smithi102.stdout:check_obsoletes = 1 2024-03-12T17:02:51.195 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum clean all 2024-03-12T17:02:51.255 DEBUG:teuthology.orchestra.run.smithi052:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-12T17:02:51.352 INFO:teuthology.orchestra.run.smithi052.stdout:check_obsoletes = 1 2024-03-12T17:02:51.357 DEBUG:teuthology.orchestra.run.smithi052:> sudo yum clean all 2024-03-12T17:02:51.557 INFO:teuthology.orchestra.run.smithi102.stdout:74 files removed 2024-03-12T17:02:51.600 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-12T17:02:51.726 INFO:teuthology.orchestra.run.smithi052.stdout:74 files removed 2024-03-12T17:02:51.765 DEBUG:teuthology.orchestra.run.smithi052:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-12T17:02:52.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph packages for x86_64 164 kB/s | 77 kB 00:00 2024-03-12T17:02:52.796 INFO:teuthology.orchestra.run.smithi052.stdout:ceph packages for x86_64 151 kB/s | 77 kB 00:00 2024-03-12T17:02:53.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph noarch packages 24 kB/s | 9.9 kB 00:00 2024-03-12T17:02:53.396 INFO:teuthology.orchestra.run.smithi052.stdout:ceph noarch packages 20 kB/s | 9.9 kB 00:00 2024-03-12T17:02:53.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph source packages 3.8 kB/s | 1.9 kB 00:00 2024-03-12T17:02:53.872 INFO:teuthology.orchestra.run.smithi052.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2024-03-12T17:02:54.592 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS-9 - AppStream 10 MB/s | 6.7 MB 00:00 2024-03-12T17:02:55.132 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS-9 - AppStream 4.8 MB/s | 6.7 MB 00:01 2024-03-12T17:02:56.343 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS-9 - BaseOS 4.5 MB/s | 2.0 MB 00:00 2024-03-12T17:02:56.779 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS-9 - BaseOS 6.1 MB/s | 2.0 MB 00:00 2024-03-12T17:03:04.292 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - BaseOS 1.1 MB/s | 8.0 MB 00:07 2024-03-12T17:03:04.322 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - BaseOS 1.1 MB/s | 8.0 MB 00:07 2024-03-12T17:03:13.116 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - AppStream 2.5 MB/s | 19 MB 00:07 2024-03-12T17:03:16.872 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - AppStream 1.7 MB/s | 19 MB 00:11 2024-03-12T17:03:23.258 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - CRB 994 kB/s | 6.1 MB 00:06 2024-03-12T17:03:25.161 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 15 kB 00:00 2024-03-12T17:03:26.485 INFO:teuthology.orchestra.run.smithi102.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-03-12T17:03:27.085 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - CRB 1.0 MB/s | 6.1 MB 00:06 2024-03-12T17:03:29.028 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 15 kB 00:00 2024-03-12T17:03:30.894 INFO:teuthology.orchestra.run.smithi052.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-03-12T17:03:32.210 INFO:teuthology.orchestra.run.smithi102.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-03-12T17:03:34.928 INFO:teuthology.orchestra.run.smithi102.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-12T17:03:34.930 INFO:teuthology.orchestra.run.smithi102.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-12T17:03:34.952 INFO:teuthology.orchestra.run.smithi102.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-12T17:03:34.953 INFO:teuthology.orchestra.run.smithi102.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-12T17:03:34.954 INFO:teuthology.orchestra.run.smithi102.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-12T17:03:34.956 INFO:teuthology.orchestra.run.smithi102.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-12T17:03:34.958 INFO:teuthology.orchestra.run.smithi102.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-12T17:03:34.962 INFO:teuthology.orchestra.run.smithi102.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-12T17:03:34.964 INFO:teuthology.orchestra.run.smithi102.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-12T17:03:35.048 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-03-12T17:03:35.065 INFO:teuthology.orchestra.run.smithi102.stdout:=========================================================================================== 2024-03-12T17:03:35.066 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-03-12T17:03:35.066 INFO:teuthology.orchestra.run.smithi102.stdout:=========================================================================================== 2024-03-12T17:03:35.066 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-03-12T17:03:35.066 INFO:teuthology.orchestra.run.smithi102.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-12T17:03:35.066 INFO:teuthology.orchestra.run.smithi102.stdout: ceph x86_64 2:18.2.2-797.g49e4a679.el9 ceph 6.5 k 2024-03-12T17:03:35.066 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base x86_64 2:18.2.2-797.g49e4a679.el9 ceph 5.2 M 2024-03-12T17:03:35.066 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse x86_64 2:18.2.2-797.g49e4a679.el9 ceph 844 k 2024-03-12T17:03:35.066 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-797.g49e4a679.el9 ceph 143 k 2024-03-12T17:03:35.066 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr x86_64 2:18.2.2-797.g49e4a679.el9 ceph 1.4 M 2024-03-12T17:03:35.066 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-cephadm noarch 2:18.2.2-797.g49e4a679.el9 ceph-noarch 137 k 2024-03-12T17:03:35.066 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-dashboard noarch 2:18.2.2-797.g49e4a679.el9 ceph-noarch 1.7 M 2024-03-12T17:03:35.067 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-797.g49e4a679.el9 ceph-noarch 7.4 M 2024-03-12T17:03:35.067 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-rook noarch 2:18.2.2-797.g49e4a679.el9 ceph-noarch 49 k 2024-03-12T17:03:35.067 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw x86_64 2:18.2.2-797.g49e4a679.el9 ceph 7.7 M 2024-03-12T17:03:35.067 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test x86_64 2:18.2.2-797.g49e4a679.el9 ceph 40 M 2024-03-12T17:03:35.067 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-volume noarch 2:18.2.2-797.g49e4a679.el9 ceph-noarch 264 k 2024-03-12T17:03:35.067 INFO:teuthology.orchestra.run.smithi102.stdout: cephadm noarch 2:18.2.2-797.g49e4a679.el9 ceph-noarch 223 k 2024-03-12T17:03:35.067 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-mirror x86_64 2:18.2.2-797.g49e4a679.el9 ceph 221 k 2024-03-12T17:03:35.067 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-top noarch 2:18.2.2-797.g49e4a679.el9 ceph-noarch 18 k 2024-03-12T17:03:35.067 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-12T17:03:35.067 INFO:teuthology.orchestra.run.smithi102.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-12T17:03:35.067 INFO:teuthology.orchestra.run.smithi102.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-12T17:03:35.067 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs-devel x86_64 2:18.2.2-797.g49e4a679.el9 ceph 31 k 2024-03-12T17:03:35.067 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2 x86_64 2:18.2.2-797.g49e4a679.el9 ceph 702 k 2024-03-12T17:03:35.067 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel x86_64 2:18.2.2-797.g49e4a679.el9 ceph 127 k 2024-03-12T17:03:35.068 INFO:teuthology.orchestra.run.smithi102.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-12T17:03:35.068 INFO:teuthology.orchestra.run.smithi102.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-12T17:03:35.068 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-12T17:03:35.068 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-12T17:03:35.068 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-12T17:03:35.068 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-12T17:03:35.068 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs x86_64 2:18.2.2-797.g49e4a679.el9 ceph 161 k 2024-03-12T17:03:35.068 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados x86_64 2:18.2.2-797.g49e4a679.el9 ceph 321 k 2024-03-12T17:03:35.068 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd x86_64 2:18.2.2-797.g49e4a679.el9 ceph 298 k 2024-03-12T17:03:35.068 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw x86_64 2:18.2.2-797.g49e4a679.el9 ceph 99 k 2024-03-12T17:03:35.068 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse x86_64 2:18.2.2-797.g49e4a679.el9 ceph 87 k 2024-03-12T17:03:35.068 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror x86_64 2:18.2.2-797.g49e4a679.el9 ceph 3.0 M 2024-03-12T17:03:35.068 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd x86_64 2:18.2.2-797.g49e4a679.el9 ceph 172 k 2024-03-12T17:03:35.068 INFO:teuthology.orchestra.run.smithi102.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-12T17:03:35.069 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrading: 2024-03-12T17:03:35.069 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-12T17:03:35.069 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-12T17:03:35.069 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-12T17:03:35.069 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-12T17:03:35.069 INFO:teuthology.orchestra.run.smithi102.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-12T17:03:35.069 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-12T17:03:35.069 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-12T17:03:35.069 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-12T17:03:35.069 INFO:teuthology.orchestra.run.smithi102.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-12T17:03:35.069 INFO:teuthology.orchestra.run.smithi102.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-12T17:03:35.069 INFO:teuthology.orchestra.run.smithi102.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-12T17:03:35.070 INFO:teuthology.orchestra.run.smithi102.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-12T17:03:35.070 INFO:teuthology.orchestra.run.smithi102.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-12T17:03:35.070 INFO:teuthology.orchestra.run.smithi102.stdout: librados2 x86_64 2:18.2.2-797.g49e4a679.el9 ceph 3.3 M 2024-03-12T17:03:35.070 INFO:teuthology.orchestra.run.smithi102.stdout: librbd1 x86_64 2:18.2.2-797.g49e4a679.el9 ceph 3.0 M 2024-03-12T17:03:35.070 INFO:teuthology.orchestra.run.smithi102.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-12T17:03:35.070 INFO:teuthology.orchestra.run.smithi102.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-12T17:03:35.070 INFO:teuthology.orchestra.run.smithi102.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-12T17:03:35.070 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-12T17:03:35.070 INFO:teuthology.orchestra.run.smithi102.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-12T17:03:35.070 INFO:teuthology.orchestra.run.smithi102.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-12T17:03:35.070 INFO:teuthology.orchestra.run.smithi102.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-12T17:03:35.070 INFO:teuthology.orchestra.run.smithi102.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-12T17:03:35.071 INFO:teuthology.orchestra.run.smithi102.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-12T17:03:35.071 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-03-12T17:03:35.071 INFO:teuthology.orchestra.run.smithi102.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-12T17:03:35.071 INFO:teuthology.orchestra.run.smithi102.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-12T17:03:35.071 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common x86_64 2:18.2.2-797.g49e4a679.el9 ceph 18 M 2024-03-12T17:03:35.071 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-grafana-dashboards noarch 2:18.2.2-797.g49e4a679.el9 ceph-noarch 23 k 2024-03-12T17:03:35.071 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds x86_64 2:18.2.2-797.g49e4a679.el9 ceph 2.1 M 2024-03-12T17:03:35.071 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-modules-core noarch 2:18.2.2-797.g49e4a679.el9 ceph-noarch 245 k 2024-03-12T17:03:35.071 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon x86_64 2:18.2.2-797.g49e4a679.el9 ceph 4.4 M 2024-03-12T17:03:35.071 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd x86_64 2:18.2.2-797.g49e4a679.el9 ceph 18 M 2024-03-12T17:03:35.071 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-prometheus-alerts noarch 2:18.2.2-797.g49e4a679.el9 ceph-noarch 15 k 2024-03-12T17:03:35.071 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-selinux x86_64 2:18.2.2-797.g49e4a679.el9 ceph 25 k 2024-03-12T17:03:35.072 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-12T17:03:35.072 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-12T17:03:35.072 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-12T17:03:35.072 INFO:teuthology.orchestra.run.smithi102.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-03-12T17:03:35.072 INFO:teuthology.orchestra.run.smithi102.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-12T17:03:35.072 INFO:teuthology.orchestra.run.smithi102.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-12T17:03:35.072 INFO:teuthology.orchestra.run.smithi102.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-12T17:03:35.072 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-12T17:03:35.072 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-12T17:03:35.072 INFO:teuthology.orchestra.run.smithi102.stdout: libcephsqlite x86_64 2:18.2.2-797.g49e4a679.el9 ceph 166 k 2024-03-12T17:03:35.072 INFO:teuthology.orchestra.run.smithi102.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-12T17:03:35.072 INFO:teuthology.orchestra.run.smithi102.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-12T17:03:35.072 INFO:teuthology.orchestra.run.smithi102.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-12T17:03:35.073 INFO:teuthology.orchestra.run.smithi102.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-12T17:03:35.073 INFO:teuthology.orchestra.run.smithi102.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-12T17:03:35.073 INFO:teuthology.orchestra.run.smithi102.stdout: libradosstriper1 x86_64 2:18.2.2-797.g49e4a679.el9 ceph 475 k 2024-03-12T17:03:35.073 INFO:teuthology.orchestra.run.smithi102.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-12T17:03:35.073 INFO:teuthology.orchestra.run.smithi102.stdout: librgw2 x86_64 2:18.2.2-797.g49e4a679.el9 ceph 4.5 M 2024-03-12T17:03:35.073 INFO:teuthology.orchestra.run.smithi102.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-12T17:03:35.073 INFO:teuthology.orchestra.run.smithi102.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-12T17:03:35.073 INFO:teuthology.orchestra.run.smithi102.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-12T17:03:35.073 INFO:teuthology.orchestra.run.smithi102.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-12T17:03:35.073 INFO:teuthology.orchestra.run.smithi102.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-12T17:03:35.073 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-12T17:03:35.073 INFO:teuthology.orchestra.run.smithi102.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-12T17:03:35.074 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-12T17:03:35.074 INFO:teuthology.orchestra.run.smithi102.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-12T17:03:35.074 INFO:teuthology.orchestra.run.smithi102.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-12T17:03:35.074 INFO:teuthology.orchestra.run.smithi102.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-12T17:03:35.074 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-12T17:03:35.074 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-argparse x86_64 2:18.2.2-797.g49e4a679.el9 ceph 45 k 2024-03-12T17:03:35.074 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-common x86_64 2:18.2.2-797.g49e4a679.el9 ceph 126 k 2024-03-12T17:03:35.074 INFO:teuthology.orchestra.run.smithi102.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-12T17:03:35.074 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-12T17:03:35.074 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-12T17:03:35.075 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-12T17:03:35.075 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-12T17:03:35.075 INFO:teuthology.orchestra.run.smithi102.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-12T17:03:35.075 INFO:teuthology.orchestra.run.smithi102.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-12T17:03:35.075 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-12T17:03:35.075 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-12T17:03:35.075 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-12T17:03:35.075 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-12T17:03:35.075 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-12T17:03:35.075 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-12T17:03:35.075 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-12T17:03:35.075 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-12T17:03:35.076 INFO:teuthology.orchestra.run.smithi102.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-12T17:03:35.076 INFO:teuthology.orchestra.run.smithi102.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-12T17:03:35.076 INFO:teuthology.orchestra.run.smithi102.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-12T17:03:35.076 INFO:teuthology.orchestra.run.smithi102.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-12T17:03:35.076 INFO:teuthology.orchestra.run.smithi102.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-12T17:03:35.076 INFO:teuthology.orchestra.run.smithi102.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-12T17:03:35.076 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-12T17:03:35.076 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-12T17:03:35.076 INFO:teuthology.orchestra.run.smithi102.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-12T17:03:35.076 INFO:teuthology.orchestra.run.smithi102.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-12T17:03:35.076 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-12T17:03:35.076 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-12T17:03:35.076 INFO:teuthology.orchestra.run.smithi102.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-12T17:03:35.076 INFO:teuthology.orchestra.run.smithi102.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-12T17:03:35.077 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-12T17:03:35.077 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-12T17:03:35.077 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-12T17:03:35.077 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-12T17:03:35.077 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-12T17:03:35.077 INFO:teuthology.orchestra.run.smithi102.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-12T17:03:35.077 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-12T17:03:35.077 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-12T17:03:35.077 INFO:teuthology.orchestra.run.smithi102.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-12T17:03:35.077 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-12T17:03:35.077 INFO:teuthology.orchestra.run.smithi102.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-12T17:03:35.077 INFO:teuthology.orchestra.run.smithi102.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-12T17:03:35.077 INFO:teuthology.orchestra.run.smithi102.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-12T17:03:35.077 INFO:teuthology.orchestra.run.smithi102.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-12T17:03:35.078 INFO:teuthology.orchestra.run.smithi102.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-12T17:03:35.078 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-12T17:03:35.078 INFO:teuthology.orchestra.run.smithi102.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-12T17:03:35.078 INFO:teuthology.orchestra.run.smithi102.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-12T17:03:35.078 INFO:teuthology.orchestra.run.smithi102.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-12T17:03:35.078 INFO:teuthology.orchestra.run.smithi102.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-12T17:03:35.078 INFO:teuthology.orchestra.run.smithi102.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-12T17:03:35.078 INFO:teuthology.orchestra.run.smithi102.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-12T17:03:35.078 INFO:teuthology.orchestra.run.smithi102.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-12T17:03:35.078 INFO:teuthology.orchestra.run.smithi102.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-12T17:03:35.078 INFO:teuthology.orchestra.run.smithi102.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-12T17:03:35.078 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:03:35.078 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-03-12T17:03:35.078 INFO:teuthology.orchestra.run.smithi102.stdout:=========================================================================================== 2024-03-12T17:03:35.079 INFO:teuthology.orchestra.run.smithi102.stdout:Install 132 Packages 2024-03-12T17:03:35.079 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrade 24 Packages 2024-03-12T17:03:35.079 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:03:35.079 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 204 M 2024-03-12T17:03:35.079 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-03-12T17:03:35.611 INFO:teuthology.orchestra.run.smithi102.stdout:(1/156): ceph-18.2.2-797.g49e4a679.el9.x86_64.r 20 kB/s | 6.5 kB 00:00 2024-03-12T17:03:35.987 INFO:teuthology.orchestra.run.smithi102.stdout:(2/156): ceph-base-18.2.2-797.g49e4a679.el9.x86 7.5 MB/s | 5.2 MB 00:00 2024-03-12T17:03:36.037 INFO:teuthology.orchestra.run.smithi102.stdout:(3/156): ceph-fuse-18.2.2-797.g49e4a679.el9.x86 1.9 MB/s | 844 kB 00:00 2024-03-12T17:03:36.071 INFO:teuthology.orchestra.run.smithi102.stdout:(4/156): ceph-immutable-object-cache-18.2.2-797 1.7 MB/s | 143 kB 00:00 2024-03-12T17:03:36.213 INFO:teuthology.orchestra.run.smithi102.stdout:(5/156): ceph-mgr-18.2.2-797.g49e4a679.el9.x86_ 10 MB/s | 1.4 MB 00:00 2024-03-12T17:03:36.488 INFO:teuthology.orchestra.run.smithi102.stdout:(6/156): ceph-mds-18.2.2-797.g49e4a679.el9.x86_ 4.7 MB/s | 2.1 MB 00:00 2024-03-12T17:03:36.614 INFO:teuthology.orchestra.run.smithi102.stdout:(7/156): ceph-mon-18.2.2-797.g49e4a679.el9.x86_ 11 MB/s | 4.4 MB 00:00 2024-03-12T17:03:36.821 INFO:teuthology.orchestra.run.smithi052.stdout:lab-extras 42 kB/s | 1.7 kB 00:00 2024-03-12T17:03:37.314 INFO:teuthology.orchestra.run.smithi102.stdout:(8/156): ceph-radosgw-18.2.2-797.g49e4a679.el9. 11 MB/s | 7.7 MB 00:00 2024-03-12T17:03:37.389 INFO:teuthology.orchestra.run.smithi102.stdout:(9/156): ceph-selinux-18.2.2-797.g49e4a679.el9. 340 kB/s | 25 kB 00:00 2024-03-12T17:03:37.989 INFO:teuthology.orchestra.run.smithi102.stdout:(10/156): ceph-common-18.2.2-797.g49e4a679.el9. 6.8 MB/s | 18 MB 00:02 2024-03-12T17:03:38.065 INFO:teuthology.orchestra.run.smithi102.stdout:(11/156): cephfs-mirror-18.2.2-797.g49e4a679.el 2.9 MB/s | 221 kB 00:00 2024-03-12T17:03:38.198 INFO:teuthology.orchestra.run.smithi102.stdout:(12/156): libcephfs-devel-18.2.2-797.g49e4a679. 234 kB/s | 31 kB 00:00 2024-03-12T17:03:38.390 INFO:teuthology.orchestra.run.smithi102.stdout:(13/156): libcephfs2-18.2.2-797.g49e4a679.el9.x 3.6 MB/s | 702 kB 00:00 2024-03-12T17:03:38.632 INFO:teuthology.orchestra.run.smithi102.stdout:(14/156): ceph-osd-18.2.2-797.g49e4a679.el9.x86 8.2 MB/s | 18 MB 00:02 2024-03-12T17:03:38.657 INFO:teuthology.orchestra.run.smithi102.stdout:(15/156): libcephsqlite-18.2.2-797.g49e4a679.el 624 kB/s | 166 kB 00:00 2024-03-12T17:03:38.708 INFO:teuthology.orchestra.run.smithi102.stdout:(16/156): librados-devel-18.2.2-797.g49e4a679.e 1.7 MB/s | 127 kB 00:00 2024-03-12T17:03:38.750 INFO:teuthology.orchestra.run.smithi102.stdout:(17/156): libradosstriper1-18.2.2-797.g49e4a679 5.1 MB/s | 475 kB 00:00 2024-03-12T17:03:38.842 INFO:teuthology.orchestra.run.smithi102.stdout:(18/156): python3-ceph-argparse-18.2.2-797.g49e 496 kB/s | 45 kB 00:00 2024-03-12T17:03:38.942 INFO:teuthology.orchestra.run.smithi102.stdout:(19/156): python3-ceph-common-18.2.2-797.g49e4a 1.2 MB/s | 126 kB 00:00 2024-03-12T17:03:39.051 INFO:teuthology.orchestra.run.smithi102.stdout:(20/156): python3-cephfs-18.2.2-797.g49e4a679.e 1.5 MB/s | 161 kB 00:00 2024-03-12T17:03:39.193 INFO:teuthology.orchestra.run.smithi102.stdout:(21/156): librgw2-18.2.2-797.g49e4a679.el9.x86_ 9.2 MB/s | 4.5 MB 00:00 2024-03-12T17:03:39.235 INFO:teuthology.orchestra.run.smithi102.stdout:(22/156): python3-rados-18.2.2-797.g49e4a679.el 1.7 MB/s | 321 kB 00:00 2024-03-12T17:03:39.276 INFO:teuthology.orchestra.run.smithi102.stdout:(23/156): python3-rbd-18.2.2-797.g49e4a679.el9. 3.5 MB/s | 298 kB 00:00 2024-03-12T17:03:39.302 INFO:teuthology.orchestra.run.smithi102.stdout:(24/156): python3-rgw-18.2.2-797.g49e4a679.el9. 1.5 MB/s | 99 kB 00:00 2024-03-12T17:03:39.344 INFO:teuthology.orchestra.run.smithi102.stdout:(25/156): rbd-fuse-18.2.2-797.g49e4a679.el9.x86 1.3 MB/s | 87 kB 00:00 2024-03-12T17:03:39.459 INFO:teuthology.orchestra.run.smithi102.stdout:(26/156): rbd-nbd-18.2.2-797.g49e4a679.el9.x86_ 1.5 MB/s | 172 kB 00:00 2024-03-12T17:03:39.569 INFO:teuthology.orchestra.run.smithi102.stdout:(27/156): ceph-grafana-dashboards-18.2.2-797.g4 214 kB/s | 23 kB 00:00 2024-03-12T17:03:39.587 INFO:teuthology.orchestra.run.smithi052.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-12T17:03:39.589 INFO:teuthology.orchestra.run.smithi052.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-12T17:03:39.608 INFO:teuthology.orchestra.run.smithi052.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-12T17:03:39.609 INFO:teuthology.orchestra.run.smithi052.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-12T17:03:39.610 INFO:teuthology.orchestra.run.smithi052.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-12T17:03:39.612 INFO:teuthology.orchestra.run.smithi052.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-12T17:03:39.615 INFO:teuthology.orchestra.run.smithi052.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-12T17:03:39.618 INFO:teuthology.orchestra.run.smithi052.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-12T17:03:39.619 INFO:teuthology.orchestra.run.smithi052.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-12T17:03:39.703 INFO:teuthology.orchestra.run.smithi102.stdout:(28/156): rbd-mirror-18.2.2-797.g49e4a679.el9.x 7.5 MB/s | 3.0 MB 00:00 2024-03-12T17:03:39.712 INFO:teuthology.orchestra.run.smithi052.stdout:Dependencies resolved. 2024-03-12T17:03:39.728 INFO:teuthology.orchestra.run.smithi102.stdout:(29/156): ceph-mgr-cephadm-18.2.2-797.g49e4a679 870 kB/s | 137 kB 00:00 2024-03-12T17:03:39.736 INFO:teuthology.orchestra.run.smithi052.stdout:=========================================================================================== 2024-03-12T17:03:39.736 INFO:teuthology.orchestra.run.smithi052.stdout: Package Arch Version Repository Size 2024-03-12T17:03:39.736 INFO:teuthology.orchestra.run.smithi052.stdout:=========================================================================================== 2024-03-12T17:03:39.736 INFO:teuthology.orchestra.run.smithi052.stdout:Installing: 2024-03-12T17:03:39.736 INFO:teuthology.orchestra.run.smithi052.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-12T17:03:39.736 INFO:teuthology.orchestra.run.smithi052.stdout: ceph x86_64 2:18.2.2-797.g49e4a679.el9 ceph 6.5 k 2024-03-12T17:03:39.736 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-base x86_64 2:18.2.2-797.g49e4a679.el9 ceph 5.2 M 2024-03-12T17:03:39.737 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-fuse x86_64 2:18.2.2-797.g49e4a679.el9 ceph 844 k 2024-03-12T17:03:39.737 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-797.g49e4a679.el9 ceph 143 k 2024-03-12T17:03:39.737 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr x86_64 2:18.2.2-797.g49e4a679.el9 ceph 1.4 M 2024-03-12T17:03:39.737 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-cephadm noarch 2:18.2.2-797.g49e4a679.el9 ceph-noarch 137 k 2024-03-12T17:03:39.737 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-dashboard noarch 2:18.2.2-797.g49e4a679.el9 ceph-noarch 1.7 M 2024-03-12T17:03:39.737 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-797.g49e4a679.el9 ceph-noarch 7.4 M 2024-03-12T17:03:39.737 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-rook noarch 2:18.2.2-797.g49e4a679.el9 ceph-noarch 49 k 2024-03-12T17:03:39.737 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-radosgw x86_64 2:18.2.2-797.g49e4a679.el9 ceph 7.7 M 2024-03-12T17:03:39.737 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-test x86_64 2:18.2.2-797.g49e4a679.el9 ceph 40 M 2024-03-12T17:03:39.737 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-volume noarch 2:18.2.2-797.g49e4a679.el9 ceph-noarch 264 k 2024-03-12T17:03:39.737 INFO:teuthology.orchestra.run.smithi052.stdout: cephadm noarch 2:18.2.2-797.g49e4a679.el9 ceph-noarch 223 k 2024-03-12T17:03:39.737 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-mirror x86_64 2:18.2.2-797.g49e4a679.el9 ceph 221 k 2024-03-12T17:03:39.737 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-top noarch 2:18.2.2-797.g49e4a679.el9 ceph-noarch 18 k 2024-03-12T17:03:39.737 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-12T17:03:39.737 INFO:teuthology.orchestra.run.smithi052.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-12T17:03:39.738 INFO:teuthology.orchestra.run.smithi052.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-12T17:03:39.738 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs-devel x86_64 2:18.2.2-797.g49e4a679.el9 ceph 31 k 2024-03-12T17:03:39.738 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs2 x86_64 2:18.2.2-797.g49e4a679.el9 ceph 702 k 2024-03-12T17:03:39.738 INFO:teuthology.orchestra.run.smithi052.stdout: librados-devel x86_64 2:18.2.2-797.g49e4a679.el9 ceph 127 k 2024-03-12T17:03:39.738 INFO:teuthology.orchestra.run.smithi052.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-12T17:03:39.738 INFO:teuthology.orchestra.run.smithi052.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-12T17:03:39.738 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-12T17:03:39.738 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-12T17:03:39.738 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-12T17:03:39.738 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-12T17:03:39.738 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cephfs x86_64 2:18.2.2-797.g49e4a679.el9 ceph 161 k 2024-03-12T17:03:39.738 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rados x86_64 2:18.2.2-797.g49e4a679.el9 ceph 321 k 2024-03-12T17:03:39.739 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rbd x86_64 2:18.2.2-797.g49e4a679.el9 ceph 298 k 2024-03-12T17:03:39.739 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rgw x86_64 2:18.2.2-797.g49e4a679.el9 ceph 99 k 2024-03-12T17:03:39.739 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-fuse x86_64 2:18.2.2-797.g49e4a679.el9 ceph 87 k 2024-03-12T17:03:39.739 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-mirror x86_64 2:18.2.2-797.g49e4a679.el9 ceph 3.0 M 2024-03-12T17:03:39.739 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-nbd x86_64 2:18.2.2-797.g49e4a679.el9 ceph 172 k 2024-03-12T17:03:39.739 INFO:teuthology.orchestra.run.smithi052.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-12T17:03:39.739 INFO:teuthology.orchestra.run.smithi052.stdout:Upgrading: 2024-03-12T17:03:39.739 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-12T17:03:39.739 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-12T17:03:39.739 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-12T17:03:39.739 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-12T17:03:39.739 INFO:teuthology.orchestra.run.smithi052.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-12T17:03:39.739 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-12T17:03:39.740 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-12T17:03:39.740 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-12T17:03:39.740 INFO:teuthology.orchestra.run.smithi052.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-12T17:03:39.740 INFO:teuthology.orchestra.run.smithi052.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-12T17:03:39.740 INFO:teuthology.orchestra.run.smithi052.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-12T17:03:39.740 INFO:teuthology.orchestra.run.smithi052.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-12T17:03:39.740 INFO:teuthology.orchestra.run.smithi052.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-12T17:03:39.740 INFO:teuthology.orchestra.run.smithi052.stdout: librados2 x86_64 2:18.2.2-797.g49e4a679.el9 ceph 3.3 M 2024-03-12T17:03:39.740 INFO:teuthology.orchestra.run.smithi052.stdout: librbd1 x86_64 2:18.2.2-797.g49e4a679.el9 ceph 3.0 M 2024-03-12T17:03:39.740 INFO:teuthology.orchestra.run.smithi052.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-12T17:03:39.740 INFO:teuthology.orchestra.run.smithi052.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-12T17:03:39.740 INFO:teuthology.orchestra.run.smithi052.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-12T17:03:39.740 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-12T17:03:39.741 INFO:teuthology.orchestra.run.smithi052.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-12T17:03:39.741 INFO:teuthology.orchestra.run.smithi052.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-12T17:03:39.741 INFO:teuthology.orchestra.run.smithi052.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-12T17:03:39.741 INFO:teuthology.orchestra.run.smithi052.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-12T17:03:39.741 INFO:teuthology.orchestra.run.smithi052.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-12T17:03:39.741 INFO:teuthology.orchestra.run.smithi052.stdout:Installing dependencies: 2024-03-12T17:03:39.741 INFO:teuthology.orchestra.run.smithi052.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-12T17:03:39.741 INFO:teuthology.orchestra.run.smithi052.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-12T17:03:39.741 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-common x86_64 2:18.2.2-797.g49e4a679.el9 ceph 18 M 2024-03-12T17:03:39.741 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-grafana-dashboards noarch 2:18.2.2-797.g49e4a679.el9 ceph-noarch 23 k 2024-03-12T17:03:39.741 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mds x86_64 2:18.2.2-797.g49e4a679.el9 ceph 2.1 M 2024-03-12T17:03:39.741 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-modules-core noarch 2:18.2.2-797.g49e4a679.el9 ceph-noarch 245 k 2024-03-12T17:03:39.741 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mon x86_64 2:18.2.2-797.g49e4a679.el9 ceph 4.4 M 2024-03-12T17:03:39.741 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-osd x86_64 2:18.2.2-797.g49e4a679.el9 ceph 18 M 2024-03-12T17:03:39.742 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-prometheus-alerts noarch 2:18.2.2-797.g49e4a679.el9 ceph-noarch 15 k 2024-03-12T17:03:39.742 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-selinux x86_64 2:18.2.2-797.g49e4a679.el9 ceph 25 k 2024-03-12T17:03:39.742 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-12T17:03:39.742 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-12T17:03:39.742 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-12T17:03:39.742 INFO:teuthology.orchestra.run.smithi052.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-03-12T17:03:39.742 INFO:teuthology.orchestra.run.smithi052.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-12T17:03:39.742 INFO:teuthology.orchestra.run.smithi052.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-12T17:03:39.742 INFO:teuthology.orchestra.run.smithi052.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-12T17:03:39.742 INFO:teuthology.orchestra.run.smithi052.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-12T17:03:39.742 INFO:teuthology.orchestra.run.smithi052.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-12T17:03:39.742 INFO:teuthology.orchestra.run.smithi052.stdout: libcephsqlite x86_64 2:18.2.2-797.g49e4a679.el9 ceph 166 k 2024-03-12T17:03:39.742 INFO:teuthology.orchestra.run.smithi052.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-12T17:03:39.742 INFO:teuthology.orchestra.run.smithi052.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-12T17:03:39.743 INFO:teuthology.orchestra.run.smithi052.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-12T17:03:39.743 INFO:teuthology.orchestra.run.smithi052.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-12T17:03:39.743 INFO:teuthology.orchestra.run.smithi052.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-12T17:03:39.743 INFO:teuthology.orchestra.run.smithi052.stdout: libradosstriper1 x86_64 2:18.2.2-797.g49e4a679.el9 ceph 475 k 2024-03-12T17:03:39.743 INFO:teuthology.orchestra.run.smithi052.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-12T17:03:39.743 INFO:teuthology.orchestra.run.smithi052.stdout: librgw2 x86_64 2:18.2.2-797.g49e4a679.el9 ceph 4.5 M 2024-03-12T17:03:39.743 INFO:teuthology.orchestra.run.smithi052.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-12T17:03:39.743 INFO:teuthology.orchestra.run.smithi052.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-12T17:03:39.743 INFO:teuthology.orchestra.run.smithi052.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-12T17:03:39.743 INFO:teuthology.orchestra.run.smithi052.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-12T17:03:39.744 INFO:teuthology.orchestra.run.smithi052.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-12T17:03:39.744 INFO:teuthology.orchestra.run.smithi052.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-12T17:03:39.744 INFO:teuthology.orchestra.run.smithi052.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-12T17:03:39.744 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-12T17:03:39.745 INFO:teuthology.orchestra.run.smithi052.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-12T17:03:39.745 INFO:teuthology.orchestra.run.smithi052.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-12T17:03:39.745 INFO:teuthology.orchestra.run.smithi052.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-12T17:03:39.745 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-12T17:03:39.745 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-argparse x86_64 2:18.2.2-797.g49e4a679.el9 ceph 45 k 2024-03-12T17:03:39.745 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-common x86_64 2:18.2.2-797.g49e4a679.el9 ceph 126 k 2024-03-12T17:03:39.745 INFO:teuthology.orchestra.run.smithi052.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-12T17:03:39.745 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-12T17:03:39.745 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-12T17:03:39.745 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-12T17:03:39.745 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-12T17:03:39.745 INFO:teuthology.orchestra.run.smithi052.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-12T17:03:39.745 INFO:teuthology.orchestra.run.smithi052.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-12T17:03:39.746 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-12T17:03:39.746 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-12T17:03:39.746 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-12T17:03:39.746 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-12T17:03:39.746 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-12T17:03:39.746 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-12T17:03:39.746 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-12T17:03:39.746 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-12T17:03:39.746 INFO:teuthology.orchestra.run.smithi052.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-12T17:03:39.746 INFO:teuthology.orchestra.run.smithi052.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-12T17:03:39.746 INFO:teuthology.orchestra.run.smithi052.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-12T17:03:39.746 INFO:teuthology.orchestra.run.smithi052.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-12T17:03:39.746 INFO:teuthology.orchestra.run.smithi052.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-12T17:03:39.747 INFO:teuthology.orchestra.run.smithi052.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-12T17:03:39.747 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-12T17:03:39.747 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-12T17:03:39.747 INFO:teuthology.orchestra.run.smithi052.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-12T17:03:39.747 INFO:teuthology.orchestra.run.smithi052.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-12T17:03:39.747 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-12T17:03:39.747 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-12T17:03:39.747 INFO:teuthology.orchestra.run.smithi052.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-12T17:03:39.747 INFO:teuthology.orchestra.run.smithi052.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-12T17:03:39.747 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-12T17:03:39.747 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-12T17:03:39.747 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-12T17:03:39.747 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-12T17:03:39.748 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-12T17:03:39.748 INFO:teuthology.orchestra.run.smithi052.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-12T17:03:39.748 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-12T17:03:39.748 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-12T17:03:39.748 INFO:teuthology.orchestra.run.smithi052.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-12T17:03:39.748 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-12T17:03:39.748 INFO:teuthology.orchestra.run.smithi052.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-12T17:03:39.748 INFO:teuthology.orchestra.run.smithi052.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-12T17:03:39.748 INFO:teuthology.orchestra.run.smithi052.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-12T17:03:39.748 INFO:teuthology.orchestra.run.smithi052.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-12T17:03:39.748 INFO:teuthology.orchestra.run.smithi052.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-12T17:03:39.748 INFO:teuthology.orchestra.run.smithi052.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-12T17:03:39.748 INFO:teuthology.orchestra.run.smithi052.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-12T17:03:39.749 INFO:teuthology.orchestra.run.smithi052.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-12T17:03:39.749 INFO:teuthology.orchestra.run.smithi052.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-12T17:03:39.749 INFO:teuthology.orchestra.run.smithi052.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-12T17:03:39.749 INFO:teuthology.orchestra.run.smithi052.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-12T17:03:39.749 INFO:teuthology.orchestra.run.smithi052.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-12T17:03:39.749 INFO:teuthology.orchestra.run.smithi052.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-12T17:03:39.750 INFO:teuthology.orchestra.run.smithi052.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-12T17:03:39.750 INFO:teuthology.orchestra.run.smithi052.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-12T17:03:39.750 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:03:39.750 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction Summary 2024-03-12T17:03:39.750 INFO:teuthology.orchestra.run.smithi052.stdout:=========================================================================================== 2024-03-12T17:03:39.750 INFO:teuthology.orchestra.run.smithi052.stdout:Install 132 Packages 2024-03-12T17:03:39.750 INFO:teuthology.orchestra.run.smithi052.stdout:Upgrade 24 Packages 2024-03-12T17:03:39.750 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:03:39.751 INFO:teuthology.orchestra.run.smithi052.stdout:Total download size: 204 M 2024-03-12T17:03:39.751 INFO:teuthology.orchestra.run.smithi052.stdout:Downloading Packages: 2024-03-12T17:03:40.346 INFO:teuthology.orchestra.run.smithi102.stdout:(30/156): ceph-test-18.2.2-797.g49e4a679.el9.x8 13 MB/s | 40 MB 00:02 2024-03-12T17:03:40.427 INFO:teuthology.orchestra.run.smithi102.stdout:(31/156): ceph-mgr-dashboard-18.2.2-797.g49e4a6 2.4 MB/s | 1.7 MB 00:00 2024-03-12T17:03:40.462 INFO:teuthology.orchestra.run.smithi102.stdout:(32/156): ceph-mgr-modules-core-18.2.2-797.g49e 2.1 MB/s | 245 kB 00:00 2024-03-12T17:03:40.487 INFO:teuthology.orchestra.run.smithi102.stdout:(33/156): ceph-mgr-rook-18.2.2-797.g49e4a679.el 826 kB/s | 49 kB 00:00 2024-03-12T17:03:40.513 INFO:teuthology.orchestra.run.smithi102.stdout:(34/156): ceph-prometheus-alerts-18.2.2-797.g49 300 kB/s | 15 kB 00:00 2024-03-12T17:03:40.546 INFO:teuthology.orchestra.run.smithi102.stdout:(35/156): ceph-volume-18.2.2-797.g49e4a679.el9. 4.5 MB/s | 264 kB 00:00 2024-03-12T17:03:40.548 INFO:teuthology.orchestra.run.smithi052.stdout:(1/156): ceph-18.2.2-797.g49e4a679.el9.x86_64.r 22 kB/s | 6.5 kB 00:00 2024-03-12T17:03:40.582 INFO:teuthology.orchestra.run.smithi102.stdout:(36/156): cephadm-18.2.2-797.g49e4a679.el9.noar 3.3 MB/s | 223 kB 00:00 2024-03-12T17:03:40.605 INFO:teuthology.orchestra.run.smithi102.stdout:(37/156): cephfs-top-18.2.2-797.g49e4a679.el9.n 315 kB/s | 18 kB 00:00 2024-03-12T17:03:40.889 INFO:teuthology.orchestra.run.smithi102.stdout:(38/156): ceph-mgr-diskprediction-local-18.2.2- 6.4 MB/s | 7.4 MB 00:01 2024-03-12T17:03:40.923 INFO:teuthology.orchestra.run.smithi102.stdout:(39/156): automake-1.16.2-8.el9.noarch.rpm 1.9 MB/s | 662 kB 00:00 2024-03-12T17:03:40.948 INFO:teuthology.orchestra.run.smithi102.stdout:(40/156): boost-program-options-1.75.0-8.el9.x8 1.9 MB/s | 106 kB 00:00 2024-03-12T17:03:40.998 INFO:teuthology.orchestra.run.smithi102.stdout:(41/156): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 307 kB 00:00 2024-03-12T17:03:41.040 INFO:teuthology.orchestra.run.smithi102.stdout:(42/156): bison-3.7.4-5.el9.x86_64.rpm 2.1 MB/s | 920 kB 00:00 2024-03-12T17:03:41.090 INFO:teuthology.orchestra.run.smithi102.stdout:(43/156): flexiblas-3.0.4-8.el9.x86_64.rpm 224 kB/s | 32 kB 00:00 2024-03-12T17:03:41.257 INFO:teuthology.orchestra.run.smithi102.stdout:(44/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-03-12T17:03:41.274 INFO:teuthology.orchestra.run.smithi102.stdout:(45/156): libaio-devel-0.3.111-13.el9.x86_64.rp 58 kB/s | 10 kB 00:00 2024-03-12T17:03:41.290 INFO:teuthology.orchestra.run.smithi052.stdout:(2/156): ceph-fuse-18.2.2-797.g49e4a679.el9.x86 1.1 MB/s | 844 kB 00:00 2024-03-12T17:03:41.299 INFO:teuthology.orchestra.run.smithi102.stdout:(46/156): flexiblas-openblas-openmp-3.0.4-8.el9 87 kB/s | 17 kB 00:00 2024-03-12T17:03:41.333 INFO:teuthology.orchestra.run.smithi102.stdout:(47/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2024-03-12T17:03:41.350 INFO:teuthology.orchestra.run.smithi102.stdout:(48/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 594 kB/s | 44 kB 00:00 2024-03-12T17:03:41.417 INFO:teuthology.orchestra.run.smithi102.stdout:(49/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.6 MB/s | 661 kB 00:00 2024-03-12T17:03:41.467 INFO:teuthology.orchestra.run.smithi102.stdout:(50/156): libtool-2.4.6-45.el9.x86_64.rpm 5.1 MB/s | 578 kB 00:00 2024-03-12T17:03:41.484 INFO:teuthology.orchestra.run.smithi102.stdout:(51/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 365 kB/s | 46 kB 00:00 2024-03-12T17:03:41.518 INFO:teuthology.orchestra.run.smithi102.stdout:(52/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.9 MB/s | 282 kB 00:00 2024-03-12T17:03:41.543 INFO:teuthology.orchestra.run.smithi102.stdout:(53/156): openblas-0.3.21-2.el9.x86_64.rpm 622 kB/s | 34 kB 00:00 2024-03-12T17:03:41.576 INFO:teuthology.orchestra.run.smithi102.stdout:(54/156): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.4 MB/s | 217 kB 00:00 2024-03-12T17:03:41.615 INFO:teuthology.orchestra.run.smithi052.stdout:(3/156): ceph-common-18.2.2-797.g49e4a679.el9.x 13 MB/s | 18 MB 00:01 2024-03-12T17:03:41.640 INFO:teuthology.orchestra.run.smithi052.stdout:(4/156): ceph-immutable-object-cache-18.2.2-797 408 kB/s | 143 kB 00:00 2024-03-12T17:03:41.758 INFO:teuthology.orchestra.run.smithi052.stdout:(5/156): ceph-mgr-18.2.2-797.g49e4a679.el9.x86_ 13 MB/s | 1.4 MB 00:00 2024-03-12T17:03:41.860 INFO:teuthology.orchestra.run.smithi102.stdout:(55/156): openblas-openmp-0.3.21-2.el9.x86_64.r 14 MB/s | 4.7 MB 00:00 2024-03-12T17:03:41.927 INFO:teuthology.orchestra.run.smithi102.stdout:(56/156): postgresql-13.11-1.el9.x86_64.rpm 4.0 MB/s | 1.5 MB 00:00 2024-03-12T17:03:41.969 INFO:teuthology.orchestra.run.smithi102.stdout:(57/156): postgresql-contrib-13.11-1.el9.x86_64 2.1 MB/s | 811 kB 00:00 2024-03-12T17:03:41.986 INFO:teuthology.orchestra.run.smithi102.stdout:(58/156): postgresql-private-libs-13.11-1.el9.x 1.2 MB/s | 134 kB 00:00 2024-03-12T17:03:42.041 INFO:teuthology.orchestra.run.smithi052.stdout:(6/156): ceph-mon-18.2.2-797.g49e4a679.el9.x86_ 16 MB/s | 4.4 MB 00:00 2024-03-12T17:03:42.360 INFO:teuthology.orchestra.run.smithi102.stdout:(59/156): postgresql-server-13.11-1.el9.x86_64. 13 MB/s | 5.7 MB 00:00 2024-03-12T17:03:42.386 INFO:teuthology.orchestra.run.smithi102.stdout:(60/156): python3-jsonpatch-1.21-16.el9.noarch. 61 kB/s | 24 kB 00:00 2024-03-12T17:03:42.603 INFO:teuthology.orchestra.run.smithi102.stdout:(61/156): python3-babel-2.9.1-2.el9.noarch.rpm 9.4 MB/s | 5.8 MB 00:00 2024-03-12T17:03:42.620 INFO:teuthology.orchestra.run.smithi102.stdout:(62/156): python3-jsonpointer-2.0-4.el9.noarch. 70 kB/s | 18 kB 00:00 2024-03-12T17:03:42.645 INFO:teuthology.orchestra.run.smithi102.stdout:(63/156): python3-mako-1.1.4-6.el9.noarch.rpm 590 kB/s | 144 kB 00:00 2024-03-12T17:03:42.704 INFO:teuthology.orchestra.run.smithi102.stdout:(64/156): python3-markupsafe-1.1.1-12.el9.x86_6 388 kB/s | 32 kB 00:00 2024-03-12T17:03:42.775 INFO:teuthology.orchestra.run.smithi052.stdout:(7/156): ceph-base-18.2.2-797.g49e4a679.el9.x86 2.1 MB/s | 5.2 MB 00:02 2024-03-12T17:03:42.896 INFO:teuthology.orchestra.run.smithi102.stdout:(65/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 18 MB/s | 4.8 MB 00:00 2024-03-12T17:03:42.929 INFO:teuthology.orchestra.run.smithi102.stdout:(66/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 778 kB/s | 217 kB 00:00 2024-03-12T17:03:42.946 INFO:teuthology.orchestra.run.smithi102.stdout:(67/156): python3-oauthlib-3.1.1-5.el9.noarch.r 764 kB/s | 174 kB 00:00 2024-03-12T17:03:42.972 INFO:teuthology.orchestra.run.smithi102.stdout:(68/156): python3-packaging-20.9-5.el9.noarch.r 970 kB/s | 69 kB 00:00 2024-03-12T17:03:43.008 INFO:teuthology.orchestra.run.smithi052.stdout:(8/156): ceph-osd-18.2.2-797.g49e4a679.el9.x86_ 18 MB/s | 18 MB 00:00 2024-03-12T17:03:43.022 INFO:teuthology.orchestra.run.smithi102.stdout:(69/156): python3-prettytable-0.7.2-27.el9.noar 560 kB/s | 41 kB 00:00 2024-03-12T17:03:43.047 INFO:teuthology.orchestra.run.smithi102.stdout:(70/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 132 kB 00:00 2024-03-12T17:03:43.058 INFO:teuthology.orchestra.run.smithi052.stdout:(9/156): ceph-selinux-18.2.2-797.g49e4a679.el9. 513 kB/s | 25 kB 00:00 2024-03-12T17:03:43.081 INFO:teuthology.orchestra.run.smithi102.stdout:(71/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.9 MB/s | 211 kB 00:00 2024-03-12T17:03:43.106 INFO:teuthology.orchestra.run.smithi102.stdout:(72/156): python3-pytz-2021.1-5.el9.noarch.rpm 844 kB/s | 47 kB 00:00 2024-03-12T17:03:43.123 INFO:teuthology.orchestra.run.smithi102.stdout:(73/156): python3-requests-oauthlib-1.3.0-12.el 589 kB/s | 43 kB 00:00 2024-03-12T17:03:43.390 INFO:teuthology.orchestra.run.smithi102.stdout:(74/156): python3-toml-0.10.2-6.el9.noarch.rpm 152 kB/s | 39 kB 00:00 2024-03-12T17:03:43.409 INFO:teuthology.orchestra.run.smithi052.stdout:(10/156): ceph-mds-18.2.2-797.g49e4a679.el9.x86 1.2 MB/s | 2.1 MB 00:01 2024-03-12T17:03:43.513 INFO:teuthology.orchestra.run.smithi102.stdout:(75/156): socat-1.7.4.1-5.el9.x86_64.rpm 1.0 MB/s | 300 kB 00:00 2024-03-12T17:03:43.642 INFO:teuthology.orchestra.run.smithi052.stdout:(11/156): cephfs-mirror-18.2.2-797.g49e4a679.el 954 kB/s | 221 kB 00:00 2024-03-12T17:03:43.757 INFO:teuthology.orchestra.run.smithi102.stdout:(76/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-03-12T17:03:43.773 INFO:teuthology.orchestra.run.smithi102.stdout:(77/156): uuid-1.6.2-55.el9.x86_64.rpm 148 kB/s | 56 kB 00:00 2024-03-12T17:03:43.776 INFO:teuthology.orchestra.run.smithi052.stdout:(12/156): libcephfs-devel-18.2.2-797.g49e4a679. 234 kB/s | 31 kB 00:00 2024-03-12T17:03:43.798 INFO:teuthology.orchestra.run.smithi102.stdout:(78/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 174 kB/s | 63 kB 00:00 2024-03-12T17:03:43.824 INFO:teuthology.orchestra.run.smithi102.stdout:(79/156): zlib-devel-1.2.11-40.el9.x86_64.rpm 674 kB/s | 44 kB 00:00 2024-03-12T17:03:43.855 INFO:teuthology.orchestra.run.smithi102.stdout:(80/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.3 MB/s | 241 kB 00:00 2024-03-12T17:03:43.897 INFO:teuthology.orchestra.run.smithi102.stdout:(81/156): fuse-2.9.9-15.el9.x86_64.rpm 812 kB/s | 79 kB 00:00 2024-03-12T17:03:43.966 INFO:teuthology.orchestra.run.smithi102.stdout:(82/156): python3-cryptography-36.0.1-4.el9.x86 8.2 MB/s | 1.1 MB 00:00 2024-03-12T17:03:43.998 INFO:teuthology.orchestra.run.smithi102.stdout:(83/156): python3-ply-3.11-14.el9.noarch.rpm 789 kB/s | 103 kB 00:00 2024-03-12T17:03:44.031 INFO:teuthology.orchestra.run.smithi102.stdout:(84/156): python3-pycparser-2.20-6.el9.noarch.r 1.9 MB/s | 124 kB 00:00 2024-03-12T17:03:44.057 INFO:teuthology.orchestra.run.smithi102.stdout:(85/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.1 MB/s | 187 kB 00:00 2024-03-12T17:03:44.260 INFO:teuthology.orchestra.run.smithi052.stdout:(13/156): libcephfs2-18.2.2-797.g49e4a679.el9.x 1.4 MB/s | 702 kB 00:00 2024-03-12T17:03:44.452 INFO:teuthology.orchestra.run.smithi052.stdout:(14/156): libcephsqlite-18.2.2-797.g49e4a679.el 871 kB/s | 166 kB 00:00 2024-03-12T17:03:44.674 INFO:teuthology.orchestra.run.smithi102.stdout:(86/156): libquadmath-11.4.1-3.el9.x86_64.rpm 317 kB/s | 195 kB 00:00 2024-03-12T17:03:44.749 INFO:teuthology.orchestra.run.smithi102.stdout:(87/156): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-03-12T17:03:44.952 INFO:teuthology.orchestra.run.smithi052.stdout:(15/156): ceph-test-18.2.2-797.g49e4a679.el9.x8 21 MB/s | 40 MB 00:01 2024-03-12T17:03:44.977 INFO:teuthology.orchestra.run.smithi052.stdout:(16/156): librados-devel-18.2.2-797.g49e4a679.e 241 kB/s | 127 kB 00:00 2024-03-12T17:03:44.983 INFO:teuthology.orchestra.run.smithi102.stdout:(88/156): libgfortran-11.4.1-3.el9.x86_64.rpm 869 kB/s | 803 kB 00:00 2024-03-12T17:03:45.133 INFO:teuthology.orchestra.run.smithi102.stdout:(89/156): elfutils-libelf-devel-0.190-2.el9.x86 56 kB/s | 21 kB 00:00 2024-03-12T17:03:45.216 INFO:teuthology.orchestra.run.smithi102.stdout:(90/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 110 kB/s | 25 kB 00:00 2024-03-12T17:03:45.278 INFO:teuthology.orchestra.run.smithi052.stdout:(17/156): librgw2-18.2.2-797.g49e4a679.el9.x86_ 15 MB/s | 4.5 MB 00:00 2024-03-12T17:03:45.311 INFO:teuthology.orchestra.run.smithi052.stdout:(18/156): libradosstriper1-18.2.2-797.g49e4a679 1.3 MB/s | 475 kB 00:00 2024-03-12T17:03:45.336 INFO:teuthology.orchestra.run.smithi052.stdout:(19/156): python3-ceph-argparse-18.2.2-797.g49e 782 kB/s | 45 kB 00:00 2024-03-12T17:03:45.395 INFO:teuthology.orchestra.run.smithi052.stdout:(20/156): python3-cephfs-18.2.2-797.g49e4a679.e 2.7 MB/s | 161 kB 00:00 2024-03-12T17:03:45.408 INFO:teuthology.orchestra.run.smithi102.stdout:(91/156): python3-devel-3.9.17-2.el9.x86_64.rpm 1.3 MB/s | 245 kB 00:00 2024-03-12T17:03:45.437 INFO:teuthology.orchestra.run.smithi052.stdout:(21/156): python3-ceph-common-18.2.2-797.g49e4a 1.0 MB/s | 126 kB 00:00 2024-03-12T17:03:45.471 INFO:teuthology.orchestra.run.smithi052.stdout:(22/156): python3-rados-18.2.2-797.g49e4a679.el 4.2 MB/s | 321 kB 00:00 2024-03-12T17:03:45.521 INFO:teuthology.orchestra.run.smithi052.stdout:(23/156): python3-rgw-18.2.2-797.g49e4a679.el9. 2.0 MB/s | 99 kB 00:00 2024-03-12T17:03:45.542 INFO:teuthology.orchestra.run.smithi102.stdout:(92/156): python3-jinja2-2.11.3-5.el9.noarch.rp 1.8 MB/s | 248 kB 00:00 2024-03-12T17:03:45.579 INFO:teuthology.orchestra.run.smithi052.stdout:(24/156): rbd-fuse-18.2.2-797.g49e4a679.el9.x86 1.5 MB/s | 87 kB 00:00 2024-03-12T17:03:45.621 INFO:teuthology.orchestra.run.smithi052.stdout:(25/156): python3-rbd-18.2.2-797.g49e4a679.el9. 1.6 MB/s | 298 kB 00:00 2024-03-12T17:03:45.634 INFO:teuthology.orchestra.run.smithi102.stdout:(93/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.0 MB/s | 94 kB 00:00 2024-03-12T17:03:45.780 INFO:teuthology.orchestra.run.smithi052.stdout:(26/156): rbd-mirror-18.2.2-797.g49e4a679.el9.x 15 MB/s | 3.0 MB 00:00 2024-03-12T17:03:45.822 INFO:teuthology.orchestra.run.smithi052.stdout:(27/156): rbd-nbd-18.2.2-797.g49e4a679.el9.x86_ 863 kB/s | 172 kB 00:00 2024-03-12T17:03:45.847 INFO:teuthology.orchestra.run.smithi052.stdout:(28/156): ceph-grafana-dashboards-18.2.2-797.g4 354 kB/s | 23 kB 00:00 2024-03-12T17:03:45.984 INFO:teuthology.orchestra.run.smithi102.stdout:(94/156): openssl-devel-3.0.7-27.el9.x86_64.rpm 4.9 MB/s | 4.1 MB 00:00 2024-03-12T17:03:46.006 INFO:teuthology.orchestra.run.smithi052.stdout:(29/156): ceph-mgr-cephadm-18.2.2-797.g49e4a679 750 kB/s | 137 kB 00:00 2024-03-12T17:03:46.065 INFO:teuthology.orchestra.run.smithi052.stdout:(30/156): ceph-mgr-dashboard-18.2.2-797.g49e4a6 8.0 MB/s | 1.7 MB 00:00 2024-03-12T17:03:46.123 INFO:teuthology.orchestra.run.smithi052.stdout:(31/156): ceph-mgr-modules-core-18.2.2-797.g49e 4.2 MB/s | 245 kB 00:00 2024-03-12T17:03:46.166 INFO:teuthology.orchestra.run.smithi052.stdout:(32/156): ceph-mgr-rook-18.2.2-797.g49e4a679.el 1.2 MB/s | 49 kB 00:00 2024-03-12T17:03:46.243 INFO:teuthology.orchestra.run.smithi102.stdout:(95/156): fmt-8.1.1-5.el9.x86_64.rpm 182 kB/s | 111 kB 00:00 2024-03-12T17:03:46.329 INFO:teuthology.orchestra.run.smithi052.stdout:(33/156): ceph-prometheus-alerts-18.2.2-797.g49 92 kB/s | 15 kB 00:00 2024-03-12T17:03:46.408 INFO:teuthology.orchestra.run.smithi052.stdout:(34/156): ceph-volume-18.2.2-797.g49e4a679.el9. 3.3 MB/s | 264 kB 00:00 2024-03-12T17:03:46.475 INFO:teuthology.orchestra.run.smithi052.stdout:(35/156): cephadm-18.2.2-797.g49e4a679.el9.noar 3.3 MB/s | 223 kB 00:00 2024-03-12T17:03:46.525 INFO:teuthology.orchestra.run.smithi052.stdout:(36/156): cephfs-top-18.2.2-797.g49e4a679.el9.n 372 kB/s | 18 kB 00:00 2024-03-12T17:03:46.601 INFO:teuthology.orchestra.run.smithi102.stdout:(96/156): jq-1.6-16.el9.x86_64.rpm 72 kB/s | 187 kB 00:02 2024-03-12T17:03:46.652 INFO:teuthology.orchestra.run.smithi102.stdout:(97/156): gperftools-libs-2.9.1-2.el9.x86_64.rp 464 kB/s | 309 kB 00:00 2024-03-12T17:03:46.785 INFO:teuthology.orchestra.run.smithi102.stdout:(98/156): liboath-2.6.7-2.el9.x86_64.rpm 368 kB/s | 49 kB 00:00 2024-03-12T17:03:46.900 INFO:teuthology.orchestra.run.smithi052.stdout:(37/156): automake-1.16.2-8.el9.noarch.rpm 1.7 MB/s | 662 kB 00:00 2024-03-12T17:03:46.928 INFO:teuthology.orchestra.run.smithi102.stdout:(99/156): libarrow-9.0.0-10.el9.x86_64.rpm 6.5 MB/s | 4.4 MB 00:00 2024-03-12T17:03:46.953 INFO:teuthology.orchestra.run.smithi102.stdout:(100/156): libunwind-1.6.2-1.el9.x86_64.rpm 404 kB/s | 67 kB 00:00 2024-03-12T17:03:47.001 INFO:teuthology.orchestra.run.smithi052.stdout:(38/156): bison-3.7.4-5.el9.x86_64.rpm 9.1 MB/s | 920 kB 00:00 2024-03-12T17:03:47.003 INFO:teuthology.orchestra.run.smithi102.stdout:(101/156): libarrow-doc-9.0.0-10.el9.noarch.rpm 65 kB/s | 26 kB 00:00 2024-03-12T17:03:47.051 INFO:teuthology.orchestra.run.smithi052.stdout:(39/156): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 106 kB 00:00 2024-03-12T17:03:47.087 INFO:teuthology.orchestra.run.smithi102.stdout:(102/156): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.2 MB/s | 838 kB 00:00 2024-03-12T17:03:47.118 INFO:teuthology.orchestra.run.smithi052.stdout:(40/156): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 307 kB 00:00 2024-03-12T17:03:47.168 INFO:teuthology.orchestra.run.smithi052.stdout:(41/156): flexiblas-3.0.4-8.el9.x86_64.rpm 645 kB/s | 32 kB 00:00 2024-03-12T17:03:47.187 INFO:teuthology.orchestra.run.smithi102.stdout:(103/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 237 kB/s | 43 kB 00:00 2024-03-12T17:03:47.221 INFO:teuthology.orchestra.run.smithi102.stdout:(104/156): python3-asyncssh-2.13.2-4.el9.noarch 2.0 MB/s | 548 kB 00:00 2024-03-12T17:03:47.238 INFO:teuthology.orchestra.run.smithi102.stdout:(105/156): python3-cachetools-4.2.4-1.el9.noarc 215 kB/s | 32 kB 00:00 2024-03-12T17:03:47.288 INFO:teuthology.orchestra.run.smithi102.stdout:(106/156): python3-certifi-2021.10.8-2.el9.noar 150 kB/s | 15 kB 00:00 2024-03-12T17:03:47.377 INFO:teuthology.orchestra.run.smithi052.stdout:(42/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-03-12T17:03:47.436 INFO:teuthology.orchestra.run.smithi052.stdout:(43/156): flexiblas-openblas-openmp-3.0.4-8.el9 299 kB/s | 17 kB 00:00 2024-03-12T17:03:47.471 INFO:teuthology.orchestra.run.smithi102.stdout:(107/156): python3-cheroot-8.6.0-4.el9.noarch.r 687 kB/s | 172 kB 00:00 2024-03-12T17:03:47.494 INFO:teuthology.orchestra.run.smithi052.stdout:(44/156): libaio-devel-0.3.111-13.el9.x86_64.rp 175 kB/s | 10 kB 00:00 2024-03-12T17:03:47.547 INFO:teuthology.orchestra.run.smithi102.stdout:(108/156): python3-cherrypy-18.6.1-2.el9.noarch 1.1 MB/s | 358 kB 00:00 2024-03-12T17:03:47.553 INFO:teuthology.orchestra.run.smithi052.stdout:(45/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2024-03-12T17:03:47.589 INFO:teuthology.orchestra.run.smithi102.stdout:(109/156): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2024-03-12T17:03:47.611 INFO:teuthology.orchestra.run.smithi052.stdout:(46/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 765 kB/s | 44 kB 00:00 2024-03-12T17:03:47.614 INFO:teuthology.orchestra.run.smithi102.stdout:(110/156): python3-google-auth-2.28.0-1.el9.noa 622 kB/s | 203 kB 00:00 2024-03-12T17:03:47.664 INFO:teuthology.orchestra.run.smithi102.stdout:(111/156): python3-jaraco-classes-3.2.1-5.el9.n 152 kB/s | 18 kB 00:00 2024-03-12T17:03:47.687 INFO:teuthology.orchestra.run.smithi052.stdout:(47/156): librdkafka-1.6.1-102.el9.x86_64.rpm 8.7 MB/s | 661 kB 00:00 2024-03-12T17:03:47.723 INFO:teuthology.orchestra.run.smithi102.stdout:(112/156): python3-jaraco-functools-3.5.0-2.el9 180 kB/s | 19 kB 00:00 2024-03-12T17:03:47.748 INFO:teuthology.orchestra.run.smithi102.stdout:(113/156): python3-jaraco-collections-3.0.0-8.e 146 kB/s | 23 kB 00:00 2024-03-12T17:03:47.762 INFO:teuthology.orchestra.run.smithi052.stdout:(48/156): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 578 kB 00:00 2024-03-12T17:03:47.782 INFO:teuthology.orchestra.run.smithi102.stdout:(114/156): python3-jaraco-text-3.2.0-6.el9.noar 168 kB/s | 20 kB 00:00 2024-03-12T17:03:47.812 INFO:teuthology.orchestra.run.smithi052.stdout:(49/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 46 kB 00:00 2024-03-12T17:03:47.874 INFO:teuthology.orchestra.run.smithi102.stdout:(115/156): python3-logutils-0.3.5-21.el9.noarch 371 kB/s | 46 kB 00:00 2024-03-12T17:03:47.879 INFO:teuthology.orchestra.run.smithi052.stdout:(50/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.2 MB/s | 282 kB 00:00 2024-03-12T17:03:47.899 INFO:teuthology.orchestra.run.smithi102.stdout:(116/156): python3-more-itertools-8.12.0-2.el9. 678 kB/s | 79 kB 00:00 2024-03-12T17:03:47.938 INFO:teuthology.orchestra.run.smithi052.stdout:(51/156): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.7 MB/s | 217 kB 00:00 2024-03-12T17:03:47.997 INFO:teuthology.orchestra.run.smithi052.stdout:(52/156): openblas-0.3.21-2.el9.x86_64.rpm 593 kB/s | 34 kB 00:00 2024-03-12T17:03:47.999 INFO:teuthology.orchestra.run.smithi102.stdout:(117/156): python3-natsort-7.1.1-5.el9.noarch.r 463 kB/s | 58 kB 00:00 2024-03-12T17:03:48.033 INFO:teuthology.orchestra.run.smithi102.stdout:(118/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-03-12T17:03:48.075 INFO:teuthology.orchestra.run.smithi102.stdout:(119/156): python3-kubernetes-26.1.0-2.el9.noar 2.9 MB/s | 1.0 MB 00:00 2024-03-12T17:03:48.117 INFO:teuthology.orchestra.run.smithi102.stdout:(120/156): python3-portend-3.1.0-2.el9.noarch.r 142 kB/s | 16 kB 00:00 2024-03-12T17:03:48.167 INFO:teuthology.orchestra.run.smithi102.stdout:(121/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 676 kB/s | 90 kB 00:00 2024-03-12T17:03:48.192 INFO:teuthology.orchestra.run.smithi102.stdout:(122/156): python3-repoze-lru-0.7-16.el9.noarch 263 kB/s | 31 kB 00:00 2024-03-12T17:03:48.243 INFO:teuthology.orchestra.run.smithi102.stdout:(123/156): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-03-12T17:03:48.255 INFO:teuthology.orchestra.run.smithi052.stdout:(53/156): openblas-openmp-0.3.21-2.el9.x86_64.r 18 MB/s | 4.7 MB 00:00 2024-03-12T17:03:48.293 INFO:teuthology.orchestra.run.smithi102.stdout:(124/156): python3-rsa-4.9-2.el9.noarch.rpm 473 kB/s | 59 kB 00:00 2024-03-12T17:03:48.318 INFO:teuthology.orchestra.run.smithi102.stdout:(125/156): python3-tempora-5.0.0-2.el9.noarch.r 286 kB/s | 36 kB 00:00 2024-03-12T17:03:48.343 INFO:teuthology.orchestra.run.smithi102.stdout:(126/156): python3-typing-extensions-4.4.0-2.el 510 kB/s | 51 kB 00:00 2024-03-12T17:03:48.372 INFO:teuthology.orchestra.run.smithi052.stdout:(54/156): postgresql-13.11-1.el9.x86_64.rpm 13 MB/s | 1.5 MB 00:00 2024-03-12T17:03:48.439 INFO:teuthology.orchestra.run.smithi052.stdout:(55/156): postgresql-contrib-13.11-1.el9.x86_64 12 MB/s | 811 kB 00:00 2024-03-12T17:03:48.494 INFO:teuthology.orchestra.run.smithi102.stdout:(127/156): python3-webob-1.8.7-6.el9.noarch.rpm 1.1 MB/s | 230 kB 00:00 2024-03-12T17:03:48.498 INFO:teuthology.orchestra.run.smithi052.stdout:(56/156): postgresql-private-libs-13.11-1.el9.x 2.3 MB/s | 134 kB 00:00 2024-03-12T17:03:48.510 INFO:teuthology.orchestra.run.smithi102.stdout:(128/156): python3-websocket-client-1.2.3-2.el9 468 kB/s | 90 kB 00:00 2024-03-12T17:03:48.544 INFO:teuthology.orchestra.run.smithi102.stdout:(129/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.1 MB/s | 427 kB 00:00 2024-03-12T17:03:48.611 INFO:teuthology.orchestra.run.smithi102.stdout:(130/156): python3-zc-lockfile-2.0-10.el9.noarc 171 kB/s | 20 kB 00:00 2024-03-12T17:03:48.653 INFO:teuthology.orchestra.run.smithi102.stdout:(131/156): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-03-12T17:03:48.765 INFO:teuthology.orchestra.run.smithi052.stdout:(57/156): postgresql-server-13.11-1.el9.x86_64. 21 MB/s | 5.7 MB 00:00 2024-03-12T17:03:48.803 INFO:teuthology.orchestra.run.smithi102.stdout:(132/156): thrift-0.15.0-2.el9.x86_64.rpm 6.1 MB/s | 1.6 MB 00:00 2024-03-12T17:03:48.881 INFO:teuthology.orchestra.run.smithi102.stdout:(133/156): xfsdump-3.1.12-4.el9.x86_64.rpm 4.2 MB/s | 321 kB 00:00 2024-03-12T17:03:48.924 INFO:teuthology.orchestra.run.smithi052.stdout:(58/156): ceph-radosgw-18.2.2-797.g49e4a679.el9 1.3 MB/s | 7.7 MB 00:06 2024-03-12T17:03:49.083 INFO:teuthology.orchestra.run.smithi052.stdout:(59/156): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 5.8 MB 00:00 2024-03-12T17:03:49.108 INFO:teuthology.orchestra.run.smithi052.stdout:(60/156): python3-jsonpatch-1.21-16.el9.noarch. 145 kB/s | 24 kB 00:00 2024-03-12T17:03:49.166 INFO:teuthology.orchestra.run.smithi052.stdout:(61/156): python3-jsonpointer-2.0-4.el9.noarch. 317 kB/s | 18 kB 00:00 2024-03-12T17:03:49.192 INFO:teuthology.orchestra.run.smithi052.stdout:(62/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 144 kB 00:00 2024-03-12T17:03:49.208 INFO:teuthology.orchestra.run.smithi052.stdout:(63/156): python3-markupsafe-1.1.1-12.el9.x86_6 779 kB/s | 32 kB 00:00 2024-03-12T17:03:49.434 INFO:teuthology.orchestra.run.smithi052.stdout:(64/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 4.8 MB 00:00 2024-03-12T17:03:49.451 INFO:teuthology.orchestra.run.smithi102.stdout:(134/156): librados2-18.2.2-797.g49e4a679.el9.x 3.9 MB/s | 3.3 MB 00:00 2024-03-12T17:03:49.459 INFO:teuthology.orchestra.run.smithi052.stdout:(65/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 871 kB/s | 217 kB 00:00 2024-03-12T17:03:49.493 INFO:teuthology.orchestra.run.smithi052.stdout:(66/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.0 MB/s | 174 kB 00:00 2024-03-12T17:03:49.518 INFO:teuthology.orchestra.run.smithi052.stdout:(67/156): python3-packaging-20.9-5.el9.noarch.r 1.2 MB/s | 69 kB 00:00 2024-03-12T17:03:49.544 INFO:teuthology.orchestra.run.smithi052.stdout:(68/156): python3-prettytable-0.7.2-27.el9.noar 846 kB/s | 41 kB 00:00 2024-03-12T17:03:49.569 INFO:teuthology.orchestra.run.smithi052.stdout:(69/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-12T17:03:49.602 INFO:teuthology.orchestra.run.smithi052.stdout:(70/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 211 kB 00:00 2024-03-12T17:03:49.628 INFO:teuthology.orchestra.run.smithi052.stdout:(71/156): python3-pytz-2021.1-5.el9.noarch.rpm 832 kB/s | 47 kB 00:00 2024-03-12T17:03:49.670 INFO:teuthology.orchestra.run.smithi052.stdout:(72/156): python3-requests-oauthlib-1.3.0-12.el 651 kB/s | 43 kB 00:00 2024-03-12T17:03:49.979 INFO:teuthology.orchestra.run.smithi052.stdout:(73/156): python3-toml-0.10.2-6.el9.noarch.rpm 130 kB/s | 39 kB 00:00 2024-03-12T17:03:50.023 INFO:teuthology.orchestra.run.smithi102.stdout:(135/156): librbd1-18.2.2-797.g49e4a679.el9.x86 2.2 MB/s | 3.0 MB 00:01 2024-03-12T17:03:50.223 INFO:teuthology.orchestra.run.smithi102.stdout:(136/156): NetworkManager-libnm-1.46.0-1.el9.x8 2.4 MB/s | 1.8 MB 00:00 2024-03-12T17:03:50.229 INFO:teuthology.orchestra.run.smithi052.stdout:(74/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-03-12T17:03:50.262 INFO:teuthology.orchestra.run.smithi052.stdout:(75/156): socat-1.7.4.1-5.el9.x86_64.rpm 1.1 MB/s | 300 kB 00:00 2024-03-12T17:03:50.281 INFO:teuthology.orchestra.run.smithi102.stdout:(137/156): NetworkManager-tui-1.46.0-1.el9.x86_ 4.2 MB/s | 248 kB 00:00 2024-03-12T17:03:50.373 INFO:teuthology.orchestra.run.smithi102.stdout:(138/156): NetworkManager-team-1.46.0-1.el9.x86 121 kB/s | 42 kB 00:00 2024-03-12T17:03:50.419 INFO:teuthology.orchestra.run.smithi102.stdout:(139/156): acl-2.3.1-4.el9.x86_64.rpm 524 kB/s | 71 kB 00:00 2024-03-12T17:03:50.552 INFO:teuthology.orchestra.run.smithi052.stdout:(76/156): uuid-1.6.2-55.el9.x86_64.rpm 177 kB/s | 56 kB 00:00 2024-03-12T17:03:50.630 INFO:teuthology.orchestra.run.smithi052.stdout:(77/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 171 kB/s | 63 kB 00:00 2024-03-12T17:03:50.655 INFO:teuthology.orchestra.run.smithi052.stdout:(78/156): zlib-devel-1.2.11-40.el9.x86_64.rpm 435 kB/s | 44 kB 00:00 2024-03-12T17:03:50.674 INFO:teuthology.orchestra.run.smithi102.stdout:(140/156): elfutils-debuginfod-client-0.190-2.e 122 kB/s | 36 kB 00:00 2024-03-12T17:03:50.688 INFO:teuthology.orchestra.run.smithi052.stdout:(79/156): fuse-2.9.9-15.el9.x86_64.rpm 1.4 MB/s | 79 kB 00:00 2024-03-12T17:03:50.722 INFO:teuthology.orchestra.run.smithi052.stdout:(80/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.5 MB/s | 241 kB 00:00 2024-03-12T17:03:50.798 INFO:teuthology.orchestra.run.smithi052.stdout:(81/156): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.1 MB 00:00 2024-03-12T17:03:50.832 INFO:teuthology.orchestra.run.smithi052.stdout:(82/156): python3-ply-3.11-14.el9.noarch.rpm 946 kB/s | 103 kB 00:00 2024-03-12T17:03:50.857 INFO:teuthology.orchestra.run.smithi052.stdout:(83/156): python3-pycparser-2.20-6.el9.noarch.r 2.1 MB/s | 124 kB 00:00 2024-03-12T17:03:50.891 INFO:teuthology.orchestra.run.smithi052.stdout:(84/156): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 187 kB 00:00 2024-03-12T17:03:51.341 INFO:teuthology.orchestra.run.smithi102.stdout:(141/156): elfutils-libelf-0.190-2.el9.x86_64.r 209 kB/s | 192 kB 00:00 2024-03-12T17:03:51.408 INFO:teuthology.orchestra.run.smithi102.stdout:(142/156): elfutils-libs-0.190-2.el9.x86_64.rpm 353 kB/s | 256 kB 00:00 2024-03-12T17:03:51.508 INFO:teuthology.orchestra.run.smithi052.stdout:(85/156): jq-1.6-16.el9.x86_64.rpm 288 kB/s | 187 kB 00:00 2024-03-12T17:03:51.550 INFO:teuthology.orchestra.run.smithi102.stdout:(143/156): libacl-2.3.1-4.el9.x86_64.rpm 111 kB/s | 23 kB 00:00 2024-03-12T17:03:51.625 INFO:teuthology.orchestra.run.smithi102.stdout:(144/156): libfdisk-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 155 kB 00:00 2024-03-12T17:03:51.675 INFO:teuthology.orchestra.run.smithi052.stdout:(86/156): ceph-mgr-diskprediction-local-18.2.2- 1.3 MB/s | 7.4 MB 00:05 2024-03-12T17:03:51.675 INFO:teuthology.orchestra.run.smithi102.stdout:(145/156): libblkid-2.37.4-18.el9.x86_64.rpm 407 kB/s | 108 kB 00:00 2024-03-12T17:03:51.731 INFO:teuthology.orchestra.run.smithi102.stdout:(146/156): libmount-2.37.4-18.el9.x86_64.rpm 1.3 MB/s | 136 kB 00:00 2024-03-12T17:03:51.758 INFO:teuthology.orchestra.run.smithi052.stdout:(87/156): libquadmath-11.4.1-3.el9.x86_64.rpm 782 kB/s | 195 kB 00:00 2024-03-12T17:03:51.808 INFO:teuthology.orchestra.run.smithi052.stdout:(88/156): elfutils-libelf-devel-0.190-2.el9.x86 441 kB/s | 21 kB 00:00 2024-03-12T17:03:51.859 INFO:teuthology.orchestra.run.smithi052.stdout:(89/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 514 kB/s | 25 kB 00:00 2024-03-12T17:03:51.892 INFO:teuthology.orchestra.run.smithi052.stdout:(90/156): libgfortran-11.4.1-3.el9.x86_64.rpm 803 kB/s | 803 kB 00:01 2024-03-12T17:03:52.076 INFO:teuthology.orchestra.run.smithi052.stdout:(91/156): python3-devel-3.9.17-2.el9.x86_64.rpm 1.3 MB/s | 245 kB 00:00 2024-03-12T17:03:52.126 INFO:teuthology.orchestra.run.smithi102.stdout:(147/156): libuuid-2.37.4-18.el9.x86_64.rpm 73 kB/s | 29 kB 00:00 2024-03-12T17:03:52.168 INFO:teuthology.orchestra.run.smithi052.stdout:(92/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.7 MB/s | 248 kB 00:00 2024-03-12T17:03:52.176 INFO:teuthology.orchestra.run.smithi102.stdout:(148/156): libsmartcols-2.37.4-18.el9.x86_64.rp 128 kB/s | 64 kB 00:00 2024-03-12T17:03:52.202 INFO:teuthology.orchestra.run.smithi052.stdout:(93/156): python3-requests-2.25.1-8.el9.noarch. 237 kB/s | 125 kB 00:00 2024-03-12T17:03:52.235 INFO:teuthology.orchestra.run.smithi102.stdout:(149/156): openssl-3.0.7-27.el9.x86_64.rpm 11 MB/s | 1.2 MB 00:00 2024-03-12T17:03:52.260 INFO:teuthology.orchestra.run.smithi052.stdout:(94/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.0 MB/s | 94 kB 00:00 2024-03-12T17:03:52.352 INFO:teuthology.orchestra.run.smithi052.stdout:(95/156): openssl-devel-3.0.7-27.el9.x86_64.rpm 8.4 MB/s | 4.1 MB 00:00 2024-03-12T17:03:52.443 INFO:teuthology.orchestra.run.smithi102.stdout:(150/156): util-linux-2.37.4-18.el9.x86_64.rpm 11 MB/s | 2.3 MB 00:00 2024-03-12T17:03:52.510 INFO:teuthology.orchestra.run.smithi102.stdout:(151/156): util-linux-core-2.37.4-18.el9.x86_64 6.9 MB/s | 465 kB 00:00 2024-03-12T17:03:52.594 INFO:teuthology.orchestra.run.smithi102.stdout:(152/156): util-linux-user-2.37.4-18.el9.x86_64 377 kB/s | 31 kB 00:00 2024-03-12T17:03:52.736 INFO:teuthology.orchestra.run.smithi102.stdout:(153/156): xfsprogs-6.3.0-1.el9.x86_64.rpm 7.6 MB/s | 1.1 MB 00:00 2024-03-12T17:03:53.084 INFO:teuthology.orchestra.run.smithi052.stdout:(96/156): fmt-8.1.1-5.el9.x86_64.rpm 126 kB/s | 111 kB 00:00 2024-03-12T17:03:53.153 INFO:teuthology.orchestra.run.smithi102.stdout:(154/156): libacl-devel-2.3.1-4.el9.x86_64.rpm 179 kB/s | 74 kB 00:00 2024-03-12T17:03:53.226 INFO:teuthology.orchestra.run.smithi052.stdout:(97/156): libarrow-doc-9.0.0-10.el9.noarch.rpm 184 kB/s | 26 kB 00:00 2024-03-12T17:03:53.360 INFO:teuthology.orchestra.run.smithi052.stdout:(98/156): liboath-2.6.7-2.el9.x86_64.rpm 366 kB/s | 49 kB 00:00 2024-03-12T17:03:53.393 INFO:teuthology.orchestra.run.smithi052.stdout:(99/156): gperftools-libs-2.9.1-2.el9.x86_64.rp 273 kB/s | 309 kB 00:01 2024-03-12T17:03:53.486 INFO:teuthology.orchestra.run.smithi102.stdout:(155/156): NetworkManager-1.46.0-1.el9.x86_64.r 509 kB/s | 2.3 MB 00:04 2024-03-12T17:03:53.594 INFO:teuthology.orchestra.run.smithi052.stdout:(100/156): libunwind-1.6.2-1.el9.x86_64.rpm 289 kB/s | 67 kB 00:00 2024-03-12T17:03:54.536 INFO:teuthology.orchestra.run.smithi052.stdout:(101/156): python3-asyncssh-2.13.2-4.el9.noarch 582 kB/s | 548 kB 00:00 2024-03-12T17:03:54.678 INFO:teuthology.orchestra.run.smithi052.stdout:(102/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 308 kB/s | 43 kB 00:00 2024-03-12T17:03:54.753 INFO:teuthology.orchestra.run.smithi052.stdout:(103/156): parquet-libs-9.0.0-10.el9.x86_64.rpm 617 kB/s | 838 kB 00:01 2024-03-12T17:03:54.812 INFO:teuthology.orchestra.run.smithi052.stdout:(104/156): python3-cachetools-4.2.4-1.el9.noarc 242 kB/s | 32 kB 00:00 2024-03-12T17:03:54.887 INFO:teuthology.orchestra.run.smithi052.stdout:(105/156): python3-certifi-2021.10.8-2.el9.noar 113 kB/s | 15 kB 00:00 2024-03-12T17:03:55.171 INFO:teuthology.orchestra.run.smithi052.stdout:(106/156): python3-cheroot-8.6.0-4.el9.noarch.r 480 kB/s | 172 kB 00:00 2024-03-12T17:03:55.228 INFO:teuthology.orchestra.run.smithi102.stdout:(156/156): openssl-libs-3.0.7-27.el9.x86_64.rpm 724 kB/s | 2.2 MB 00:03 2024-03-12T17:03:55.236 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-03-12T17:03:55.236 INFO:teuthology.orchestra.run.smithi102.stdout:Total 10 MB/s | 204 MB 00:20 2024-03-12T17:03:55.579 INFO:teuthology.orchestra.run.smithi052.stdout:(107/156): python3-cherrypy-18.6.1-2.el9.noarch 518 kB/s | 358 kB 00:00 2024-03-12T17:03:55.655 INFO:teuthology.orchestra.run.smithi052.stdout:(108/156): python3-google-auth-2.28.0-1.el9.noa 419 kB/s | 203 kB 00:00 2024-03-12T17:03:55.730 INFO:teuthology.orchestra.run.smithi052.stdout:(109/156): python3-jaraco-8.2.1-3.el9.noarch.rp 71 kB/s | 11 kB 00:00 2024-03-12T17:03:55.813 INFO:teuthology.orchestra.run.smithi052.stdout:(110/156): python3-jaraco-classes-3.2.1-5.el9.n 112 kB/s | 18 kB 00:00 2024-03-12T17:03:55.889 INFO:teuthology.orchestra.run.smithi052.stdout:(111/156): python3-jaraco-collections-3.0.0-8.e 146 kB/s | 23 kB 00:00 2024-03-12T17:03:55.956 INFO:teuthology.orchestra.run.smithi052.stdout:(112/156): python3-jaraco-functools-3.5.0-2.el9 137 kB/s | 19 kB 00:00 2024-03-12T17:03:56.181 INFO:teuthology.orchestra.run.smithi052.stdout:(113/156): python3-jaraco-text-3.2.0-6.el9.noar 67 kB/s | 20 kB 00:00 2024-03-12T17:03:56.331 INFO:teuthology.orchestra.run.smithi052.stdout:(114/156): python3-logutils-0.3.5-21.el9.noarch 310 kB/s | 46 kB 00:00 2024-03-12T17:03:56.460 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-03-12T17:03:56.582 INFO:teuthology.orchestra.run.smithi052.stdout:(115/156): python3-more-itertools-8.12.0-2.el9. 316 kB/s | 79 kB 00:00 2024-03-12T17:03:56.732 INFO:teuthology.orchestra.run.smithi052.stdout:(116/156): python3-natsort-7.1.1-5.el9.noarch.r 386 kB/s | 58 kB 00:00 2024-03-12T17:03:56.827 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-03-12T17:03:56.827 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-03-12T17:03:57.324 INFO:teuthology.orchestra.run.smithi052.stdout:(117/156): python3-pecan-1.4.2-3.el9.noarch.rpm 461 kB/s | 272 kB 00:00 2024-03-12T17:03:57.474 INFO:teuthology.orchestra.run.smithi052.stdout:(118/156): python3-portend-3.1.0-2.el9.noarch.r 110 kB/s | 16 kB 00:00 2024-03-12T17:03:57.741 INFO:teuthology.orchestra.run.smithi052.stdout:(119/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 338 kB/s | 90 kB 00:00 2024-03-12T17:03:57.883 INFO:teuthology.orchestra.run.smithi052.stdout:(120/156): python3-repoze-lru-0.7-16.el9.noarch 219 kB/s | 31 kB 00:00 2024-03-12T17:03:58.200 INFO:teuthology.orchestra.run.smithi052.stdout:(121/156): python3-routes-2.5.1-5.el9.noarch.rp 596 kB/s | 188 kB 00:00 2024-03-12T17:03:58.334 INFO:teuthology.orchestra.run.smithi052.stdout:(122/156): python3-rsa-4.9-2.el9.noarch.rpm 445 kB/s | 59 kB 00:00 2024-03-12T17:03:58.467 INFO:teuthology.orchestra.run.smithi052.stdout:(123/156): python3-kubernetes-26.1.0-2.el9.noar 416 kB/s | 1.0 MB 00:02 2024-03-12T17:03:58.501 INFO:teuthology.orchestra.run.smithi052.stdout:(124/156): python3-tempora-5.0.0-2.el9.noarch.r 215 kB/s | 36 kB 00:00 2024-03-12T17:03:58.593 INFO:teuthology.orchestra.run.smithi052.stdout:(125/156): python3-typing-extensions-4.4.0-2.el 409 kB/s | 51 kB 00:00 2024-03-12T17:03:58.827 INFO:teuthology.orchestra.run.smithi052.stdout:(126/156): python3-websocket-client-1.2.3-2.el9 385 kB/s | 90 kB 00:00 2024-03-12T17:03:58.952 INFO:teuthology.orchestra.run.smithi052.stdout:(127/156): python3-webob-1.8.7-6.el9.noarch.rpm 510 kB/s | 230 kB 00:00 2024-03-12T17:03:59.034 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-03-12T17:03:59.034 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-03-12T17:03:59.086 INFO:teuthology.orchestra.run.smithi052.stdout:(128/156): python3-zc-lockfile-2.0-10.el9.noarc 151 kB/s | 20 kB 00:00 2024-03-12T17:03:59.403 INFO:teuthology.orchestra.run.smithi052.stdout:(129/156): re2-20211101-3.el9.x86_64.rpm 606 kB/s | 192 kB 00:00 2024-03-12T17:03:59.570 INFO:teuthology.orchestra.run.smithi052.stdout:(130/156): python3-werkzeug-2.0.3-3.el9.1.noarc 576 kB/s | 427 kB 00:00 2024-03-12T17:04:01.229 INFO:teuthology.orchestra.run.smithi052.stdout:(131/156): librados2-18.2.2-797.g49e4a679.el9.x 2.0 MB/s | 3.3 MB 00:01 2024-03-12T17:04:02.170 INFO:teuthology.orchestra.run.smithi052.stdout:(132/156): libarrow-9.0.0-10.el9.x86_64.rpm 466 kB/s | 4.4 MB 00:09 2024-03-12T17:04:02.254 INFO:teuthology.orchestra.run.smithi052.stdout:(133/156): xfsdump-3.1.12-4.el9.x86_64.rpm 3.8 MB/s | 321 kB 00:00 2024-03-12T17:04:03.454 INFO:teuthology.orchestra.run.smithi052.stdout:(134/156): NetworkManager-1.46.0-1.el9.x86_64.r 1.9 MB/s | 2.3 MB 00:01 2024-03-12T17:04:03.530 INFO:teuthology.orchestra.run.smithi052.stdout:(135/156): librbd1-18.2.2-797.g49e4a679.el9.x86 1.3 MB/s | 3.0 MB 00:02 2024-03-12T17:04:03.630 INFO:teuthology.orchestra.run.smithi052.stdout:(136/156): thrift-0.15.0-2.el9.x86_64.rpm 383 kB/s | 1.6 MB 00:04 2024-03-12T17:04:03.788 INFO:teuthology.orchestra.run.smithi052.stdout:(137/156): NetworkManager-libnm-1.46.0-1.el9.x8 5.5 MB/s | 1.8 MB 00:00 2024-03-12T17:04:03.905 INFO:teuthology.orchestra.run.smithi052.stdout:(138/156): acl-2.3.1-4.el9.x86_64.rpm 614 kB/s | 71 kB 00:00 2024-03-12T17:04:03.922 INFO:teuthology.orchestra.run.smithi052.stdout:(139/156): NetworkManager-team-1.46.0-1.el9.x86 108 kB/s | 42 kB 00:00 2024-03-12T17:04:04.014 INFO:teuthology.orchestra.run.smithi052.stdout:(140/156): elfutils-debuginfod-client-0.190-2.e 338 kB/s | 36 kB 00:00 2024-03-12T17:04:04.389 INFO:teuthology.orchestra.run.smithi052.stdout:(141/156): elfutils-libelf-0.190-2.el9.x86_64.r 412 kB/s | 192 kB 00:00 2024-03-12T17:04:04.473 INFO:teuthology.orchestra.run.smithi052.stdout:(142/156): elfutils-libs-0.190-2.el9.x86_64.rpm 558 kB/s | 256 kB 00:00 2024-03-12T17:04:04.498 INFO:teuthology.orchestra.run.smithi052.stdout:(143/156): libacl-2.3.1-4.el9.x86_64.rpm 213 kB/s | 23 kB 00:00 2024-03-12T17:04:04.617 INFO:teuthology.orchestra.run.smithi052.stdout:(144/156): libblkid-2.37.4-18.el9.x86_64.rpm 757 kB/s | 108 kB 00:00 2024-03-12T17:04:04.642 INFO:teuthology.orchestra.run.smithi052.stdout:(145/156): NetworkManager-tui-1.46.0-1.el9.x86_ 245 kB/s | 248 kB 00:01 2024-03-12T17:04:04.668 INFO:teuthology.orchestra.run.smithi052.stdout:(146/156): libfdisk-2.37.4-18.el9.x86_64.rpm 918 kB/s | 155 kB 00:00 2024-03-12T17:04:04.768 INFO:teuthology.orchestra.run.smithi052.stdout:(147/156): libmount-2.37.4-18.el9.x86_64.rpm 905 kB/s | 136 kB 00:00 2024-03-12T17:04:04.794 INFO:teuthology.orchestra.run.smithi052.stdout:(148/156): libsmartcols-2.37.4-18.el9.x86_64.rp 425 kB/s | 64 kB 00:00 2024-03-12T17:04:04.819 INFO:teuthology.orchestra.run.smithi052.stdout:(149/156): libuuid-2.37.4-18.el9.x86_64.rpm 191 kB/s | 29 kB 00:00 2024-03-12T17:04:04.927 INFO:teuthology.orchestra.run.smithi052.stdout:(150/156): openssl-3.0.7-27.el9.x86_64.rpm 7.6 MB/s | 1.2 MB 00:00 2024-03-12T17:04:05.178 INFO:teuthology.orchestra.run.smithi052.stdout:(151/156): util-linux-core-2.37.4-18.el9.x86_64 1.8 MB/s | 465 kB 00:00 2024-03-12T17:04:05.328 INFO:teuthology.orchestra.run.smithi052.stdout:(152/156): util-linux-user-2.37.4-18.el9.x86_64 208 kB/s | 31 kB 00:00 2024-03-12T17:04:05.934 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-03-12T17:04:06.087 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/180 2024-03-12T17:04:06.134 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/180 2024-03-12T17:04:06.181 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/180 2024-03-12T17:04:06.293 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/180 2024-03-12T17:04:06.338 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/180 2024-03-12T17:04:06.414 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/180 2024-03-12T17:04:06.475 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/180 2024-03-12T17:04:06.521 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/180 2024-03-12T17:04:06.614 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/180 2024-03-12T17:04:07.019 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/180 2024-03-12T17:04:07.069 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : librados2-2:18.2.2-797.g49e4a679.el9.x86_64 10/180 2024-03-12T17:04:07.216 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-2:18.2.2-797.g49e4a679.el9.x86_64 10/180 2024-03-12T17:04:07.270 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs2-2:18.2.2-797.g49e4a679.el9.x86_64 11/180 2024-03-12T17:04:07.396 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephfs2-2:18.2.2-797.g49e4a679.el9.x86_64 11/180 2024-03-12T17:04:07.430 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rados-2:18.2.2-797.g49e4a679.el9.x86_64 12/180 2024-03-12T17:04:07.508 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/180 2024-03-12T17:04:07.511 INFO:teuthology.orchestra.run.smithi052.stdout:(153/156): util-linux-2.37.4-18.el9.x86_64.rpm 869 kB/s | 2.3 MB 00:02 2024-03-12T17:04:07.562 INFO:teuthology.orchestra.run.smithi052.stdout:(154/156): xfsprogs-6.3.0-1.el9.x86_64.rpm 492 kB/s | 1.1 MB 00:02 2024-03-12T17:04:07.662 INFO:teuthology.orchestra.run.smithi052.stdout:(155/156): libacl-devel-2.3.1-4.el9.x86_64.rpm 511 kB/s | 74 kB 00:00 2024-03-12T17:04:07.819 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/180 2024-03-12T17:04:07.875 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/180 2024-03-12T17:04:07.925 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephsqlite-2:18.2.2-797.g49e4a679.el9.x86_64 16/180 2024-03-12T17:04:08.058 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephsqlite-2:18.2.2-797.g49e4a679.el9.x86_64 16/180 2024-03-12T17:04:08.084 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libradosstriper1-2:18.2.2-797.g49e4a679.el9.x86_ 17/180 2024-03-12T17:04:08.166 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libradosstriper1-2:18.2.2-797.g49e4a679.el9.x86_ 17/180 2024-03-12T17:04:08.244 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/180 2024-03-12T17:04:08.295 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 19/180 2024-03-12T17:04:08.636 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 20/180 2024-03-12T17:04:08.670 INFO:teuthology.orchestra.run.smithi052.stdout:(156/156): openssl-libs-3.0.7-27.el9.x86_64.rpm 570 kB/s | 2.2 MB 00:03 2024-03-12T17:04:08.681 INFO:teuthology.orchestra.run.smithi052.stdout:-------------------------------------------------------------------------------- 2024-03-12T17:04:08.681 INFO:teuthology.orchestra.run.smithi052.stdout:Total 7.1 MB/s | 204 MB 00:28 2024-03-12T17:04:08.835 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 21/180 2024-03-12T17:04:08.932 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 21/180 2024-03-12T17:04:09.144 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 22/180 2024-03-12T17:04:09.234 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 22/180 2024-03-12T17:04:09.664 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 22/180 2024-03-12T17:04:09.710 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 23/180 2024-03-12T17:04:09.747 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : liboath-2.6.7-2.el9.x86_64 24/180 2024-03-12T17:04:09.809 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 25/180 2024-03-12T17:04:09.924 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 26/180 2024-03-12T17:04:09.993 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction check 2024-03-12T17:04:09.994 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/180 2024-03-12T17:04:10.052 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/180 2024-03-12T17:04:10.155 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 29/180 2024-03-12T17:04:10.222 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 30/180 2024-03-12T17:04:10.266 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 31/180 2024-03-12T17:04:10.380 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 32/180 2024-03-12T17:04:10.383 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction check succeeded. 2024-03-12T17:04:10.383 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction test 2024-03-12T17:04:10.429 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ceph-common-2:18.2.2-797.g49e4a679.el9.x 33/180 2024-03-12T17:04:10.480 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ceph-argparse-2:18.2.2-797.g49e4a679.el9 34/180 2024-03-12T17:04:10.586 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cephfs-2:18.2.2-797.g49e4a679.el9.x86_64 35/180 2024-03-12T17:04:10.766 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/180 2024-03-12T17:04:10.830 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/180 2024-03-12T17:04:11.554 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/180 2024-03-12T17:04:11.655 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/180 2024-03-12T17:04:11.785 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/180 2024-03-12T17:04:11.863 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 41/180 2024-03-12T17:04:11.900 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 41/180 2024-03-12T17:04:11.962 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 42/180 2024-03-12T17:04:12.012 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librados-devel-2:18.2.2-797.g49e4a679.el9.x86_64 43/180 2024-03-12T17:04:12.061 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/180 2024-03-12T17:04:12.513 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 45/180 2024-03-12T17:04:12.691 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction test succeeded. 2024-03-12T17:04:12.691 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction 2024-03-12T17:04:12.764 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 46/180 2024-03-12T17:04:12.875 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 47/180 2024-03-12T17:04:12.931 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 48/180 2024-03-12T17:04:13.131 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/180 2024-03-12T17:04:13.177 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 50/180 2024-03-12T17:04:13.318 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : re2-1:20211101-3.el9.x86_64 51/180 2024-03-12T17:04:13.356 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 52/180 2024-03-12T17:04:13.428 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/180 2024-03-12T17:04:13.533 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/180 2024-03-12T17:04:13.573 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/180 2024-03-12T17:04:13.612 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/180 2024-03-12T17:04:13.678 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/180 2024-03-12T17:04:13.737 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/180 2024-03-12T17:04:13.804 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 59/180 2024-03-12T17:04:13.996 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 60/180 2024-03-12T17:04:14.046 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 61/180 2024-03-12T17:04:14.110 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 62/180 2024-03-12T17:04:14.241 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 63/180 2024-03-12T17:04:14.287 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 64/180 2024-03-12T17:04:14.334 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 65/180 2024-03-12T17:04:14.394 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 66/180 2024-03-12T17:04:14.434 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 67/180 2024-03-12T17:04:14.602 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : fmt-8.1.1-5.el9.x86_64 68/180 2024-03-12T17:04:14.661 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 69/180 2024-03-12T17:04:14.763 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ply-3.11-14.el9.noarch 70/180 2024-03-12T17:04:14.856 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 71/180 2024-03-12T17:04:15.102 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 72/180 2024-03-12T17:04:15.163 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 73/180 2024-03-12T17:04:15.331 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 74/180 2024-03-12T17:04:15.386 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 75/180 2024-03-12T17:04:15.436 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 76/180 2024-03-12T17:04:15.483 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : fuse-2.9.9-15.el9.x86_64 77/180 2024-03-12T17:04:15.524 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 78/180 2024-03-12T17:04:15.571 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 79/180 2024-03-12T17:04:15.633 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : uuid-1.6.2-55.el9.x86_64 80/180 2024-03-12T17:04:15.710 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 81/180 2024-03-12T17:04:15.855 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 82/180 2024-03-12T17:04:15.899 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 83/180 2024-03-12T17:04:15.944 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 84/180 2024-03-12T17:04:16.008 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 85/180 2024-03-12T17:04:16.054 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 86/180 2024-03-12T17:04:16.209 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 87/180 2024-03-12T17:04:16.252 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 88/180 2024-03-12T17:04:16.313 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 89/180 2024-03-12T17:04:16.496 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-packaging-20.9-5.el9.noarch 90/180 2024-03-12T17:04:16.566 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 91/180 2024-03-12T17:04:17.519 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/180 2024-03-12T17:04:17.574 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 93/180 2024-03-12T17:04:17.627 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 94/180 2024-03-12T17:04:17.666 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 95/180 2024-03-12T17:04:17.916 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openblas-0.3.21-2.el9.x86_64 96/180 2024-03-12T17:04:17.970 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 97/180 2024-03-12T17:04:18.075 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 98/180 2024-03-12T17:04:18.852 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 99/180 2024-03-12T17:04:19.007 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 100/180 2024-03-12T17:04:19.587 INFO:teuthology.orchestra.run.smithi052.stdout: Preparing : 1/1 2024-03-12T17:04:19.678 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/180 2024-03-12T17:04:19.718 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/180 2024-03-12T17:04:19.788 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/180 2024-03-12T17:04:19.920 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/180 2024-03-12T17:04:19.964 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/180 2024-03-12T17:04:20.006 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/180 2024-03-12T17:04:20.054 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/180 2024-03-12T17:04:20.088 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/180 2024-03-12T17:04:20.292 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/180 2024-03-12T17:04:20.530 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 101/180 2024-03-12T17:04:20.624 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 102/180 2024-03-12T17:04:20.663 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 103/180 2024-03-12T17:04:20.694 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : jq-1.6-16.el9.x86_64 104/180 2024-03-12T17:04:20.706 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/180 2024-03-12T17:04:20.735 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 105/180 2024-03-12T17:04:20.753 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : librados2-2:18.2.2-797.g49e4a679.el9.x86_64 10/180 2024-03-12T17:04:20.911 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librados2-2:18.2.2-797.g49e4a679.el9.x86_64 10/180 2024-03-12T17:04:20.937 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libcephfs2-2:18.2.2-797.g49e4a679.el9.x86_64 11/180 2024-03-12T17:04:21.047 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libcephfs2-2:18.2.2-797.g49e4a679.el9.x86_64 11/180 2024-03-12T17:04:21.093 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 106/180 2024-03-12T17:04:21.104 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rados-2:18.2.2-797.g49e4a679.el9.x86_64 12/180 2024-03-12T17:04:21.155 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : librbd1-2:18.2.2-797.g49e4a679.el9.x86_64 107/180 2024-03-12T17:04:21.176 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/180 2024-03-12T17:04:21.280 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-2:18.2.2-797.g49e4a679.el9.x86_64 107/180 2024-03-12T17:04:21.314 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rbd-2:18.2.2-797.g49e4a679.el9.x86_64 108/180 2024-03-12T17:04:21.370 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 109/180 2024-03-12T17:04:21.464 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/180 2024-03-12T17:04:21.518 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/180 2024-03-12T17:04:21.544 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libcephsqlite-2:18.2.2-797.g49e4a679.el9.x86_64 16/180 2024-03-12T17:04:21.688 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libcephsqlite-2:18.2.2-797.g49e4a679.el9.x86_64 16/180 2024-03-12T17:04:21.720 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libradosstriper1-2:18.2.2-797.g49e4a679.el9.x86_ 17/180 2024-03-12T17:04:21.862 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 110/180 2024-03-12T17:04:21.902 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librgw2-2:18.2.2-797.g49e4a679.el9.x86_64 111/180 2024-03-12T17:04:21.945 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libradosstriper1-2:18.2.2-797.g49e4a679.el9.x86_ 17/180 2024-03-12T17:04:21.994 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librgw2-2:18.2.2-797.g49e4a679.el9.x86_64 111/180 2024-03-12T17:04:22.008 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/180 2024-03-12T17:04:22.020 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rgw-2:18.2.2-797.g49e4a679.el9.x86_64 112/180 2024-03-12T17:04:22.053 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 19/180 2024-03-12T17:04:22.424 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 20/180 2024-03-12T17:04:22.559 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 21/180 2024-03-12T17:04:22.633 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 21/180 2024-03-12T17:04:22.877 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 22/180 2024-03-12T17:04:22.929 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 22/180 2024-03-12T17:04:23.445 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 22/180 2024-03-12T17:04:23.493 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 23/180 2024-03-12T17:04:23.534 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : liboath-2.6.7-2.el9.x86_64 24/180 2024-03-12T17:04:23.614 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 25/180 2024-03-12T17:04:23.737 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 26/180 2024-03-12T17:04:23.807 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/180 2024-03-12T17:04:23.870 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/180 2024-03-12T17:04:23.976 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 29/180 2024-03-12T17:04:24.023 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 30/180 2024-03-12T17:04:24.057 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 31/180 2024-03-12T17:04:24.172 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 32/180 2024-03-12T17:04:24.212 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-ceph-common-2:18.2.2-797.g49e4a679.el9.x 33/180 2024-03-12T17:04:24.257 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-ceph-argparse-2:18.2.2-797.g49e4a679.el9 34/180 2024-03-12T17:04:24.375 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cephfs-2:18.2.2-797.g49e4a679.el9.x86_64 35/180 2024-03-12T17:04:24.619 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/180 2024-03-12T17:04:24.689 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/180 2024-03-12T17:04:24.870 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-2:18.2.2-797.g49e4a679.el9.x86_64 113/180 2024-03-12T17:04:24.918 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-common-2:18.2.2-797.g49e4a679.el9.x86_64 113/180 2024-03-12T17:04:25.409 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/180 2024-03-12T17:04:25.508 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/180 2024-03-12T17:04:25.547 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-2:18.2.2-797.g49e4a679.el9.x86_64 113/180 2024-03-12T17:04:25.596 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-base-2:18.2.2-797.g49e4a679.el9.x86_64 114/180 2024-03-12T17:04:25.631 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/180 2024-03-12T17:04:25.677 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 41/180 2024-03-12T17:04:25.711 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 41/180 2024-03-12T17:04:25.747 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-base-2:18.2.2-797.g49e4a679.el9.x86_64 114/180 2024-03-12T17:04:25.747 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-12T17:04:25.747 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-12T17:04:25.747 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:04:25.771 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 42/180 2024-03-12T17:04:25.822 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-selinux-2:18.2.2-797.g49e4a679.el9.x86_64 115/180 2024-03-12T17:04:25.824 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librados-devel-2:18.2.2-797.g49e4a679.el9.x86_64 43/180 2024-03-12T17:04:25.883 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/180 2024-03-12T17:04:26.292 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 45/180 2024-03-12T17:04:26.499 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 46/180 2024-03-12T17:04:26.589 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 47/180 2024-03-12T17:04:26.646 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 48/180 2024-03-12T17:04:26.852 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/180 2024-03-12T17:04:26.907 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 50/180 2024-03-12T17:04:27.080 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : re2-1:20211101-3.el9.x86_64 51/180 2024-03-12T17:04:27.119 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 52/180 2024-03-12T17:04:27.189 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/180 2024-03-12T17:04:27.292 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/180 2024-03-12T17:04:27.782 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/180 2024-03-12T17:04:27.836 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/180 2024-03-12T17:04:27.921 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/180 2024-03-12T17:04:27.990 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/180 2024-03-12T17:04:28.049 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 59/180 2024-03-12T17:04:28.273 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 60/180 2024-03-12T17:04:28.326 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 61/180 2024-03-12T17:04:28.376 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 62/180 2024-03-12T17:04:28.520 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 63/180 2024-03-12T17:04:28.566 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 64/180 2024-03-12T17:04:28.633 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 65/180 2024-03-12T17:04:28.665 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 66/180 2024-03-12T17:04:28.695 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 67/180 2024-03-12T17:04:28.867 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : fmt-8.1.1-5.el9.x86_64 68/180 2024-03-12T17:04:28.931 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 69/180 2024-03-12T17:04:29.012 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-ply-3.11-14.el9.noarch 70/180 2024-03-12T17:04:29.094 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 71/180 2024-03-12T17:04:29.363 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 72/180 2024-03-12T17:04:29.425 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 73/180 2024-03-12T17:04:29.604 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 74/180 2024-03-12T17:04:29.655 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 75/180 2024-03-12T17:04:29.695 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 76/180 2024-03-12T17:04:29.734 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : fuse-2.9.9-15.el9.x86_64 77/180 2024-03-12T17:04:29.773 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 78/180 2024-03-12T17:04:29.819 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 79/180 2024-03-12T17:04:29.881 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : uuid-1.6.2-55.el9.x86_64 80/180 2024-03-12T17:04:29.930 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 81/180 2024-03-12T17:04:30.019 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 82/180 2024-03-12T17:04:30.064 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 83/180 2024-03-12T17:04:30.116 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 84/180 2024-03-12T17:04:30.174 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 85/180 2024-03-12T17:04:30.216 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 86/180 2024-03-12T17:04:30.365 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 87/180 2024-03-12T17:04:30.408 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 88/180 2024-03-12T17:04:30.462 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 89/180 2024-03-12T17:04:30.613 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-packaging-20.9-5.el9.noarch 90/180 2024-03-12T17:04:30.681 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 91/180 2024-03-12T17:04:31.705 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/180 2024-03-12T17:04:31.754 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 93/180 2024-03-12T17:04:31.788 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 94/180 2024-03-12T17:04:31.820 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 95/180 2024-03-12T17:04:32.070 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : openblas-0.3.21-2.el9.x86_64 96/180 2024-03-12T17:04:32.091 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 97/180 2024-03-12T17:04:32.215 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 98/180 2024-03-12T17:04:33.058 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 99/180 2024-03-12T17:04:33.221 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 100/180 2024-03-12T17:04:34.852 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 101/180 2024-03-12T17:04:34.945 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 102/180 2024-03-12T17:04:34.974 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 103/180 2024-03-12T17:04:35.005 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : jq-1.6-16.el9.x86_64 104/180 2024-03-12T17:04:35.046 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 105/180 2024-03-12T17:04:35.427 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 106/180 2024-03-12T17:04:35.468 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : librbd1-2:18.2.2-797.g49e4a679.el9.x86_64 107/180 2024-03-12T17:04:35.590 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librbd1-2:18.2.2-797.g49e4a679.el9.x86_64 107/180 2024-03-12T17:04:35.625 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rbd-2:18.2.2-797.g49e4a679.el9.x86_64 108/180 2024-03-12T17:04:35.707 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 109/180 2024-03-12T17:04:36.209 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 110/180 2024-03-12T17:04:36.237 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librgw2-2:18.2.2-797.g49e4a679.el9.x86_64 111/180 2024-03-12T17:04:36.330 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librgw2-2:18.2.2-797.g49e4a679.el9.x86_64 111/180 2024-03-12T17:04:36.356 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rgw-2:18.2.2-797.g49e4a679.el9.x86_64 112/180 2024-03-12T17:04:38.472 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-selinux-2:18.2.2-797.g49e4a679.el9.x86_64 115/180 2024-03-12T17:04:38.472 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /sys 2024-03-12T17:04:38.472 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /proc 2024-03-12T17:04:38.472 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /mnt 2024-03-12T17:04:38.472 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /var/tmp 2024-03-12T17:04:38.473 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /home 2024-03-12T17:04:38.473 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /root 2024-03-12T17:04:38.473 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /tmp 2024-03-12T17:04:38.473 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:04:38.727 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-volume-2:18.2.2-797.g49e4a679.el9.noarch 116/180 2024-03-12T17:04:38.773 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-volume-2:18.2.2-797.g49e4a679.el9.noarch 116/180 2024-03-12T17:04:38.773 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T17:04:38.773 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-12T17:04:38.773 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:04:38.820 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-common-2:18.2.2-797.g49e4a679.el9.x86_64 113/180 2024-03-12T17:04:38.852 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-common-2:18.2.2-797.g49e4a679.el9.x86_64 113/180 2024-03-12T17:04:39.506 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-common-2:18.2.2-797.g49e4a679.el9.x86_64 113/180 2024-03-12T17:04:39.556 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-base-2:18.2.2-797.g49e4a679.el9.x86_64 114/180 2024-03-12T17:04:39.740 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-base-2:18.2.2-797.g49e4a679.el9.x86_64 114/180 2024-03-12T17:04:39.740 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-12T17:04:39.740 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-12T17:04:39.740 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:04:39.773 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-selinux-2:18.2.2-797.g49e4a679.el9.x86_64 115/180 2024-03-12T17:04:40.497 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-osd-2:18.2.2-797.g49e4a679.el9.x86_64 117/180 2024-03-12T17:04:40.579 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-osd-2:18.2.2-797.g49e4a679.el9.x86_64 117/180 2024-03-12T17:04:40.579 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T17:04:40.579 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-12T17:04:40.579 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-12T17:04:40.579 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-12T17:04:40.579 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:04:40.805 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mds-2:18.2.2-797.g49e4a679.el9.x86_64 118/180 2024-03-12T17:04:40.873 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mds-2:18.2.2-797.g49e4a679.el9.x86_64 118/180 2024-03-12T17:04:40.873 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T17:04:40.874 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-12T17:04:40.874 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-12T17:04:40.874 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-12T17:04:40.874 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:04:41.372 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mon-2:18.2.2-797.g49e4a679.el9.x86_64 119/180 2024-03-12T17:04:41.439 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mon-2:18.2.2-797.g49e4a679.el9.x86_64 119/180 2024-03-12T17:04:41.439 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T17:04:41.439 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-12T17:04:41.440 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-12T17:04:41.440 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-12T17:04:41.440 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:04:41.583 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : automake-1.16.2-8.el9.noarch 120/180 2024-03-12T17:04:42.196 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: cephadm-2:18.2.2-797.g49e4a679.el9.noarch 121/180 2024-03-12T17:04:42.243 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephadm-2:18.2.2-797.g49e4a679.el9.noarch 121/180 2024-03-12T17:04:42.291 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-797.g49e4a679.el 122/180 2024-03-12T17:04:42.407 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-797.g49e4a679.e 123/180 2024-03-12T17:04:42.438 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-797.g49e4a679.el9.noar 124/180 2024-03-12T17:04:42.742 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-797.g49e4a679.el9.noar 124/180 2024-03-12T17:04:42.780 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-797.g49e4a679.el9.no 125/180 2024-03-12T17:04:43.877 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-797.g49e4a679.el9.no 125/180 2024-03-12T17:04:43.914 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-797.g49e4 126/180 2024-03-12T17:04:44.055 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-797.g49e4 126/180 2024-03-12T17:04:44.221 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-797.g49e4a679.el9 127/180 2024-03-12T17:04:44.266 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-2:18.2.2-797.g49e4a679.el9.x86_64 128/180 2024-03-12T17:04:44.338 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-2:18.2.2-797.g49e4a679.el9.x86_64 128/180 2024-03-12T17:04:44.339 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T17:04:44.339 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-12T17:04:44.339 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-12T17:04:44.339 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-12T17:04:44.339 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:04:44.392 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-rook-2:18.2.2-797.g49e4a679.el9.noarch 129/180 2024-03-12T17:04:44.431 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-797.g49e4a679.el9.noarch 129/180 2024-03-12T17:04:44.546 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-2:18.2.2-797.g49e4a679.el9.x86_64 130/180 2024-03-12T17:04:45.340 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libtool-2.4.6-45.el9.x86_64 131/180 2024-03-12T17:04:45.377 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-radosgw-2:18.2.2-797.g49e4a679.el9.x86_64 132/180 2024-03-12T17:04:45.444 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-797.g49e4a679.el9.x86_64 132/180 2024-03-12T17:04:45.445 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T17:04:45.445 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-12T17:04:45.445 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-12T17:04:45.445 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-12T17:04:45.445 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:04:45.503 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-797.g49e4a6 133/180 2024-03-12T17:04:45.572 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-797.g49e4a6 133/180 2024-03-12T17:04:45.573 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T17:04:45.573 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-12T17:04:45.573 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:04:45.654 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephfs-mirror-2:18.2.2-797.g49e4a679.el9.x86_64 134/180 2024-03-12T17:04:45.751 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-797.g49e4a679.el9.x86_64 134/180 2024-03-12T17:04:45.752 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T17:04:45.752 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-12T17:04:45.752 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-12T17:04:45.752 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-12T17:04:45.752 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:04:46.112 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-mirror-2:18.2.2-797.g49e4a679.el9.x86_64 135/180 2024-03-12T17:04:46.184 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: rbd-mirror-2:18.2.2-797.g49e4a679.el9.x86_64 135/180 2024-03-12T17:04:46.184 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T17:04:46.184 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-12T17:04:46.184 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-12T17:04:46.184 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-12T17:04:46.184 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:04:50.168 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-test-2:18.2.2-797.g49e4a679.el9.x86_64 136/180 2024-03-12T17:04:50.215 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-fuse-2:18.2.2-797.g49e4a679.el9.x86_64 137/180 2024-03-12T17:04:50.274 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-nbd-2:18.2.2-797.g49e4a679.el9.x86_64 138/180 2024-03-12T17:04:50.542 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 139/180 2024-03-12T17:04:50.682 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 140/180 2024-03-12T17:04:50.743 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-fuse-2:18.2.2-797.g49e4a679.el9.x86_64 141/180 2024-03-12T17:04:50.810 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/180 2024-03-12T17:04:50.901 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 143/180 2024-03-12T17:04:50.955 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 144/180 2024-03-12T17:04:50.998 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs-devel-2:18.2.2-797.g49e4a679.el9.x86_6 145/180 2024-03-12T17:04:51.043 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 146/180 2024-03-12T17:04:51.094 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 147/180 2024-03-12T17:04:51.185 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 148/180 2024-03-12T17:04:51.323 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 149/180 2024-03-12T17:04:51.376 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 150/180 2024-03-12T17:04:52.381 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 151/180 2024-03-12T17:04:52.423 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 151/180 2024-03-12T17:04:52.478 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 151/180 2024-03-12T17:04:52.999 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-selinux-2:18.2.2-797.g49e4a679.el9.x86_64 115/180 2024-03-12T17:04:52.999 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /sys 2024-03-12T17:04:52.999 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /proc 2024-03-12T17:04:52.999 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /mnt 2024-03-12T17:04:52.999 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /var/tmp 2024-03-12T17:04:52.999 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /home 2024-03-12T17:04:52.999 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /root 2024-03-12T17:04:52.999 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /tmp 2024-03-12T17:04:53.000 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:04:53.226 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-volume-2:18.2.2-797.g49e4a679.el9.noarch 116/180 2024-03-12T17:04:53.274 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-volume-2:18.2.2-797.g49e4a679.el9.noarch 116/180 2024-03-12T17:04:53.274 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T17:04:53.274 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-12T17:04:53.274 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:04:53.808 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephfs-top-2:18.2.2-797.g49e4a679.el9.noarch 152/180 2024-03-12T17:04:53.914 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 153/180 2024-03-12T17:04:53.967 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/180 2024-03-12T17:04:54.100 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/180 2024-03-12T17:04:54.150 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/180 2024-03-12T17:04:54.183 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 157/180 2024-03-12T17:04:54.217 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 158/180 2024-03-12T17:04:54.259 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 159/180 2024-03-12T17:04:54.260 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/180 2024-03-12T17:04:54.404 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/180 2024-03-12T17:04:54.472 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 161/180 2024-03-12T17:04:54.472 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 162/180 2024-03-12T17:04:54.602 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 162/180 2024-03-12T17:04:54.646 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 163/180 2024-03-12T17:04:54.691 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 164/180 2024-03-12T17:04:54.750 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 165/180 2024-03-12T17:04:54.782 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 166/180 2024-03-12T17:04:54.850 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 167/180 2024-03-12T17:04:54.932 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 168/180 2024-03-12T17:04:54.987 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 169/180 2024-03-12T17:04:55.019 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-osd-2:18.2.2-797.g49e4a679.el9.x86_64 117/180 2024-03-12T17:04:55.020 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 170/180 2024-03-12T17:04:55.086 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 171/180 2024-03-12T17:04:55.098 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-osd-2:18.2.2-797.g49e4a679.el9.x86_64 117/180 2024-03-12T17:04:55.098 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T17:04:55.098 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-12T17:04:55.099 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-12T17:04:55.099 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-12T17:04:55.099 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:04:55.121 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 172/180 2024-03-12T17:04:55.154 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 173/180 2024-03-12T17:04:55.201 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 174/180 2024-03-12T17:04:55.201 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 174/180 2024-03-12T17:04:55.328 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mds-2:18.2.2-797.g49e4a679.el9.x86_64 118/180 2024-03-12T17:04:55.400 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mds-2:18.2.2-797.g49e4a679.el9.x86_64 118/180 2024-03-12T17:04:55.400 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T17:04:55.400 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-12T17:04:55.401 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-12T17:04:55.401 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-12T17:04:55.401 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:04:55.513 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 174/180 2024-03-12T17:04:55.513 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 175/180 2024-03-12T17:04:55.652 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 175/180 2024-03-12T17:04:55.682 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 176/180 2024-03-12T17:04:55.741 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 177/180 2024-03-12T17:04:55.777 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 178/180 2024-03-12T17:04:55.817 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 179/180 2024-03-12T17:04:55.936 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 180/180 2024-03-12T17:04:55.954 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mon-2:18.2.2-797.g49e4a679.el9.x86_64 119/180 2024-03-12T17:04:56.028 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mon-2:18.2.2-797.g49e4a679.el9.x86_64 119/180 2024-03-12T17:04:56.028 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T17:04:56.028 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-12T17:04:56.028 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-12T17:04:56.029 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-12T17:04:56.029 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:04:56.156 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : automake-1.16.2-8.el9.noarch 120/180 2024-03-12T17:04:56.204 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 180/180 2024-03-12T17:04:56.204 INFO:teuthology.orchestra.run.smithi102.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-12T17:04:56.205 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:04:56.762 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: cephadm-2:18.2.2-797.g49e4a679.el9.noarch 121/180 2024-03-12T17:04:56.791 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cephadm-2:18.2.2-797.g49e4a679.el9.noarch 121/180 2024-03-12T17:04:56.831 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-797.g49e4a679.el 122/180 2024-03-12T17:04:56.936 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-797.g49e4a679.e 123/180 2024-03-12T17:04:57.002 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-797.g49e4a679.el9.noar 124/180 2024-03-12T17:04:57.332 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-797.g49e4a679.el9.noar 124/180 2024-03-12T17:04:57.369 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-797.g49e4a679.el9.no 125/180 2024-03-12T17:04:58.549 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-797.g49e4a679.el9.no 125/180 2024-03-12T17:04:58.586 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-797.g49e4 126/180 2024-03-12T17:04:58.738 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-797.g49e4 126/180 2024-03-12T17:04:58.908 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-797.g49e4a679.el9 127/180 2024-03-12T17:04:58.947 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-2:18.2.2-797.g49e4a679.el9.x86_64 128/180 2024-03-12T17:04:59.017 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-2:18.2.2-797.g49e4a679.el9.x86_64 128/180 2024-03-12T17:04:59.017 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T17:04:59.017 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-12T17:04:59.017 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-12T17:04:59.017 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-12T17:04:59.017 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:04:59.072 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-rook-2:18.2.2-797.g49e4a679.el9.noarch 129/180 2024-03-12T17:04:59.112 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-797.g49e4a679.el9.noarch 129/180 2024-03-12T17:04:59.201 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-2:18.2.2-797.g49e4a679.el9.x86_64 130/180 2024-03-12T17:05:00.071 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libtool-2.4.6-45.el9.x86_64 131/180 2024-03-12T17:05:00.107 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-radosgw-2:18.2.2-797.g49e4a679.el9.x86_64 132/180 2024-03-12T17:05:00.179 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-797.g49e4a679.el9.x86_64 132/180 2024-03-12T17:05:00.179 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T17:05:00.179 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-12T17:05:00.179 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-12T17:05:00.179 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-12T17:05:00.179 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:00.241 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-797.g49e4a6 133/180 2024-03-12T17:05:00.317 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-797.g49e4a6 133/180 2024-03-12T17:05:00.317 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T17:05:00.317 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-12T17:05:00.317 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:00.392 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cephfs-mirror-2:18.2.2-797.g49e4a679.el9.x86_64 134/180 2024-03-12T17:05:00.471 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-797.g49e4a679.el9.x86_64 134/180 2024-03-12T17:05:00.471 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T17:05:00.471 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-12T17:05:00.471 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-12T17:05:00.471 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-12T17:05:00.471 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:00.859 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : rbd-mirror-2:18.2.2-797.g49e4a679.el9.x86_64 135/180 2024-03-12T17:05:00.936 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: rbd-mirror-2:18.2.2-797.g49e4a679.el9.x86_64 135/180 2024-03-12T17:05:00.936 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-12T17:05:00.936 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-12T17:05:00.936 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-12T17:05:00.936 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-12T17:05:00.936 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:01.008 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-2:18.2.2-797.g49e4a679.el9.x86_64 1/180 2024-03-12T17:05:01.008 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-base-2:18.2.2-797.g49e4a679.el9.x86_64 2/180 2024-03-12T17:05:01.009 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-common-2:18.2.2-797.g49e4a679.el9.x86_64 3/180 2024-03-12T17:05:01.009 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-fuse-2:18.2.2-797.g49e4a679.el9.x86_64 4/180 2024-03-12T17:05:01.009 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-797.g49e4a6 5/180 2024-03-12T17:05:01.009 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mds-2:18.2.2-797.g49e4a679.el9.x86_64 6/180 2024-03-12T17:05:01.009 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-2:18.2.2-797.g49e4a679.el9.x86_64 7/180 2024-03-12T17:05:01.009 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mon-2:18.2.2-797.g49e4a679.el9.x86_64 8/180 2024-03-12T17:05:01.009 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-osd-2:18.2.2-797.g49e4a679.el9.x86_64 9/180 2024-03-12T17:05:01.009 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-radosgw-2:18.2.2-797.g49e4a679.el9.x86_64 10/180 2024-03-12T17:05:01.009 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-selinux-2:18.2.2-797.g49e4a679.el9.x86_64 11/180 2024-03-12T17:05:01.009 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-test-2:18.2.2-797.g49e4a679.el9.x86_64 12/180 2024-03-12T17:05:01.009 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephfs-mirror-2:18.2.2-797.g49e4a679.el9.x86_64 13/180 2024-03-12T17:05:01.009 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs-devel-2:18.2.2-797.g49e4a679.el9.x86_6 14/180 2024-03-12T17:05:01.009 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs2-2:18.2.2-797.g49e4a679.el9.x86_64 15/180 2024-03-12T17:05:01.010 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephsqlite-2:18.2.2-797.g49e4a679.el9.x86_64 16/180 2024-03-12T17:05:01.010 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados-devel-2:18.2.2-797.g49e4a679.el9.x86_64 17/180 2024-03-12T17:05:01.010 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libradosstriper1-2:18.2.2-797.g49e4a679.el9.x86_ 18/180 2024-03-12T17:05:01.010 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librgw2-2:18.2.2-797.g49e4a679.el9.x86_64 19/180 2024-03-12T17:05:01.010 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ceph-argparse-2:18.2.2-797.g49e4a679.el9 20/180 2024-03-12T17:05:01.010 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ceph-common-2:18.2.2-797.g49e4a679.el9.x 21/180 2024-03-12T17:05:01.010 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cephfs-2:18.2.2-797.g49e4a679.el9.x86_64 22/180 2024-03-12T17:05:01.010 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rados-2:18.2.2-797.g49e4a679.el9.x86_64 23/180 2024-03-12T17:05:01.010 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rbd-2:18.2.2-797.g49e4a679.el9.x86_64 24/180 2024-03-12T17:05:01.010 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rgw-2:18.2.2-797.g49e4a679.el9.x86_64 25/180 2024-03-12T17:05:01.010 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-fuse-2:18.2.2-797.g49e4a679.el9.x86_64 26/180 2024-03-12T17:05:01.010 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-mirror-2:18.2.2-797.g49e4a679.el9.x86_64 27/180 2024-03-12T17:05:01.010 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-nbd-2:18.2.2-797.g49e4a679.el9.x86_64 28/180 2024-03-12T17:05:01.011 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-797.g49e4a679.e 29/180 2024-03-12T17:05:01.011 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-797.g49e4a679.el9.noar 30/180 2024-03-12T17:05:01.011 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-797.g49e4a679.el9.no 31/180 2024-03-12T17:05:01.011 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-797.g49e4 32/180 2024-03-12T17:05:01.024 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-797.g49e4a679.el9 33/180 2024-03-12T17:05:01.024 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-rook-2:18.2.2-797.g49e4a679.el9.noarch 34/180 2024-03-12T17:05:01.024 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-797.g49e4a679.el 35/180 2024-03-12T17:05:01.025 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-volume-2:18.2.2-797.g49e4a679.el9.noarch 36/180 2024-03-12T17:05:01.025 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephadm-2:18.2.2-797.g49e4a679.el9.noarch 37/180 2024-03-12T17:05:01.025 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephfs-top-2:18.2.2-797.g49e4a679.el9.noarch 38/180 2024-03-12T17:05:01.025 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/180 2024-03-12T17:05:01.025 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/180 2024-03-12T17:05:01.025 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/180 2024-03-12T17:05:01.025 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flex-2.6.4-9.el9.x86_64 42/180 2024-03-12T17:05:01.025 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 43/180 2024-03-12T17:05:01.025 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 44/180 2024-03-12T17:05:01.025 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 45/180 2024-03-12T17:05:01.025 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 46/180 2024-03-12T17:05:01.025 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 47/180 2024-03-12T17:05:01.025 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 48/180 2024-03-12T17:05:01.026 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 49/180 2024-03-12T17:05:01.026 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 50/180 2024-03-12T17:05:01.026 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 51/180 2024-03-12T17:05:01.026 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 52/180 2024-03-12T17:05:01.026 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 53/180 2024-03-12T17:05:01.026 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 54/180 2024-03-12T17:05:01.026 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 55/180 2024-03-12T17:05:01.026 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-13.11-1.el9.x86_64 56/180 2024-03-12T17:05:01.026 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 57/180 2024-03-12T17:05:01.026 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 58/180 2024-03-12T17:05:01.026 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 59/180 2024-03-12T17:05:01.026 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/180 2024-03-12T17:05:01.026 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/180 2024-03-12T17:05:01.027 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/180 2024-03-12T17:05:01.027 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/180 2024-03-12T17:05:01.027 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/180 2024-03-12T17:05:01.027 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/180 2024-03-12T17:05:01.027 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/180 2024-03-12T17:05:01.029 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/180 2024-03-12T17:05:01.029 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/180 2024-03-12T17:05:01.029 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/180 2024-03-12T17:05:01.029 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 70/180 2024-03-12T17:05:01.029 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 71/180 2024-03-12T17:05:01.029 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 72/180 2024-03-12T17:05:01.029 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 73/180 2024-03-12T17:05:01.029 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 74/180 2024-03-12T17:05:01.029 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 75/180 2024-03-12T17:05:01.029 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 76/180 2024-03-12T17:05:01.029 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 77/180 2024-03-12T17:05:01.030 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 78/180 2024-03-12T17:05:01.030 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 79/180 2024-03-12T17:05:01.030 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 80/180 2024-03-12T17:05:01.030 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 81/180 2024-03-12T17:05:01.030 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 82/180 2024-03-12T17:05:01.030 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ply-3.11-14.el9.noarch 83/180 2024-03-12T17:05:01.030 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 84/180 2024-03-12T17:05:01.030 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 85/180 2024-03-12T17:05:01.030 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : jq-1.6-16.el9.x86_64 86/180 2024-03-12T17:05:01.030 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 87/180 2024-03-12T17:05:01.030 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 88/180 2024-03-12T17:05:01.030 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 89/180 2024-03-12T17:05:01.030 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 90/180 2024-03-12T17:05:01.031 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 91/180 2024-03-12T17:05:01.031 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 92/180 2024-03-12T17:05:01.031 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 93/180 2024-03-12T17:05:01.031 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 94/180 2024-03-12T17:05:01.031 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 95/180 2024-03-12T17:05:01.031 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 96/180 2024-03-12T17:05:01.031 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 97/180 2024-03-12T17:05:01.031 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 98/180 2024-03-12T17:05:01.031 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 99/180 2024-03-12T17:05:01.031 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 100/180 2024-03-12T17:05:01.031 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 101/180 2024-03-12T17:05:01.031 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 102/180 2024-03-12T17:05:01.031 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 103/180 2024-03-12T17:05:01.031 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 104/180 2024-03-12T17:05:01.032 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 105/180 2024-03-12T17:05:01.032 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 106/180 2024-03-12T17:05:01.032 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 107/180 2024-03-12T17:05:01.032 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 108/180 2024-03-12T17:05:01.032 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 109/180 2024-03-12T17:05:01.032 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 110/180 2024-03-12T17:05:01.032 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 111/180 2024-03-12T17:05:01.032 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 112/180 2024-03-12T17:05:01.032 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 113/180 2024-03-12T17:05:01.032 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 114/180 2024-03-12T17:05:01.032 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/180 2024-03-12T17:05:01.032 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/180 2024-03-12T17:05:01.032 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/180 2024-03-12T17:05:01.032 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/180 2024-03-12T17:05:01.033 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/180 2024-03-12T17:05:01.033 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/180 2024-03-12T17:05:01.033 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/180 2024-03-12T17:05:01.033 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/180 2024-03-12T17:05:01.033 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/180 2024-03-12T17:05:01.033 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/180 2024-03-12T17:05:01.033 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/180 2024-03-12T17:05:01.033 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/180 2024-03-12T17:05:01.033 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/180 2024-03-12T17:05:01.033 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/180 2024-03-12T17:05:01.033 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/180 2024-03-12T17:05:01.033 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/180 2024-03-12T17:05:01.033 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/180 2024-03-12T17:05:01.033 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/180 2024-03-12T17:05:01.034 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-2:18.2.2-797.g49e4a679.el9.x86_64 133/180 2024-03-12T17:05:01.034 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/180 2024-03-12T17:05:01.034 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-2:18.2.2-797.g49e4a679.el9.x86_64 135/180 2024-03-12T17:05:01.034 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/180 2024-03-12T17:05:01.034 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 137/180 2024-03-12T17:05:01.034 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 138/180 2024-03-12T17:05:01.034 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 139/180 2024-03-12T17:05:01.034 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 140/180 2024-03-12T17:05:01.034 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 141/180 2024-03-12T17:05:01.034 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 142/180 2024-03-12T17:05:01.034 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 143/180 2024-03-12T17:05:01.034 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 144/180 2024-03-12T17:05:01.034 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 145/180 2024-03-12T17:05:01.035 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 146/180 2024-03-12T17:05:01.035 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : acl-2.3.1-4.el9.x86_64 147/180 2024-03-12T17:05:01.035 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : acl-2.3.1-3.el9.x86_64 148/180 2024-03-12T17:05:01.035 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 149/180 2024-03-12T17:05:01.035 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 150/180 2024-03-12T17:05:01.035 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 151/180 2024-03-12T17:05:01.035 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 152/180 2024-03-12T17:05:01.035 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 153/180 2024-03-12T17:05:01.035 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 154/180 2024-03-12T17:05:01.035 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 155/180 2024-03-12T17:05:01.035 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 156/180 2024-03-12T17:05:01.035 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 157/180 2024-03-12T17:05:01.035 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 158/180 2024-03-12T17:05:01.035 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 159/180 2024-03-12T17:05:01.036 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 160/180 2024-03-12T17:05:01.036 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 161/180 2024-03-12T17:05:01.036 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 162/180 2024-03-12T17:05:01.036 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 163/180 2024-03-12T17:05:01.036 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 164/180 2024-03-12T17:05:01.036 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 165/180 2024-03-12T17:05:01.036 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 166/180 2024-03-12T17:05:01.036 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 167/180 2024-03-12T17:05:01.036 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 168/180 2024-03-12T17:05:01.036 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 169/180 2024-03-12T17:05:01.036 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 170/180 2024-03-12T17:05:01.036 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 171/180 2024-03-12T17:05:01.036 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 172/180 2024-03-12T17:05:01.036 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 173/180 2024-03-12T17:05:01.036 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 174/180 2024-03-12T17:05:01.037 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 175/180 2024-03-12T17:05:01.037 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 176/180 2024-03-12T17:05:01.037 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 177/180 2024-03-12T17:05:01.037 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 178/180 2024-03-12T17:05:01.037 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 179/180 2024-03-12T17:05:03.491 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 180/180 2024-03-12T17:05:03.491 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:05:03.491 INFO:teuthology.orchestra.run.smithi102.stdout:Upgraded: 2024-03-12T17:05:03.491 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-12T17:05:03.491 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-12T17:05:03.491 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-12T17:05:03.491 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-12T17:05:03.491 INFO:teuthology.orchestra.run.smithi102.stdout: acl-2.3.1-4.el9.x86_64 2024-03-12T17:05:03.492 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-12T17:05:03.492 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-12T17:05:03.492 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-12T17:05:03.492 INFO:teuthology.orchestra.run.smithi102.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-12T17:05:03.492 INFO:teuthology.orchestra.run.smithi102.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-12T17:05:03.492 INFO:teuthology.orchestra.run.smithi102.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-12T17:05:03.492 INFO:teuthology.orchestra.run.smithi102.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-12T17:05:03.492 INFO:teuthology.orchestra.run.smithi102.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-12T17:05:03.492 INFO:teuthology.orchestra.run.smithi102.stdout: librados2-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:03.492 INFO:teuthology.orchestra.run.smithi102.stdout: librbd1-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:03.492 INFO:teuthology.orchestra.run.smithi102.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-12T17:05:03.492 INFO:teuthology.orchestra.run.smithi102.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-12T17:05:03.492 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-12T17:05:03.493 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-12T17:05:03.493 INFO:teuthology.orchestra.run.smithi102.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-12T17:05:03.493 INFO:teuthology.orchestra.run.smithi102.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-12T17:05:03.493 INFO:teuthology.orchestra.run.smithi102.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-12T17:05:03.493 INFO:teuthology.orchestra.run.smithi102.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-12T17:05:03.493 INFO:teuthology.orchestra.run.smithi102.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-12T17:05:03.493 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-03-12T17:05:03.493 INFO:teuthology.orchestra.run.smithi102.stdout: automake-1.16.2-8.el9.noarch 2024-03-12T17:05:03.493 INFO:teuthology.orchestra.run.smithi102.stdout: bison-3.7.4-5.el9.x86_64 2024-03-12T17:05:03.493 INFO:teuthology.orchestra.run.smithi102.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-12T17:05:03.493 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:03.493 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:03.494 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:03.494 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:03.494 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-grafana-dashboards-2:18.2.2-797.g49e4a679.el9.noarch 2024-03-12T17:05:03.494 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-immutable-object-cache-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:03.494 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:03.494 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:03.494 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-cephadm-2:18.2.2-797.g49e4a679.el9.noarch 2024-03-12T17:05:03.494 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-dashboard-2:18.2.2-797.g49e4a679.el9.noarch 2024-03-12T17:05:03.494 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-diskprediction-local-2:18.2.2-797.g49e4a679.el9.noarch 2024-03-12T17:05:03.494 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-modules-core-2:18.2.2-797.g49e4a679.el9.noarch 2024-03-12T17:05:03.494 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-rook-2:18.2.2-797.g49e4a679.el9.noarch 2024-03-12T17:05:03.494 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:03.494 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:03.495 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-prometheus-alerts-2:18.2.2-797.g49e4a679.el9.noarch 2024-03-12T17:05:03.495 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:03.495 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-selinux-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:03.495 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:03.495 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-volume-2:18.2.2-797.g49e4a679.el9.noarch 2024-03-12T17:05:03.495 INFO:teuthology.orchestra.run.smithi102.stdout: cephadm-2:18.2.2-797.g49e4a679.el9.noarch 2024-03-12T17:05:03.495 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-mirror-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:03.495 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-top-2:18.2.2-797.g49e4a679.el9.noarch 2024-03-12T17:05:03.495 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-12T17:05:03.495 INFO:teuthology.orchestra.run.smithi102.stdout: flex-2.6.4-9.el9.x86_64 2024-03-12T17:05:03.495 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-12T17:05:03.495 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-12T17:05:03.495 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-12T17:05:03.496 INFO:teuthology.orchestra.run.smithi102.stdout: fmt-8.1.1-5.el9.x86_64 2024-03-12T17:05:03.496 INFO:teuthology.orchestra.run.smithi102.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-12T17:05:03.496 INFO:teuthology.orchestra.run.smithi102.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-12T17:05:03.496 INFO:teuthology.orchestra.run.smithi102.stdout: jq-1.6-16.el9.x86_64 2024-03-12T17:05:03.496 INFO:teuthology.orchestra.run.smithi102.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-12T17:05:03.496 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-12T17:05:03.496 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-12T17:05:03.496 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs-devel-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:03.496 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:03.496 INFO:teuthology.orchestra.run.smithi102.stdout: libcephsqlite-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:03.496 INFO:teuthology.orchestra.run.smithi102.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-12T17:05:03.496 INFO:teuthology.orchestra.run.smithi102.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-12T17:05:03.496 INFO:teuthology.orchestra.run.smithi102.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-12T17:05:03.497 INFO:teuthology.orchestra.run.smithi102.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-12T17:05:03.497 INFO:teuthology.orchestra.run.smithi102.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-12T17:05:03.497 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:03.497 INFO:teuthology.orchestra.run.smithi102.stdout: libradosstriper1-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:03.497 INFO:teuthology.orchestra.run.smithi102.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-12T17:05:03.497 INFO:teuthology.orchestra.run.smithi102.stdout: librgw2-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:03.497 INFO:teuthology.orchestra.run.smithi102.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-12T17:05:03.497 INFO:teuthology.orchestra.run.smithi102.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-12T17:05:03.497 INFO:teuthology.orchestra.run.smithi102.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-12T17:05:03.497 INFO:teuthology.orchestra.run.smithi102.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-12T17:05:03.497 INFO:teuthology.orchestra.run.smithi102.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-12T17:05:03.497 INFO:teuthology.orchestra.run.smithi102.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-12T17:05:03.497 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-12T17:05:03.497 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-12T17:05:03.498 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-12T17:05:03.498 INFO:teuthology.orchestra.run.smithi102.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-12T17:05:03.498 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-13.11-1.el9.x86_64 2024-03-12T17:05:03.498 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-12T17:05:03.498 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-12T17:05:03.498 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-12T17:05:03.498 INFO:teuthology.orchestra.run.smithi102.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-12T17:05:03.498 INFO:teuthology.orchestra.run.smithi102.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-12T17:05:03.498 INFO:teuthology.orchestra.run.smithi102.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-12T17:05:03.498 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-12T17:05:03.498 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-argparse-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:03.498 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-common-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:03.498 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:03.498 INFO:teuthology.orchestra.run.smithi102.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-12T17:05:03.499 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-12T17:05:03.499 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-12T17:05:03.499 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-12T17:05:03.499 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-12T17:05:03.499 INFO:teuthology.orchestra.run.smithi102.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-12T17:05:03.499 INFO:teuthology.orchestra.run.smithi102.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-12T17:05:03.499 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-12T17:05:03.499 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-12T17:05:03.499 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-12T17:05:03.499 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-12T17:05:03.499 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-12T17:05:03.499 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-12T17:05:03.499 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-12T17:05:03.499 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-12T17:05:03.499 INFO:teuthology.orchestra.run.smithi102.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-12T17:05:03.500 INFO:teuthology.orchestra.run.smithi102.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-12T17:05:03.500 INFO:teuthology.orchestra.run.smithi102.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-12T17:05:03.500 INFO:teuthology.orchestra.run.smithi102.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-12T17:05:03.500 INFO:teuthology.orchestra.run.smithi102.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-12T17:05:03.500 INFO:teuthology.orchestra.run.smithi102.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-12T17:05:03.500 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-12T17:05:03.500 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-12T17:05:03.500 INFO:teuthology.orchestra.run.smithi102.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-12T17:05:03.500 INFO:teuthology.orchestra.run.smithi102.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-12T17:05:03.500 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-12T17:05:03.500 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ply-3.11-14.el9.noarch 2024-03-12T17:05:03.500 INFO:teuthology.orchestra.run.smithi102.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-12T17:05:03.500 INFO:teuthology.orchestra.run.smithi102.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-12T17:05:03.500 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-12T17:05:03.501 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-12T17:05:03.501 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-12T17:05:03.501 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-12T17:05:03.501 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-12T17:05:03.501 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:03.501 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:03.501 INFO:teuthology.orchestra.run.smithi102.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-12T17:05:03.501 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-12T17:05:03.501 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-12T17:05:03.501 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:03.501 INFO:teuthology.orchestra.run.smithi102.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-12T17:05:03.501 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-12T17:05:03.501 INFO:teuthology.orchestra.run.smithi102.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-12T17:05:03.501 INFO:teuthology.orchestra.run.smithi102.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-12T17:05:03.502 INFO:teuthology.orchestra.run.smithi102.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-12T17:05:03.502 INFO:teuthology.orchestra.run.smithi102.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-12T17:05:03.502 INFO:teuthology.orchestra.run.smithi102.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-12T17:05:03.502 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-12T17:05:03.502 INFO:teuthology.orchestra.run.smithi102.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-12T17:05:03.502 INFO:teuthology.orchestra.run.smithi102.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-12T17:05:03.502 INFO:teuthology.orchestra.run.smithi102.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-12T17:05:03.502 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:03.502 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:03.502 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:03.502 INFO:teuthology.orchestra.run.smithi102.stdout: re2-1:20211101-3.el9.x86_64 2024-03-12T17:05:03.502 INFO:teuthology.orchestra.run.smithi102.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-12T17:05:03.502 INFO:teuthology.orchestra.run.smithi102.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-12T17:05:03.503 INFO:teuthology.orchestra.run.smithi102.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-12T17:05:03.503 INFO:teuthology.orchestra.run.smithi102.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-12T17:05:03.503 INFO:teuthology.orchestra.run.smithi102.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-12T17:05:03.503 INFO:teuthology.orchestra.run.smithi102.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-12T17:05:03.503 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:05:03.503 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-03-12T17:05:04.129 DEBUG:teuthology.parallel:result is None 2024-03-12T17:05:05.173 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-test-2:18.2.2-797.g49e4a679.el9.x86_64 136/180 2024-03-12T17:05:05.229 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : rbd-fuse-2:18.2.2-797.g49e4a679.el9.x86_64 137/180 2024-03-12T17:05:05.262 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : rbd-nbd-2:18.2.2-797.g49e4a679.el9.x86_64 138/180 2024-03-12T17:05:05.549 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 139/180 2024-03-12T17:05:05.676 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 140/180 2024-03-12T17:05:05.723 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-fuse-2:18.2.2-797.g49e4a679.el9.x86_64 141/180 2024-03-12T17:05:05.778 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/180 2024-03-12T17:05:05.838 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 143/180 2024-03-12T17:05:05.899 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 144/180 2024-03-12T17:05:05.934 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libcephfs-devel-2:18.2.2-797.g49e4a679.el9.x86_6 145/180 2024-03-12T17:05:05.980 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 146/180 2024-03-12T17:05:06.032 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 147/180 2024-03-12T17:05:06.150 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 148/180 2024-03-12T17:05:06.272 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 149/180 2024-03-12T17:05:06.313 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 150/180 2024-03-12T17:05:07.251 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 151/180 2024-03-12T17:05:07.293 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 151/180 2024-03-12T17:05:07.363 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 151/180 2024-03-12T17:05:08.786 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cephfs-top-2:18.2.2-797.g49e4a679.el9.noarch 152/180 2024-03-12T17:05:08.889 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 153/180 2024-03-12T17:05:08.947 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/180 2024-03-12T17:05:09.087 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/180 2024-03-12T17:05:09.136 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/180 2024-03-12T17:05:09.169 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 157/180 2024-03-12T17:05:09.203 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 158/180 2024-03-12T17:05:09.247 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 159/180 2024-03-12T17:05:09.247 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/180 2024-03-12T17:05:09.399 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/180 2024-03-12T17:05:09.440 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 161/180 2024-03-12T17:05:09.440 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 162/180 2024-03-12T17:05:09.610 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 162/180 2024-03-12T17:05:09.665 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 163/180 2024-03-12T17:05:09.710 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 164/180 2024-03-12T17:05:09.743 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 165/180 2024-03-12T17:05:09.775 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 166/180 2024-03-12T17:05:09.846 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 167/180 2024-03-12T17:05:09.924 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 168/180 2024-03-12T17:05:09.980 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 169/180 2024-03-12T17:05:10.013 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 170/180 2024-03-12T17:05:10.047 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 171/180 2024-03-12T17:05:10.106 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 172/180 2024-03-12T17:05:10.139 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 173/180 2024-03-12T17:05:10.183 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 174/180 2024-03-12T17:05:10.183 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 174/180 2024-03-12T17:05:10.498 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 174/180 2024-03-12T17:05:10.498 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 175/180 2024-03-12T17:05:10.661 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 175/180 2024-03-12T17:05:10.700 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 176/180 2024-03-12T17:05:10.734 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 177/180 2024-03-12T17:05:10.770 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 178/180 2024-03-12T17:05:10.810 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 179/180 2024-03-12T17:05:10.941 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 180/180 2024-03-12T17:05:11.197 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 180/180 2024-03-12T17:05:11.197 INFO:teuthology.orchestra.run.smithi052.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-12T17:05:11.197 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:15.900 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-2:18.2.2-797.g49e4a679.el9.x86_64 1/180 2024-03-12T17:05:15.900 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-base-2:18.2.2-797.g49e4a679.el9.x86_64 2/180 2024-03-12T17:05:15.900 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-common-2:18.2.2-797.g49e4a679.el9.x86_64 3/180 2024-03-12T17:05:15.900 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-fuse-2:18.2.2-797.g49e4a679.el9.x86_64 4/180 2024-03-12T17:05:15.900 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-797.g49e4a6 5/180 2024-03-12T17:05:15.901 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mds-2:18.2.2-797.g49e4a679.el9.x86_64 6/180 2024-03-12T17:05:15.901 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-2:18.2.2-797.g49e4a679.el9.x86_64 7/180 2024-03-12T17:05:15.901 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mon-2:18.2.2-797.g49e4a679.el9.x86_64 8/180 2024-03-12T17:05:15.901 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-osd-2:18.2.2-797.g49e4a679.el9.x86_64 9/180 2024-03-12T17:05:15.901 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-radosgw-2:18.2.2-797.g49e4a679.el9.x86_64 10/180 2024-03-12T17:05:15.901 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-selinux-2:18.2.2-797.g49e4a679.el9.x86_64 11/180 2024-03-12T17:05:15.901 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-test-2:18.2.2-797.g49e4a679.el9.x86_64 12/180 2024-03-12T17:05:15.901 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cephfs-mirror-2:18.2.2-797.g49e4a679.el9.x86_64 13/180 2024-03-12T17:05:15.901 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libcephfs-devel-2:18.2.2-797.g49e4a679.el9.x86_6 14/180 2024-03-12T17:05:15.901 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libcephfs2-2:18.2.2-797.g49e4a679.el9.x86_64 15/180 2024-03-12T17:05:15.901 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libcephsqlite-2:18.2.2-797.g49e4a679.el9.x86_64 16/180 2024-03-12T17:05:15.901 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librados-devel-2:18.2.2-797.g49e4a679.el9.x86_64 17/180 2024-03-12T17:05:15.902 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libradosstriper1-2:18.2.2-797.g49e4a679.el9.x86_ 18/180 2024-03-12T17:05:15.902 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librgw2-2:18.2.2-797.g49e4a679.el9.x86_64 19/180 2024-03-12T17:05:15.902 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-ceph-argparse-2:18.2.2-797.g49e4a679.el9 20/180 2024-03-12T17:05:15.902 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-ceph-common-2:18.2.2-797.g49e4a679.el9.x 21/180 2024-03-12T17:05:15.902 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cephfs-2:18.2.2-797.g49e4a679.el9.x86_64 22/180 2024-03-12T17:05:15.902 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rados-2:18.2.2-797.g49e4a679.el9.x86_64 23/180 2024-03-12T17:05:15.902 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rbd-2:18.2.2-797.g49e4a679.el9.x86_64 24/180 2024-03-12T17:05:15.902 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rgw-2:18.2.2-797.g49e4a679.el9.x86_64 25/180 2024-03-12T17:05:15.916 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : rbd-fuse-2:18.2.2-797.g49e4a679.el9.x86_64 26/180 2024-03-12T17:05:15.916 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : rbd-mirror-2:18.2.2-797.g49e4a679.el9.x86_64 27/180 2024-03-12T17:05:15.917 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : rbd-nbd-2:18.2.2-797.g49e4a679.el9.x86_64 28/180 2024-03-12T17:05:15.917 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-797.g49e4a679.e 29/180 2024-03-12T17:05:15.917 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-797.g49e4a679.el9.noar 30/180 2024-03-12T17:05:15.917 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-797.g49e4a679.el9.no 31/180 2024-03-12T17:05:15.917 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-797.g49e4 32/180 2024-03-12T17:05:15.917 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-797.g49e4a679.el9 33/180 2024-03-12T17:05:15.917 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-rook-2:18.2.2-797.g49e4a679.el9.noarch 34/180 2024-03-12T17:05:15.917 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-797.g49e4a679.el 35/180 2024-03-12T17:05:15.917 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-volume-2:18.2.2-797.g49e4a679.el9.noarch 36/180 2024-03-12T17:05:15.917 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cephadm-2:18.2.2-797.g49e4a679.el9.noarch 37/180 2024-03-12T17:05:15.917 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cephfs-top-2:18.2.2-797.g49e4a679.el9.noarch 38/180 2024-03-12T17:05:15.917 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/180 2024-03-12T17:05:15.917 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/180 2024-03-12T17:05:15.918 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/180 2024-03-12T17:05:15.918 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : flex-2.6.4-9.el9.x86_64 42/180 2024-03-12T17:05:15.918 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 43/180 2024-03-12T17:05:15.918 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 44/180 2024-03-12T17:05:15.918 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 45/180 2024-03-12T17:05:15.918 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 46/180 2024-03-12T17:05:15.918 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 47/180 2024-03-12T17:05:15.918 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 48/180 2024-03-12T17:05:15.918 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 49/180 2024-03-12T17:05:15.918 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 50/180 2024-03-12T17:05:15.918 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 51/180 2024-03-12T17:05:15.918 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 52/180 2024-03-12T17:05:15.919 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 53/180 2024-03-12T17:05:15.919 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 54/180 2024-03-12T17:05:15.919 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 55/180 2024-03-12T17:05:15.919 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-13.11-1.el9.x86_64 56/180 2024-03-12T17:05:15.919 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 57/180 2024-03-12T17:05:15.919 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 58/180 2024-03-12T17:05:15.919 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 59/180 2024-03-12T17:05:15.919 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/180 2024-03-12T17:05:15.919 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/180 2024-03-12T17:05:15.919 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/180 2024-03-12T17:05:15.919 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/180 2024-03-12T17:05:15.919 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/180 2024-03-12T17:05:15.919 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/180 2024-03-12T17:05:15.919 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/180 2024-03-12T17:05:15.920 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/180 2024-03-12T17:05:15.920 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/180 2024-03-12T17:05:15.920 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/180 2024-03-12T17:05:15.920 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 70/180 2024-03-12T17:05:15.920 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 71/180 2024-03-12T17:05:15.920 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 72/180 2024-03-12T17:05:15.920 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 73/180 2024-03-12T17:05:15.920 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 74/180 2024-03-12T17:05:15.920 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 75/180 2024-03-12T17:05:15.920 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 76/180 2024-03-12T17:05:15.920 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 77/180 2024-03-12T17:05:15.920 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 78/180 2024-03-12T17:05:15.920 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 79/180 2024-03-12T17:05:15.921 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 80/180 2024-03-12T17:05:15.921 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 81/180 2024-03-12T17:05:15.921 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 82/180 2024-03-12T17:05:15.921 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-ply-3.11-14.el9.noarch 83/180 2024-03-12T17:05:15.921 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 84/180 2024-03-12T17:05:15.921 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 85/180 2024-03-12T17:05:15.921 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : jq-1.6-16.el9.x86_64 86/180 2024-03-12T17:05:15.921 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 87/180 2024-03-12T17:05:15.921 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 88/180 2024-03-12T17:05:15.921 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 89/180 2024-03-12T17:05:15.921 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 90/180 2024-03-12T17:05:15.921 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 91/180 2024-03-12T17:05:15.922 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 92/180 2024-03-12T17:05:15.922 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 93/180 2024-03-12T17:05:15.922 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 94/180 2024-03-12T17:05:15.922 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 95/180 2024-03-12T17:05:15.922 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 96/180 2024-03-12T17:05:15.922 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 97/180 2024-03-12T17:05:15.922 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 98/180 2024-03-12T17:05:15.922 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 99/180 2024-03-12T17:05:15.922 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 100/180 2024-03-12T17:05:15.922 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 101/180 2024-03-12T17:05:15.922 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 102/180 2024-03-12T17:05:15.922 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 103/180 2024-03-12T17:05:15.922 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 104/180 2024-03-12T17:05:15.923 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 105/180 2024-03-12T17:05:15.923 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 106/180 2024-03-12T17:05:15.923 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 107/180 2024-03-12T17:05:15.923 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 108/180 2024-03-12T17:05:15.923 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 109/180 2024-03-12T17:05:15.923 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 110/180 2024-03-12T17:05:15.923 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 111/180 2024-03-12T17:05:15.923 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 112/180 2024-03-12T17:05:15.923 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 113/180 2024-03-12T17:05:15.923 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 114/180 2024-03-12T17:05:15.923 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/180 2024-03-12T17:05:15.923 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/180 2024-03-12T17:05:15.923 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/180 2024-03-12T17:05:15.923 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/180 2024-03-12T17:05:15.924 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/180 2024-03-12T17:05:15.924 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/180 2024-03-12T17:05:15.924 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/180 2024-03-12T17:05:15.924 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/180 2024-03-12T17:05:15.924 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/180 2024-03-12T17:05:15.924 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/180 2024-03-12T17:05:15.924 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/180 2024-03-12T17:05:15.924 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/180 2024-03-12T17:05:15.924 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/180 2024-03-12T17:05:15.924 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/180 2024-03-12T17:05:15.924 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/180 2024-03-12T17:05:15.924 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/180 2024-03-12T17:05:15.924 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/180 2024-03-12T17:05:15.925 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/180 2024-03-12T17:05:15.925 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librados2-2:18.2.2-797.g49e4a679.el9.x86_64 133/180 2024-03-12T17:05:15.925 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/180 2024-03-12T17:05:15.925 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librbd1-2:18.2.2-797.g49e4a679.el9.x86_64 135/180 2024-03-12T17:05:15.925 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/180 2024-03-12T17:05:15.925 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 137/180 2024-03-12T17:05:15.925 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 138/180 2024-03-12T17:05:15.925 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 139/180 2024-03-12T17:05:15.925 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 140/180 2024-03-12T17:05:15.925 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 141/180 2024-03-12T17:05:15.925 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 142/180 2024-03-12T17:05:15.925 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 143/180 2024-03-12T17:05:15.925 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 144/180 2024-03-12T17:05:15.926 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 145/180 2024-03-12T17:05:15.926 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 146/180 2024-03-12T17:05:15.926 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : acl-2.3.1-4.el9.x86_64 147/180 2024-03-12T17:05:15.926 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : acl-2.3.1-3.el9.x86_64 148/180 2024-03-12T17:05:15.926 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 149/180 2024-03-12T17:05:15.926 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 150/180 2024-03-12T17:05:15.926 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 151/180 2024-03-12T17:05:15.926 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 152/180 2024-03-12T17:05:15.926 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 153/180 2024-03-12T17:05:15.926 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 154/180 2024-03-12T17:05:15.926 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 155/180 2024-03-12T17:05:15.926 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 156/180 2024-03-12T17:05:15.926 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 157/180 2024-03-12T17:05:15.927 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 158/180 2024-03-12T17:05:15.927 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 159/180 2024-03-12T17:05:15.927 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 160/180 2024-03-12T17:05:15.927 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 161/180 2024-03-12T17:05:15.927 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 162/180 2024-03-12T17:05:15.927 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 163/180 2024-03-12T17:05:15.927 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 164/180 2024-03-12T17:05:15.927 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 165/180 2024-03-12T17:05:15.927 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 166/180 2024-03-12T17:05:15.927 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 167/180 2024-03-12T17:05:15.927 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 168/180 2024-03-12T17:05:15.927 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 169/180 2024-03-12T17:05:15.927 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 170/180 2024-03-12T17:05:15.927 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 171/180 2024-03-12T17:05:15.928 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 172/180 2024-03-12T17:05:15.928 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 173/180 2024-03-12T17:05:15.928 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 174/180 2024-03-12T17:05:15.928 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 175/180 2024-03-12T17:05:15.928 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 176/180 2024-03-12T17:05:15.928 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 177/180 2024-03-12T17:05:15.928 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 178/180 2024-03-12T17:05:15.928 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 179/180 2024-03-12T17:05:18.393 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 180/180 2024-03-12T17:05:18.394 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:18.394 INFO:teuthology.orchestra.run.smithi052.stdout:Upgraded: 2024-03-12T17:05:18.394 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-12T17:05:18.394 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-12T17:05:18.394 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-12T17:05:18.394 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-12T17:05:18.394 INFO:teuthology.orchestra.run.smithi052.stdout: acl-2.3.1-4.el9.x86_64 2024-03-12T17:05:18.394 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-12T17:05:18.394 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-12T17:05:18.394 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-12T17:05:18.395 INFO:teuthology.orchestra.run.smithi052.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-12T17:05:18.395 INFO:teuthology.orchestra.run.smithi052.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-12T17:05:18.395 INFO:teuthology.orchestra.run.smithi052.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-12T17:05:18.395 INFO:teuthology.orchestra.run.smithi052.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-12T17:05:18.395 INFO:teuthology.orchestra.run.smithi052.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-12T17:05:18.395 INFO:teuthology.orchestra.run.smithi052.stdout: librados2-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:18.395 INFO:teuthology.orchestra.run.smithi052.stdout: librbd1-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:18.395 INFO:teuthology.orchestra.run.smithi052.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-12T17:05:18.395 INFO:teuthology.orchestra.run.smithi052.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-12T17:05:18.395 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-12T17:05:18.395 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-12T17:05:18.395 INFO:teuthology.orchestra.run.smithi052.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-12T17:05:18.395 INFO:teuthology.orchestra.run.smithi052.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-12T17:05:18.396 INFO:teuthology.orchestra.run.smithi052.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-12T17:05:18.396 INFO:teuthology.orchestra.run.smithi052.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-12T17:05:18.396 INFO:teuthology.orchestra.run.smithi052.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-12T17:05:18.396 INFO:teuthology.orchestra.run.smithi052.stdout:Installed: 2024-03-12T17:05:18.396 INFO:teuthology.orchestra.run.smithi052.stdout: automake-1.16.2-8.el9.noarch 2024-03-12T17:05:18.396 INFO:teuthology.orchestra.run.smithi052.stdout: bison-3.7.4-5.el9.x86_64 2024-03-12T17:05:18.396 INFO:teuthology.orchestra.run.smithi052.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-12T17:05:18.396 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:18.396 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-base-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:18.396 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-common-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:18.396 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-fuse-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:18.396 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-grafana-dashboards-2:18.2.2-797.g49e4a679.el9.noarch 2024-03-12T17:05:18.397 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-immutable-object-cache-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:18.397 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mds-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:18.397 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:18.397 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-cephadm-2:18.2.2-797.g49e4a679.el9.noarch 2024-03-12T17:05:18.397 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-dashboard-2:18.2.2-797.g49e4a679.el9.noarch 2024-03-12T17:05:18.397 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-diskprediction-local-2:18.2.2-797.g49e4a679.el9.noarch 2024-03-12T17:05:18.397 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-modules-core-2:18.2.2-797.g49e4a679.el9.noarch 2024-03-12T17:05:18.397 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-rook-2:18.2.2-797.g49e4a679.el9.noarch 2024-03-12T17:05:18.397 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mon-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:18.397 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-osd-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:18.397 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-prometheus-alerts-2:18.2.2-797.g49e4a679.el9.noarch 2024-03-12T17:05:18.397 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-radosgw-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:18.397 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-selinux-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:18.398 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-test-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:18.398 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-volume-2:18.2.2-797.g49e4a679.el9.noarch 2024-03-12T17:05:18.398 INFO:teuthology.orchestra.run.smithi052.stdout: cephadm-2:18.2.2-797.g49e4a679.el9.noarch 2024-03-12T17:05:18.398 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-mirror-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:18.398 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-top-2:18.2.2-797.g49e4a679.el9.noarch 2024-03-12T17:05:18.398 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-12T17:05:18.398 INFO:teuthology.orchestra.run.smithi052.stdout: flex-2.6.4-9.el9.x86_64 2024-03-12T17:05:18.398 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-12T17:05:18.398 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-12T17:05:18.398 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-12T17:05:18.398 INFO:teuthology.orchestra.run.smithi052.stdout: fmt-8.1.1-5.el9.x86_64 2024-03-12T17:05:18.398 INFO:teuthology.orchestra.run.smithi052.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-12T17:05:18.399 INFO:teuthology.orchestra.run.smithi052.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-12T17:05:18.399 INFO:teuthology.orchestra.run.smithi052.stdout: jq-1.6-16.el9.x86_64 2024-03-12T17:05:18.399 INFO:teuthology.orchestra.run.smithi052.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-12T17:05:18.399 INFO:teuthology.orchestra.run.smithi052.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-12T17:05:18.399 INFO:teuthology.orchestra.run.smithi052.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-12T17:05:18.399 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs-devel-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:18.399 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs2-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:18.399 INFO:teuthology.orchestra.run.smithi052.stdout: libcephsqlite-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:18.399 INFO:teuthology.orchestra.run.smithi052.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-12T17:05:18.399 INFO:teuthology.orchestra.run.smithi052.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-12T17:05:18.399 INFO:teuthology.orchestra.run.smithi052.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-12T17:05:18.399 INFO:teuthology.orchestra.run.smithi052.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-12T17:05:18.399 INFO:teuthology.orchestra.run.smithi052.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-12T17:05:18.400 INFO:teuthology.orchestra.run.smithi052.stdout: librados-devel-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:18.400 INFO:teuthology.orchestra.run.smithi052.stdout: libradosstriper1-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:18.400 INFO:teuthology.orchestra.run.smithi052.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-12T17:05:18.400 INFO:teuthology.orchestra.run.smithi052.stdout: librgw2-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:18.400 INFO:teuthology.orchestra.run.smithi052.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-12T17:05:18.400 INFO:teuthology.orchestra.run.smithi052.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-12T17:05:18.400 INFO:teuthology.orchestra.run.smithi052.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-12T17:05:18.400 INFO:teuthology.orchestra.run.smithi052.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-12T17:05:18.400 INFO:teuthology.orchestra.run.smithi052.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-12T17:05:18.400 INFO:teuthology.orchestra.run.smithi052.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-12T17:05:18.400 INFO:teuthology.orchestra.run.smithi052.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-12T17:05:18.400 INFO:teuthology.orchestra.run.smithi052.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-12T17:05:18.400 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-12T17:05:18.401 INFO:teuthology.orchestra.run.smithi052.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-12T17:05:18.401 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-13.11-1.el9.x86_64 2024-03-12T17:05:18.401 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-12T17:05:18.401 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-12T17:05:18.401 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-12T17:05:18.401 INFO:teuthology.orchestra.run.smithi052.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-12T17:05:18.401 INFO:teuthology.orchestra.run.smithi052.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-12T17:05:18.401 INFO:teuthology.orchestra.run.smithi052.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-12T17:05:18.401 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-12T17:05:18.401 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-argparse-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:18.401 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-common-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:18.401 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cephfs-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:18.401 INFO:teuthology.orchestra.run.smithi052.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-12T17:05:18.401 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-12T17:05:18.402 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-12T17:05:18.402 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-12T17:05:18.402 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-12T17:05:18.402 INFO:teuthology.orchestra.run.smithi052.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-12T17:05:18.402 INFO:teuthology.orchestra.run.smithi052.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-12T17:05:18.402 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-12T17:05:18.402 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-12T17:05:18.402 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-12T17:05:18.402 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-12T17:05:18.402 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-12T17:05:18.402 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-12T17:05:18.402 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-12T17:05:18.403 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-12T17:05:18.403 INFO:teuthology.orchestra.run.smithi052.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-12T17:05:18.403 INFO:teuthology.orchestra.run.smithi052.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-12T17:05:18.403 INFO:teuthology.orchestra.run.smithi052.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-12T17:05:18.403 INFO:teuthology.orchestra.run.smithi052.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-12T17:05:18.403 INFO:teuthology.orchestra.run.smithi052.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-12T17:05:18.403 INFO:teuthology.orchestra.run.smithi052.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-12T17:05:18.403 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-12T17:05:18.403 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-12T17:05:18.403 INFO:teuthology.orchestra.run.smithi052.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-12T17:05:18.403 INFO:teuthology.orchestra.run.smithi052.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-12T17:05:18.403 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-12T17:05:18.403 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ply-3.11-14.el9.noarch 2024-03-12T17:05:18.404 INFO:teuthology.orchestra.run.smithi052.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-12T17:05:18.404 INFO:teuthology.orchestra.run.smithi052.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-12T17:05:18.404 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-12T17:05:18.404 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-12T17:05:18.404 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-12T17:05:18.404 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-12T17:05:18.404 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-12T17:05:18.404 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rados-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:18.404 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rbd-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:18.404 INFO:teuthology.orchestra.run.smithi052.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-12T17:05:18.404 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-12T17:05:18.404 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-12T17:05:18.404 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rgw-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:18.404 INFO:teuthology.orchestra.run.smithi052.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-12T17:05:18.405 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-12T17:05:18.405 INFO:teuthology.orchestra.run.smithi052.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-12T17:05:18.405 INFO:teuthology.orchestra.run.smithi052.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-12T17:05:18.405 INFO:teuthology.orchestra.run.smithi052.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-12T17:05:18.405 INFO:teuthology.orchestra.run.smithi052.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-12T17:05:18.405 INFO:teuthology.orchestra.run.smithi052.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-12T17:05:18.405 INFO:teuthology.orchestra.run.smithi052.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-12T17:05:18.405 INFO:teuthology.orchestra.run.smithi052.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-12T17:05:18.405 INFO:teuthology.orchestra.run.smithi052.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-12T17:05:18.405 INFO:teuthology.orchestra.run.smithi052.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-12T17:05:18.405 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-fuse-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:18.405 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-mirror-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:18.405 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-nbd-2:18.2.2-797.g49e4a679.el9.x86_64 2024-03-12T17:05:18.406 INFO:teuthology.orchestra.run.smithi052.stdout: re2-1:20211101-3.el9.x86_64 2024-03-12T17:05:18.406 INFO:teuthology.orchestra.run.smithi052.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-12T17:05:18.406 INFO:teuthology.orchestra.run.smithi052.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-12T17:05:18.406 INFO:teuthology.orchestra.run.smithi052.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-12T17:05:18.406 INFO:teuthology.orchestra.run.smithi052.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-12T17:05:18.406 INFO:teuthology.orchestra.run.smithi052.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-12T17:05:18.406 INFO:teuthology.orchestra.run.smithi052.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-12T17:05:18.406 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:18.406 INFO:teuthology.orchestra.run.smithi052.stdout:Complete! 2024-03-12T17:05:19.048 DEBUG:teuthology.parallel:result is None 2024-03-12T17:05:19.049 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=49e4a6794d5502bd66908faf714fc54896c51712 2024-03-12T17:05:19.192 DEBUG:teuthology.orchestra.run.smithi052:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-12T17:05:19.249 INFO:teuthology.orchestra.run.smithi052.stdout:18.2.2-797.g49e4a679.el9 2024-03-12T17:05:19.250 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-797.g49e4a679.el9 2024-03-12T17:05:19.250 INFO:teuthology.task.install:The correct ceph version 18.2.2-797.g49e4a679 is installed. 2024-03-12T17:05:19.252 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=49e4a6794d5502bd66908faf714fc54896c51712 2024-03-12T17:05:19.564 DEBUG:teuthology.orchestra.run.smithi102:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-12T17:05:19.636 INFO:teuthology.orchestra.run.smithi102.stdout:18.2.2-797.g49e4a679.el9 2024-03-12T17:05:19.636 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-797.g49e4a679.el9 2024-03-12T17:05:19.636 INFO:teuthology.task.install:The correct ceph version 18.2.2-797.g49e4a679 is installed. 2024-03-12T17:05:19.638 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-12T17:05:19.638 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:05:19.639 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-12T17:05:19.716 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T17:05:19.716 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-12T17:05:19.818 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-12T17:05:19.819 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:05:19.819 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/usr/bin/daemon-helper 2024-03-12T17:05:19.900 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-12T17:05:20.013 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T17:05:20.013 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/daemon-helper 2024-03-12T17:05:20.113 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-12T17:05:20.252 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-12T17:05:20.252 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:05:20.252 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-12T17:05:20.336 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-12T17:05:20.454 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T17:05:20.454 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-12T17:05:20.546 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-12T17:05:20.683 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-12T17:05:20.684 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:05:20.684 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/usr/bin/stdin-killer 2024-03-12T17:05:20.775 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-12T17:05:20.850 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T17:05:20.850 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/stdin-killer 2024-03-12T17:05:20.951 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-12T17:05:21.042 INFO:teuthology.run_tasks:Running task ceph... 2024-03-12T17:05:21.114 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-12T17:05:21.114 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 777 /var/log/ceph 2024-03-12T17:05:21.122 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 777 /var/log/ceph 2024-03-12T17:05:21.208 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-12T17:05:21.208 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-12T17:05:21.213 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-12T17:05:21.299 INFO:tasks.ceph:Creating extra log directories... 2024-03-12T17:05:21.299 DEBUG:teuthology.orchestra.run.smithi052:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-12T17:05:21.304 DEBUG:teuthology.orchestra.run.smithi102:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-12T17:05:21.398 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-12T17:05:21.398 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-12T17:05:21.398 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-12T17:05:21.399 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:05:21.399 DEBUG:teuthology.orchestra.run.smithi052:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-12T17:05:21.440 DEBUG:teuthology.orchestra.run.smithi052:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-12T17:05:21.652 DEBUG:teuthology.orchestra.run.smithi052:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-12T17:05:21.727 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T17:05:21.727 DEBUG:teuthology.orchestra.run.smithi102:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-12T17:05:21.782 DEBUG:teuthology.orchestra.run.smithi102:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-12T17:05:22.031 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-12T17:05:22.133 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-12T17:05:22.133 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Reduced data availability', 'Degraded data redundancy'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-12T17:05:22.133 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2024-03-12_14:59:27-fs-wip-yuri11-testing-2024-03-11-0838-reef-distro-default-smithi/7593724', 'branch': 'wip-yuri11-testing-2024-03-11-0838-reef', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/acls}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7593724', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2024-03-12_14:59:27-fs-wip-yuri11-testing-2024-03-11-0838-reef-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-yuri11-testing-2024-03-11-0838-reef'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '49e4a6794d5502bd66908faf714fc54896c51712'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '49e4a6794d5502bd66908faf714fc54896c51712'}}, 'kclient': {'syntax': 'v2'}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri11-testing-2024-03-11-0838-reef', 'sha1': '49e4a6794d5502bd66908faf714fc54896c51712'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 71, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 9096, 'sha1': '49e4a6794d5502bd66908faf714fc54896c51712', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'fs', 'suite_branch': 'wip-yuri11-testing-2024-03-11-0838-reef', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_49e4a6794d5502bd66908faf714fc54896c51712/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '49e4a6794d5502bd66908faf714fc54896c51712', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi052.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNP9kBxoQIrVoXMCHmLIBeBNFhyJHWPOQVv9rSZGWE09a9SJHaMmnq62ps8uuT4MnufX+/MMAvqYSiGfz0/5UbM=', 'smithi102.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEiIR1hJQrQZGb6VRJkvAr1V6lA0j7iFz+k9F4XpdzY2y1WZkNwMFGYXVspkew3cAc+Ow+rLYDlJyPFQcugo9NA='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '49e4a6794d5502bd66908faf714fc54896c51712'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '49e4a6794d5502bd66908faf714fc54896c51712', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_acls']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-12_14:59:27', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-12T17:05:22.134 DEBUG:teuthology.orchestra.run.smithi052:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-12T17:05:22.139 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-12T17:05:22.195 DEBUG:teuthology.orchestra.run.smithi052:> sudo install -d -m0777 -- /var/run/ceph 2024-03-12T17:05:22.223 DEBUG:teuthology.orchestra.run.smithi102:> sudo install -d -m0777 -- /var/run/ceph 2024-03-12T17:05:22.321 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:05:22.321 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/scratch_devs of=/dev/stdout 2024-03-12T17:05:22.362 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-12T17:05:22.362 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_1 2024-03-12T17:05:22.440 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-12T17:05:22.440 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T17:05:22.440 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 640 Links: 1 2024-03-12T17:05:22.440 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T17:05:22.440 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2024-03-12T17:05:22.440 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-03-12 17:05:11.709216430 +0000 2024-03-12T17:05:22.440 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-03-12 17:02:17.310476651 +0000 2024-03-12T17:05:22.440 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-03-12 17:02:17.310476651 +0000 2024-03-12T17:05:22.440 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-03-12 17:02:17.310476651 +0000 2024-03-12T17:05:22.441 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-12T17:05:22.551 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-03-12T17:05:22.551 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-03-12T17:05:22.551 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.000234321 s, 2.2 MB/s 2024-03-12T17:05:22.554 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-12T17:05:22.595 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_2 2024-03-12T17:05:22.680 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-12T17:05:22.680 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T17:05:22.680 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 657 Links: 1 2024-03-12T17:05:22.680 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T17:05:22.680 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2024-03-12T17:05:22.680 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-03-12 17:05:11.710216417 +0000 2024-03-12T17:05:22.680 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-03-12 17:02:17.871469217 +0000 2024-03-12T17:05:22.680 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-03-12 17:02:17.871469217 +0000 2024-03-12T17:05:22.680 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-03-12 17:02:17.871469217 +0000 2024-03-12T17:05:22.681 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-12T17:05:22.799 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-03-12T17:05:22.799 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-03-12T17:05:22.799 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.000164854 s, 3.1 MB/s 2024-03-12T17:05:22.803 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-12T17:05:22.847 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_3 2024-03-12T17:05:22.929 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-12T17:05:22.929 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T17:05:22.929 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 678 Links: 1 2024-03-12T17:05:22.929 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T17:05:22.929 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2024-03-12T17:05:22.929 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-03-12 17:05:11.710216417 +0000 2024-03-12T17:05:22.930 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-03-12 17:02:18.456461464 +0000 2024-03-12T17:05:22.930 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-03-12 17:02:18.456461464 +0000 2024-03-12T17:05:22.930 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-03-12 17:02:18.456461464 +0000 2024-03-12T17:05:22.930 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-12T17:05:23.036 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-03-12T17:05:23.036 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-03-12T17:05:23.036 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.000172818 s, 3.0 MB/s 2024-03-12T17:05:23.039 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-12T17:05:23.080 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_4 2024-03-12T17:05:23.160 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-12T17:05:23.160 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T17:05:23.161 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 697 Links: 1 2024-03-12T17:05:23.161 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T17:05:23.161 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2024-03-12T17:05:23.161 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-03-12 17:05:11.711216404 +0000 2024-03-12T17:05:23.161 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-03-12 17:02:19.041453712 +0000 2024-03-12T17:05:23.161 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-03-12 17:02:19.041453712 +0000 2024-03-12T17:05:23.161 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-03-12 17:02:19.041453712 +0000 2024-03-12T17:05:23.161 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-12T17:05:23.265 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-03-12T17:05:23.265 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-03-12T17:05:23.265 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.000186746 s, 2.7 MB/s 2024-03-12T17:05:23.267 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-12T17:05:23.308 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-12T17:05:23.308 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T17:05:23.308 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/scratch_devs of=/dev/stdout 2024-03-12T17:05:23.366 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-12T17:05:23.367 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_1 2024-03-12T17:05:23.418 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-12T17:05:23.419 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T17:05:23.419 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 632 Links: 1 2024-03-12T17:05:23.419 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T17:05:23.419 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-03-12T17:05:23.419 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-03-12 17:04:56.681705154 +0000 2024-03-12T17:05:23.419 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-03-12 17:02:08.087577163 +0000 2024-03-12T17:05:23.419 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-03-12 17:02:08.087577163 +0000 2024-03-12T17:05:23.419 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-03-12 17:02:08.087577163 +0000 2024-03-12T17:05:23.419 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-12T17:05:23.502 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-03-12T17:05:23.502 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-03-12T17:05:23.502 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000377156 s, 1.4 MB/s 2024-03-12T17:05:23.506 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-12T17:05:23.559 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_2 2024-03-12T17:05:23.604 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-12T17:05:23.604 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T17:05:23.604 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 651 Links: 1 2024-03-12T17:05:23.604 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T17:05:23.604 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-03-12T17:05:23.604 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-03-12 17:04:56.682705134 +0000 2024-03-12T17:05:23.604 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-03-12 17:02:08.633566021 +0000 2024-03-12T17:05:23.604 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-03-12 17:02:08.633566021 +0000 2024-03-12T17:05:23.605 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-03-12 17:02:08.633566021 +0000 2024-03-12T17:05:23.605 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-12T17:05:23.739 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-03-12T17:05:23.739 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-03-12T17:05:23.739 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000340006 s, 1.5 MB/s 2024-03-12T17:05:23.744 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-12T17:05:23.798 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_3 2024-03-12T17:05:23.845 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-12T17:05:23.845 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T17:05:23.845 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 670 Links: 1 2024-03-12T17:05:23.845 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T17:05:23.845 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-03-12T17:05:23.845 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-03-12 17:04:56.683705113 +0000 2024-03-12T17:05:23.845 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-03-12 17:02:09.236553716 +0000 2024-03-12T17:05:23.845 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-03-12 17:02:09.236553716 +0000 2024-03-12T17:05:23.845 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-03-12 17:02:09.236553716 +0000 2024-03-12T17:05:23.846 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-12T17:05:23.976 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-03-12T17:05:23.976 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-03-12T17:05:23.976 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000293677 s, 1.7 MB/s 2024-03-12T17:05:23.980 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-12T17:05:24.034 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_4 2024-03-12T17:05:24.084 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-12T17:05:24.084 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T17:05:24.084 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 691 Links: 1 2024-03-12T17:05:24.084 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T17:05:24.084 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-03-12T17:05:24.084 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-03-12 17:04:56.683705113 +0000 2024-03-12T17:05:24.084 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-03-12 17:02:09.816541880 +0000 2024-03-12T17:05:24.084 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-03-12 17:02:09.816541880 +0000 2024-03-12T17:05:24.084 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-03-12 17:02:09.816541880 +0000 2024-03-12T17:05:24.085 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-12T17:05:24.170 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-03-12T17:05:24.170 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-03-12T17:05:24.170 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.00034706 s, 1.5 MB/s 2024-03-12T17:05:24.174 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-12T17:05:24.227 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-12T17:05:24.227 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi052.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi102.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-12T17:05:24.227 INFO:tasks.ceph:Generating config... 2024-03-12T17:05:24.229 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-12T17:05:24.229 INFO:tasks.ceph:[client] debug client = 20 2024-03-12T17:05:24.229 INFO:tasks.ceph:[client] debug ms = 1 2024-03-12T17:05:24.229 INFO:tasks.ceph:[client] pid file = 2024-03-12T17:05:24.229 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-12T17:05:24.229 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-12T17:05:24.229 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-03-12T17:05:24.229 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-12T17:05:24.229 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-12T17:05:24.229 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-12T17:05:24.230 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-12T17:05:24.230 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-12T17:05:24.230 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-12T17:05:24.230 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-12T17:05:24.230 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-12T17:05:24.230 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-12T17:05:24.230 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-12T17:05:24.230 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-12T17:05:24.230 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-12T17:05:24.230 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-12T17:05:24.230 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-12T17:05:24.230 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-12T17:05:24.230 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-12T17:05:24.230 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-12T17:05:24.231 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-12T17:05:24.231 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-03-12T17:05:24.231 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-03-12T17:05:24.231 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-12T17:05:24.231 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-12T17:05:24.231 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-03-12T17:05:24.231 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-12T17:05:24.231 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-12T17:05:24.231 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-12T17:05:24.231 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-12T17:05:24.231 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-12T17:05:24.231 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-12T17:05:24.231 INFO:tasks.ceph:Setting up mon.a... 2024-03-12T17:05:24.232 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-12T17:05:24.381 INFO:teuthology.orchestra.run.smithi052.stdout:creating /etc/ceph/ceph.keyring 2024-03-12T17:05:24.390 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-12T17:05:24.520 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-12T17:05:24.642 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.52'), ('mon.b', '172.21.15.102'), ('mon.c', '[v2:172.21.15.102:3301,v1:172.21.15.102:6790]')] 2024-03-12T17:05:24.643 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.52,172.21.15.102,[v2:172.21.15.102:3301,v1:172.21.15.102:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-12T17:05:24.644 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:05:24.644 DEBUG:teuthology.orchestra.run.smithi052:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-12T17:05:24.700 DEBUG:teuthology.orchestra.run.smithi052:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.52 --add b 172.21.15.102 --addv c '[v2:172.21.15.102:3301,v1:172.21.15.102:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-12T17:05:24.861 INFO:teuthology.orchestra.run.smithi052.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-12T17:05:24.861 INFO:teuthology.orchestra.run.smithi052.stdout:monmaptool: generated fsid 524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3 2024-03-12T17:05:24.861 INFO:teuthology.orchestra.run.smithi052.stdout:setting min_mon_release = pacific 2024-03-12T17:05:24.861 INFO:teuthology.orchestra.run.smithi052.stdout:epoch 0 2024-03-12T17:05:24.861 INFO:teuthology.orchestra.run.smithi052.stdout:fsid 524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3 2024-03-12T17:05:24.861 INFO:teuthology.orchestra.run.smithi052.stdout:last_changed 2024-03-12T17:05:24.868997+0000 2024-03-12T17:05:24.861 INFO:teuthology.orchestra.run.smithi052.stdout:created 2024-03-12T17:05:24.868997+0000 2024-03-12T17:05:24.861 INFO:teuthology.orchestra.run.smithi052.stdout:min_mon_release 16 (pacific) 2024-03-12T17:05:24.861 INFO:teuthology.orchestra.run.smithi052.stdout:election_strategy: 1 2024-03-12T17:05:24.862 INFO:teuthology.orchestra.run.smithi052.stdout:0: [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon.a 2024-03-12T17:05:24.862 INFO:teuthology.orchestra.run.smithi052.stdout:1: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.b 2024-03-12T17:05:24.862 INFO:teuthology.orchestra.run.smithi052.stdout:2: [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] mon.c 2024-03-12T17:05:24.862 INFO:teuthology.orchestra.run.smithi052.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-12T17:05:24.864 DEBUG:teuthology.orchestra.run.smithi052:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-12T17:05:24.918 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3... 2024-03-12T17:05:24.919 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-12T17:05:24.926 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-12T17:05:25.096 INFO:teuthology.orchestra.run.smithi102.stdout:[global] 2024-03-12T17:05:25.097 INFO:teuthology.orchestra.run.smithi102.stdout: chdir = "" 2024-03-12T17:05:25.097 INFO:teuthology.orchestra.run.smithi102.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-12T17:05:25.097 INFO:teuthology.orchestra.run.smithi102.stdout: auth supported = cephx 2024-03-12T17:05:25.097 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:05:25.097 INFO:teuthology.orchestra.run.smithi102.stdout: filestore xattr use omap = true 2024-03-12T17:05:25.097 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:05:25.097 INFO:teuthology.orchestra.run.smithi102.stdout: mon clock drift allowed = 1.000 2024-03-12T17:05:25.097 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:05:25.097 INFO:teuthology.orchestra.run.smithi102.stdout: osd crush chooseleaf type = 0 2024-03-12T17:05:25.097 INFO:teuthology.orchestra.run.smithi102.stdout: auth debug = true 2024-03-12T17:05:25.097 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:05:25.097 INFO:teuthology.orchestra.run.smithi102.stdout: ms die on old message = true 2024-03-12T17:05:25.097 INFO:teuthology.orchestra.run.smithi102.stdout: ms die on bug = true 2024-03-12T17:05:25.098 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:05:25.098 INFO:teuthology.orchestra.run.smithi102.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-12T17:05:25.098 INFO:teuthology.orchestra.run.smithi102.stdout: mon pg warn max object skew = 0 2024-03-12T17:05:25.098 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:05:25.098 INFO:teuthology.orchestra.run.smithi102.stdout: # disable pg_autoscaler by default for new pools 2024-03-12T17:05:25.098 INFO:teuthology.orchestra.run.smithi102.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-12T17:05:25.098 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:05:25.098 INFO:teuthology.orchestra.run.smithi102.stdout: osd pool default size = 2 2024-03-12T17:05:25.098 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:05:25.098 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd allow primary affinity = true 2024-03-12T17:05:25.098 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd allow pg remap = true 2024-03-12T17:05:25.098 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on legacy crush tunables = false 2024-03-12T17:05:25.098 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on crush straw calc version zero = false 2024-03-12T17:05:25.099 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on no sortbitwise = false 2024-03-12T17:05:25.099 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on osd down out interval zero = false 2024-03-12T17:05:25.099 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on too few osds = false 2024-03-12T17:05:25.099 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-12T17:05:25.099 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-12T17:05:25.099 INFO:teuthology.orchestra.run.smithi102.stdout: mon_allow_pool_size_one = true 2024-03-12T17:05:25.099 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:05:25.099 INFO:teuthology.orchestra.run.smithi102.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-12T17:05:25.099 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:05:25.099 INFO:teuthology.orchestra.run.smithi102.stdout: osd default data pool replay window = 5 2024-03-12T17:05:25.099 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:05:25.099 INFO:teuthology.orchestra.run.smithi102.stdout: mon allow pool delete = true 2024-03-12T17:05:25.099 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:05:25.099 INFO:teuthology.orchestra.run.smithi102.stdout: mon cluster log file level = debug 2024-03-12T17:05:25.099 INFO:teuthology.orchestra.run.smithi102.stdout: debug asserts on shutdown = true 2024-03-12T17:05:25.100 INFO:teuthology.orchestra.run.smithi102.stdout: mon health detail to clog = false 2024-03-12T17:05:25.100 INFO:teuthology.orchestra.run.smithi102.stdout: mon host = "172.21.15.52,172.21.15.102,[v2:172.21.15.102:3301,v1:172.21.15.102:6790]" 2024-03-12T17:05:25.100 INFO:teuthology.orchestra.run.smithi102.stdout: ms die on skipped message = False 2024-03-12T17:05:25.100 INFO:teuthology.orchestra.run.smithi102.stdout: fsid = 524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3 2024-03-12T17:05:25.100 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:05:25.100 INFO:teuthology.orchestra.run.smithi102.stdout:[osd] 2024-03-12T17:05:25.100 INFO:teuthology.orchestra.run.smithi102.stdout: osd journal size = 100 2024-03-12T17:05:25.100 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:05:25.100 INFO:teuthology.orchestra.run.smithi102.stdout: osd scrub load threshold = 5.0 2024-03-12T17:05:25.100 INFO:teuthology.orchestra.run.smithi102.stdout: osd scrub max interval = 600 2024-03-12T17:05:25.100 INFO:teuthology.orchestra.run.smithi102.stdout: osd mclock profile = high_recovery_ops 2024-03-12T17:05:25.100 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:05:25.100 INFO:teuthology.orchestra.run.smithi102.stdout: osd recover clone overlap = true 2024-03-12T17:05:25.101 INFO:teuthology.orchestra.run.smithi102.stdout: osd recovery max chunk = 1048576 2024-03-12T17:05:25.101 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:05:25.101 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug shutdown = true 2024-03-12T17:05:25.101 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug op order = true 2024-03-12T17:05:25.101 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug verify stray on activate = true 2024-03-12T17:05:25.101 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:05:25.101 INFO:teuthology.orchestra.run.smithi102.stdout: osd open classes on start = true 2024-03-12T17:05:25.101 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug pg log writeout = true 2024-03-12T17:05:25.101 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:05:25.101 INFO:teuthology.orchestra.run.smithi102.stdout: osd deep scrub update digest min age = 30 2024-03-12T17:05:25.101 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:05:25.101 INFO:teuthology.orchestra.run.smithi102.stdout: osd map max advance = 10 2024-03-12T17:05:25.101 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:05:25.101 INFO:teuthology.orchestra.run.smithi102.stdout: journal zero on create = true 2024-03-12T17:05:25.101 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:05:25.101 INFO:teuthology.orchestra.run.smithi102.stdout: filestore ondisk finisher threads = 3 2024-03-12T17:05:25.102 INFO:teuthology.orchestra.run.smithi102.stdout: filestore apply finisher threads = 3 2024-03-12T17:05:25.102 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:05:25.102 INFO:teuthology.orchestra.run.smithi102.stdout: bdev debug aio = true 2024-03-12T17:05:25.102 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug misdirected ops = true 2024-03-12T17:05:25.102 INFO:teuthology.orchestra.run.smithi102.stdout: bluestore block size = 96636764160 2024-03-12T17:05:25.102 INFO:teuthology.orchestra.run.smithi102.stdout: bluestore fsck on mount = True 2024-03-12T17:05:25.102 INFO:teuthology.orchestra.run.smithi102.stdout: debug bluefs = 20 2024-03-12T17:05:25.102 INFO:teuthology.orchestra.run.smithi102.stdout: debug bluestore = 20 2024-03-12T17:05:25.102 INFO:teuthology.orchestra.run.smithi102.stdout: debug ms = 1 2024-03-12T17:05:25.102 INFO:teuthology.orchestra.run.smithi102.stdout: debug osd = 20 2024-03-12T17:05:25.102 INFO:teuthology.orchestra.run.smithi102.stdout: debug rocksdb = 10 2024-03-12T17:05:25.102 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd backfillfull_ratio = 0.85 2024-03-12T17:05:25.102 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd full ratio = 0.9 2024-03-12T17:05:25.102 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd nearfull ratio = 0.8 2024-03-12T17:05:25.103 INFO:teuthology.orchestra.run.smithi102.stdout: osd failsafe full ratio = 0.95 2024-03-12T17:05:25.103 INFO:teuthology.orchestra.run.smithi102.stdout: osd objectstore = bluestore 2024-03-12T17:05:25.103 INFO:teuthology.orchestra.run.smithi102.stdout: osd op complaint time = 180 2024-03-12T17:05:25.103 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:05:25.103 INFO:teuthology.orchestra.run.smithi102.stdout:[mgr] 2024-03-12T17:05:25.103 INFO:teuthology.orchestra.run.smithi102.stdout: debug ms = 1 2024-03-12T17:05:25.103 INFO:teuthology.orchestra.run.smithi102.stdout: debug mgr = 20 2024-03-12T17:05:25.103 INFO:teuthology.orchestra.run.smithi102.stdout: debug mon = 20 2024-03-12T17:05:25.103 INFO:teuthology.orchestra.run.smithi102.stdout: debug auth = 20 2024-03-12T17:05:25.103 INFO:teuthology.orchestra.run.smithi102.stdout: mon reweight min pgs per osd = 4 2024-03-12T17:05:25.103 INFO:teuthology.orchestra.run.smithi102.stdout: mon reweight min bytes per osd = 10 2024-03-12T17:05:25.103 INFO:teuthology.orchestra.run.smithi102.stdout: mgr/telemetry/nag = false 2024-03-12T17:05:25.103 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:05:25.103 INFO:teuthology.orchestra.run.smithi102.stdout:[mon] 2024-03-12T17:05:25.103 INFO:teuthology.orchestra.run.smithi102.stdout: debug ms = 1 2024-03-12T17:05:25.104 INFO:teuthology.orchestra.run.smithi102.stdout: debug mon = 20 2024-03-12T17:05:25.104 INFO:teuthology.orchestra.run.smithi102.stdout: debug paxos = 20 2024-03-12T17:05:25.104 INFO:teuthology.orchestra.run.smithi102.stdout: debug auth = 20 2024-03-12T17:05:25.104 INFO:teuthology.orchestra.run.smithi102.stdout: mon data avail warn = 5 2024-03-12T17:05:25.104 INFO:teuthology.orchestra.run.smithi102.stdout: mon mgr mkfs grace = 240 2024-03-12T17:05:25.104 INFO:teuthology.orchestra.run.smithi102.stdout: mon reweight min pgs per osd = 4 2024-03-12T17:05:25.104 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd reporter subtree level = osd 2024-03-12T17:05:25.104 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd prime pg temp = true 2024-03-12T17:05:25.104 INFO:teuthology.orchestra.run.smithi102.stdout: mon reweight min bytes per osd = 10 2024-03-12T17:05:25.104 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:05:25.104 INFO:teuthology.orchestra.run.smithi102.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-12T17:05:25.104 INFO:teuthology.orchestra.run.smithi102.stdout: auth mon ticket ttl = 660 # 11m 2024-03-12T17:05:25.104 INFO:teuthology.orchestra.run.smithi102.stdout: auth service ticket ttl = 240 # 4m 2024-03-12T17:05:25.104 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:05:25.104 INFO:teuthology.orchestra.run.smithi102.stdout: # don't complain about insecure global_id in the test suite 2024-03-12T17:05:25.105 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-12T17:05:25.105 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-12T17:05:25.105 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:05:25.105 INFO:teuthology.orchestra.run.smithi102.stdout: # 1m isn't quite enough 2024-03-12T17:05:25.105 INFO:teuthology.orchestra.run.smithi102.stdout: mon_down_mkfs_grace = 2m 2024-03-12T17:05:25.105 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:05:25.105 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_filestore_osds = false 2024-03-12T17:05:25.105 INFO:teuthology.orchestra.run.smithi102.stdout: mon op complaint time = 120 2024-03-12T17:05:25.105 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-12T17:05:25.105 INFO:teuthology.orchestra.run.smithi102.stdout:[client] 2024-03-12T17:05:25.105 INFO:teuthology.orchestra.run.smithi102.stdout: rgw cache enabled = true 2024-03-12T17:05:25.105 INFO:teuthology.orchestra.run.smithi102.stdout: rgw enable ops log = true 2024-03-12T17:05:25.105 INFO:teuthology.orchestra.run.smithi102.stdout: rgw enable usage log = true 2024-03-12T17:05:25.106 INFO:teuthology.orchestra.run.smithi102.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-12T17:05:25.106 INFO:teuthology.orchestra.run.smithi102.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-12T17:05:25.106 INFO:teuthology.orchestra.run.smithi102.stdout: client mount timeout = 600 2024-03-12T17:05:25.106 INFO:teuthology.orchestra.run.smithi102.stdout: debug client = 20 2024-03-12T17:05:25.106 INFO:teuthology.orchestra.run.smithi102.stdout: debug ms = 1 2024-03-12T17:05:25.106 INFO:teuthology.orchestra.run.smithi102.stdout: pid file = "" 2024-03-12T17:05:25.106 INFO:teuthology.orchestra.run.smithi102.stdout: rados mon op timeout = 900 2024-03-12T17:05:25.106 INFO:teuthology.orchestra.run.smithi102.stdout: rados osd op timeout = 900 2024-03-12T17:05:25.106 INFO:teuthology.orchestra.run.smithi102.stdout:[mon.a] 2024-03-12T17:05:25.106 INFO:teuthology.orchestra.run.smithi102.stdout:[mon.b] 2024-03-12T17:05:25.106 INFO:teuthology.orchestra.run.smithi102.stdout:[mon.c] 2024-03-12T17:05:25.106 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.a] 2024-03-12T17:05:25.106 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.c] 2024-03-12T17:05:25.106 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.b] 2024-03-12T17:05:25.107 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.d] 2024-03-12T17:05:25.107 INFO:teuthology.orchestra.run.smithi102.stdout:[mds] 2024-03-12T17:05:25.107 INFO:teuthology.orchestra.run.smithi102.stdout: debug mds = 20 2024-03-12T17:05:25.107 INFO:teuthology.orchestra.run.smithi102.stdout: debug mds balancer = 20 2024-03-12T17:05:25.107 INFO:teuthology.orchestra.run.smithi102.stdout: debug ms = 1 2024-03-12T17:05:25.107 INFO:teuthology.orchestra.run.smithi102.stdout: mds debug frag = True 2024-03-12T17:05:25.107 INFO:teuthology.orchestra.run.smithi102.stdout: mds debug scatterstat = True 2024-03-12T17:05:25.107 INFO:teuthology.orchestra.run.smithi102.stdout: mds op complaint time = 180 2024-03-12T17:05:25.107 INFO:teuthology.orchestra.run.smithi102.stdout: mds verify scatter = True 2024-03-12T17:05:25.107 INFO:teuthology.orchestra.run.smithi102.stdout: osd op complaint time = 180 2024-03-12T17:05:25.107 INFO:teuthology.orchestra.run.smithi102.stdout: rados mon op timeout = 900 2024-03-12T17:05:25.107 INFO:teuthology.orchestra.run.smithi102.stdout: rados osd op timeout = 900 2024-03-12T17:05:25.108 INFO:teuthology.orchestra.run.smithi052.stdout:[global] 2024-03-12T17:05:25.108 INFO:teuthology.orchestra.run.smithi052.stdout: chdir = "" 2024-03-12T17:05:25.108 INFO:teuthology.orchestra.run.smithi052.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-12T17:05:25.108 INFO:teuthology.orchestra.run.smithi052.stdout: auth supported = cephx 2024-03-12T17:05:25.108 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:25.108 INFO:teuthology.orchestra.run.smithi052.stdout: filestore xattr use omap = true 2024-03-12T17:05:25.108 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:25.108 INFO:teuthology.orchestra.run.smithi052.stdout: mon clock drift allowed = 1.000 2024-03-12T17:05:25.108 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:25.109 INFO:teuthology.orchestra.run.smithi052.stdout: osd crush chooseleaf type = 0 2024-03-12T17:05:25.109 INFO:teuthology.orchestra.run.smithi052.stdout: auth debug = true 2024-03-12T17:05:25.109 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:25.109 INFO:teuthology.orchestra.run.smithi052.stdout: ms die on old message = true 2024-03-12T17:05:25.109 INFO:teuthology.orchestra.run.smithi052.stdout: ms die on bug = true 2024-03-12T17:05:25.109 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:25.109 INFO:teuthology.orchestra.run.smithi052.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-12T17:05:25.109 INFO:teuthology.orchestra.run.smithi052.stdout: mon pg warn max object skew = 0 2024-03-12T17:05:25.109 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:25.109 INFO:teuthology.orchestra.run.smithi052.stdout: # disable pg_autoscaler by default for new pools 2024-03-12T17:05:25.109 INFO:teuthology.orchestra.run.smithi052.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-12T17:05:25.109 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:25.109 INFO:teuthology.orchestra.run.smithi052.stdout: osd pool default size = 2 2024-03-12T17:05:25.109 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:25.109 INFO:teuthology.orchestra.run.smithi052.stdout: mon osd allow primary affinity = true 2024-03-12T17:05:25.110 INFO:teuthology.orchestra.run.smithi052.stdout: mon osd allow pg remap = true 2024-03-12T17:05:25.110 INFO:teuthology.orchestra.run.smithi052.stdout: mon warn on legacy crush tunables = false 2024-03-12T17:05:25.110 INFO:teuthology.orchestra.run.smithi052.stdout: mon warn on crush straw calc version zero = false 2024-03-12T17:05:25.110 INFO:teuthology.orchestra.run.smithi052.stdout: mon warn on no sortbitwise = false 2024-03-12T17:05:25.110 INFO:teuthology.orchestra.run.smithi052.stdout: mon warn on osd down out interval zero = false 2024-03-12T17:05:25.110 INFO:teuthology.orchestra.run.smithi052.stdout: mon warn on too few osds = false 2024-03-12T17:05:25.110 INFO:teuthology.orchestra.run.smithi052.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-12T17:05:25.110 INFO:teuthology.orchestra.run.smithi052.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-12T17:05:25.110 INFO:teuthology.orchestra.run.smithi052.stdout: mon_allow_pool_size_one = true 2024-03-12T17:05:25.110 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:25.110 INFO:teuthology.orchestra.run.smithi052.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-12T17:05:25.110 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:25.110 INFO:teuthology.orchestra.run.smithi052.stdout: osd default data pool replay window = 5 2024-03-12T17:05:25.111 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:25.111 INFO:teuthology.orchestra.run.smithi052.stdout: mon allow pool delete = true 2024-03-12T17:05:25.111 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:25.111 INFO:teuthology.orchestra.run.smithi052.stdout: mon cluster log file level = debug 2024-03-12T17:05:25.111 INFO:teuthology.orchestra.run.smithi052.stdout: debug asserts on shutdown = true 2024-03-12T17:05:25.111 INFO:teuthology.orchestra.run.smithi052.stdout: mon health detail to clog = false 2024-03-12T17:05:25.111 INFO:teuthology.orchestra.run.smithi052.stdout: mon host = "172.21.15.52,172.21.15.102,[v2:172.21.15.102:3301,v1:172.21.15.102:6790]" 2024-03-12T17:05:25.111 INFO:teuthology.orchestra.run.smithi052.stdout: ms die on skipped message = False 2024-03-12T17:05:25.111 INFO:teuthology.orchestra.run.smithi052.stdout: fsid = 524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3 2024-03-12T17:05:25.111 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:25.112 INFO:teuthology.orchestra.run.smithi052.stdout:[osd] 2024-03-12T17:05:25.112 INFO:teuthology.orchestra.run.smithi052.stdout: osd journal size = 100 2024-03-12T17:05:25.112 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:25.112 INFO:teuthology.orchestra.run.smithi052.stdout: osd scrub load threshold = 5.0 2024-03-12T17:05:25.112 INFO:teuthology.orchestra.run.smithi052.stdout: osd scrub max interval = 600 2024-03-12T17:05:25.112 INFO:teuthology.orchestra.run.smithi052.stdout: osd mclock profile = high_recovery_ops 2024-03-12T17:05:25.112 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:25.112 INFO:teuthology.orchestra.run.smithi052.stdout: osd recover clone overlap = true 2024-03-12T17:05:25.112 INFO:teuthology.orchestra.run.smithi052.stdout: osd recovery max chunk = 1048576 2024-03-12T17:05:25.112 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:25.112 INFO:teuthology.orchestra.run.smithi052.stdout: osd debug shutdown = true 2024-03-12T17:05:25.112 INFO:teuthology.orchestra.run.smithi052.stdout: osd debug op order = true 2024-03-12T17:05:25.112 INFO:teuthology.orchestra.run.smithi052.stdout: osd debug verify stray on activate = true 2024-03-12T17:05:25.112 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:25.113 INFO:teuthology.orchestra.run.smithi052.stdout: osd open classes on start = true 2024-03-12T17:05:25.113 INFO:teuthology.orchestra.run.smithi052.stdout: osd debug pg log writeout = true 2024-03-12T17:05:25.113 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:25.113 INFO:teuthology.orchestra.run.smithi052.stdout: osd deep scrub update digest min age = 30 2024-03-12T17:05:25.113 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:25.113 INFO:teuthology.orchestra.run.smithi052.stdout: osd map max advance = 10 2024-03-12T17:05:25.113 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:25.113 INFO:teuthology.orchestra.run.smithi052.stdout: journal zero on create = true 2024-03-12T17:05:25.113 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:25.113 INFO:teuthology.orchestra.run.smithi052.stdout: filestore ondisk finisher threads = 3 2024-03-12T17:05:25.113 INFO:teuthology.orchestra.run.smithi052.stdout: filestore apply finisher threads = 3 2024-03-12T17:05:25.113 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:25.113 INFO:teuthology.orchestra.run.smithi052.stdout: bdev debug aio = true 2024-03-12T17:05:25.113 INFO:teuthology.orchestra.run.smithi052.stdout: osd debug misdirected ops = true 2024-03-12T17:05:25.114 INFO:teuthology.orchestra.run.smithi052.stdout: bluestore block size = 96636764160 2024-03-12T17:05:25.114 INFO:teuthology.orchestra.run.smithi052.stdout: bluestore fsck on mount = True 2024-03-12T17:05:25.114 INFO:teuthology.orchestra.run.smithi052.stdout: debug bluefs = 20 2024-03-12T17:05:25.114 INFO:teuthology.orchestra.run.smithi052.stdout: debug bluestore = 20 2024-03-12T17:05:25.114 INFO:teuthology.orchestra.run.smithi052.stdout: debug ms = 1 2024-03-12T17:05:25.114 INFO:teuthology.orchestra.run.smithi052.stdout: debug osd = 20 2024-03-12T17:05:25.114 INFO:teuthology.orchestra.run.smithi052.stdout: debug rocksdb = 10 2024-03-12T17:05:25.114 INFO:teuthology.orchestra.run.smithi052.stdout: mon osd backfillfull_ratio = 0.85 2024-03-12T17:05:25.114 INFO:teuthology.orchestra.run.smithi052.stdout: mon osd full ratio = 0.9 2024-03-12T17:05:25.114 INFO:teuthology.orchestra.run.smithi052.stdout: mon osd nearfull ratio = 0.8 2024-03-12T17:05:25.114 INFO:teuthology.orchestra.run.smithi052.stdout: osd failsafe full ratio = 0.95 2024-03-12T17:05:25.114 INFO:teuthology.orchestra.run.smithi052.stdout: osd objectstore = bluestore 2024-03-12T17:05:25.114 INFO:teuthology.orchestra.run.smithi052.stdout: osd op complaint time = 180 2024-03-12T17:05:25.114 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:25.115 INFO:teuthology.orchestra.run.smithi052.stdout:[mgr] 2024-03-12T17:05:25.115 INFO:teuthology.orchestra.run.smithi052.stdout: debug ms = 1 2024-03-12T17:05:25.115 INFO:teuthology.orchestra.run.smithi052.stdout: debug mgr = 20 2024-03-12T17:05:25.115 INFO:teuthology.orchestra.run.smithi052.stdout: debug mon = 20 2024-03-12T17:05:25.115 INFO:teuthology.orchestra.run.smithi052.stdout: debug auth = 20 2024-03-12T17:05:25.115 INFO:teuthology.orchestra.run.smithi052.stdout: mon reweight min pgs per osd = 4 2024-03-12T17:05:25.115 INFO:teuthology.orchestra.run.smithi052.stdout: mon reweight min bytes per osd = 10 2024-03-12T17:05:25.115 INFO:teuthology.orchestra.run.smithi052.stdout: mgr/telemetry/nag = false 2024-03-12T17:05:25.115 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:25.115 INFO:teuthology.orchestra.run.smithi052.stdout:[mon] 2024-03-12T17:05:25.115 INFO:teuthology.orchestra.run.smithi052.stdout: debug ms = 1 2024-03-12T17:05:25.115 INFO:teuthology.orchestra.run.smithi052.stdout: debug mon = 20 2024-03-12T17:05:25.115 INFO:teuthology.orchestra.run.smithi052.stdout: debug paxos = 20 2024-03-12T17:05:25.115 INFO:teuthology.orchestra.run.smithi052.stdout: debug auth = 20 2024-03-12T17:05:25.116 INFO:teuthology.orchestra.run.smithi052.stdout: mon data avail warn = 5 2024-03-12T17:05:25.116 INFO:teuthology.orchestra.run.smithi052.stdout: mon mgr mkfs grace = 240 2024-03-12T17:05:25.116 INFO:teuthology.orchestra.run.smithi052.stdout: mon reweight min pgs per osd = 4 2024-03-12T17:05:25.116 INFO:teuthology.orchestra.run.smithi052.stdout: mon osd reporter subtree level = osd 2024-03-12T17:05:25.116 INFO:teuthology.orchestra.run.smithi052.stdout: mon osd prime pg temp = true 2024-03-12T17:05:25.116 INFO:teuthology.orchestra.run.smithi052.stdout: mon reweight min bytes per osd = 10 2024-03-12T17:05:25.116 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:25.116 INFO:teuthology.orchestra.run.smithi052.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-12T17:05:25.116 INFO:teuthology.orchestra.run.smithi052.stdout: auth mon ticket ttl = 660 # 11m 2024-03-12T17:05:25.116 INFO:teuthology.orchestra.run.smithi052.stdout: auth service ticket ttl = 240 # 4m 2024-03-12T17:05:25.116 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:25.116 INFO:teuthology.orchestra.run.smithi052.stdout: # don't complain about insecure global_id in the test suite 2024-03-12T17:05:25.116 INFO:teuthology.orchestra.run.smithi052.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-12T17:05:25.116 INFO:teuthology.orchestra.run.smithi052.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-12T17:05:25.116 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:25.117 INFO:teuthology.orchestra.run.smithi052.stdout: # 1m isn't quite enough 2024-03-12T17:05:25.117 INFO:teuthology.orchestra.run.smithi052.stdout: mon_down_mkfs_grace = 2m 2024-03-12T17:05:25.117 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:25.117 INFO:teuthology.orchestra.run.smithi052.stdout: mon_warn_on_filestore_osds = false 2024-03-12T17:05:25.117 INFO:teuthology.orchestra.run.smithi052.stdout: mon op complaint time = 120 2024-03-12T17:05:25.117 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:05:25.117 INFO:teuthology.orchestra.run.smithi052.stdout:[client] 2024-03-12T17:05:25.117 INFO:teuthology.orchestra.run.smithi052.stdout: rgw cache enabled = true 2024-03-12T17:05:25.117 INFO:teuthology.orchestra.run.smithi052.stdout: rgw enable ops log = true 2024-03-12T17:05:25.117 INFO:teuthology.orchestra.run.smithi052.stdout: rgw enable usage log = true 2024-03-12T17:05:25.117 INFO:teuthology.orchestra.run.smithi052.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-12T17:05:25.117 INFO:teuthology.orchestra.run.smithi052.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-12T17:05:25.117 INFO:teuthology.orchestra.run.smithi052.stdout: client mount timeout = 600 2024-03-12T17:05:25.117 INFO:teuthology.orchestra.run.smithi052.stdout: debug client = 20 2024-03-12T17:05:25.117 INFO:teuthology.orchestra.run.smithi052.stdout: debug ms = 1 2024-03-12T17:05:25.117 INFO:teuthology.orchestra.run.smithi052.stdout: pid file = "" 2024-03-12T17:05:25.118 INFO:teuthology.orchestra.run.smithi052.stdout: rados mon op timeout = 900 2024-03-12T17:05:25.118 INFO:teuthology.orchestra.run.smithi052.stdout: rados osd op timeout = 900 2024-03-12T17:05:25.118 INFO:teuthology.orchestra.run.smithi052.stdout:[mon.a] 2024-03-12T17:05:25.118 INFO:teuthology.orchestra.run.smithi052.stdout:[mon.b] 2024-03-12T17:05:25.118 INFO:teuthology.orchestra.run.smithi052.stdout:[mon.c] 2024-03-12T17:05:25.118 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.a] 2024-03-12T17:05:25.118 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.c] 2024-03-12T17:05:25.118 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.b] 2024-03-12T17:05:25.118 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.d] 2024-03-12T17:05:25.118 INFO:teuthology.orchestra.run.smithi052.stdout:[mds] 2024-03-12T17:05:25.118 INFO:teuthology.orchestra.run.smithi052.stdout: debug mds = 20 2024-03-12T17:05:25.118 INFO:teuthology.orchestra.run.smithi052.stdout: debug mds balancer = 20 2024-03-12T17:05:25.118 INFO:teuthology.orchestra.run.smithi052.stdout: debug ms = 1 2024-03-12T17:05:25.118 INFO:teuthology.orchestra.run.smithi052.stdout: mds debug frag = True 2024-03-12T17:05:25.118 INFO:teuthology.orchestra.run.smithi052.stdout: mds debug scatterstat = True 2024-03-12T17:05:25.119 INFO:teuthology.orchestra.run.smithi052.stdout: mds op complaint time = 180 2024-03-12T17:05:25.119 INFO:teuthology.orchestra.run.smithi052.stdout: mds verify scatter = True 2024-03-12T17:05:25.119 INFO:teuthology.orchestra.run.smithi052.stdout: osd op complaint time = 180 2024-03-12T17:05:25.119 INFO:teuthology.orchestra.run.smithi052.stdout: rados mon op timeout = 900 2024-03-12T17:05:25.119 INFO:teuthology.orchestra.run.smithi052.stdout: rados osd op timeout = 900 2024-03-12T17:05:25.148 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-12T17:05:25.149 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-12T17:05:25.302 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-12T17:05:25.302 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:05:25.302 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-12T17:05:25.355 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:05:25.355 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-12T17:05:25.410 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi052.front.sepia.ceph.com 2024-03-12T17:05:25.410 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:05:25.410 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-12T17:05:25.410 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-12T17:05:25.565 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:05:25.565 DEBUG:teuthology.orchestra.run.smithi052:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-12T17:05:25.662 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi102.front.sepia.ceph.com 2024-03-12T17:05:25.662 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T17:05:25.662 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-12T17:05:25.662 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-12T17:05:25.812 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T17:05:25.882 DEBUG:teuthology.orchestra.run.smithi102:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-12T17:05:25.937 INFO:tasks.ceph:Setting up mon nodes... 2024-03-12T17:05:25.937 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-12T17:05:25.937 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-12T17:05:26.121 INFO:teuthology.orchestra.run.smithi052.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-12T17:05:26.132 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-12T17:05:26.348 INFO:teuthology.orchestra.run.smithi102.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-12T17:05:26.357 INFO:tasks.ceph:Setting up mds nodes... 2024-03-12T17:05:26.357 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-12T17:05:26.558 INFO:teuthology.orchestra.run.smithi052.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-12T17:05:26.567 DEBUG:teuthology.orchestra.run.smithi052:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-12T17:05:26.672 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-03-12T17:05:26.866 INFO:teuthology.orchestra.run.smithi052.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-03-12T17:05:26.875 DEBUG:teuthology.orchestra.run.smithi052:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-03-12T17:05:26.972 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-12T17:05:27.142 INFO:teuthology.orchestra.run.smithi102.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-12T17:05:27.148 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-12T17:05:27.229 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-03-12T17:05:27.389 INFO:teuthology.orchestra.run.smithi102.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-03-12T17:05:27.395 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-03-12T17:05:27.473 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-12T17:05:27.473 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-12T17:05:27.611 INFO:teuthology.orchestra.run.smithi052.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-12T17:05:27.678 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-03-12T17:05:27.813 INFO:teuthology.orchestra.run.smithi052.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-03-12T17:05:27.866 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-12T17:05:27.989 INFO:teuthology.orchestra.run.smithi102.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-12T17:05:28.042 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-03-12T17:05:28.165 INFO:teuthology.orchestra.run.smithi102.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-03-12T17:05:28.220 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-12T17:05:28.220 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi052.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi102.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-12T17:05:28.220 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-12T17:05:28.328 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-12T17:05:28.329 INFO:tasks.ceph:role: osd.0 2024-03-12T17:05:28.329 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi052.front.sepia.ceph.com 2024-03-12T17:05:28.329 DEBUG:teuthology.orchestra.run.smithi052:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-12T17:05:28.428 INFO:teuthology.orchestra.run.smithi052.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-12T17:05:28.428 INFO:teuthology.orchestra.run.smithi052.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-12T17:05:28.428 INFO:teuthology.orchestra.run.smithi052.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-12T17:05:28.428 INFO:teuthology.orchestra.run.smithi052.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-12T17:05:28.428 INFO:teuthology.orchestra.run.smithi052.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-12T17:05:28.428 INFO:teuthology.orchestra.run.smithi052.stdout: = sunit=0 swidth=0 blks 2024-03-12T17:05:28.428 INFO:teuthology.orchestra.run.smithi052.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-12T17:05:28.428 INFO:teuthology.orchestra.run.smithi052.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-12T17:05:28.428 INFO:teuthology.orchestra.run.smithi052.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-12T17:05:28.428 INFO:teuthology.orchestra.run.smithi052.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-12T17:05:28.464 INFO:teuthology.orchestra.run.smithi052.stdout:Discarding blocks...Done. 2024-03-12T17:05:28.469 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi052.front.sepia.ceph.com -o noatime 2024-03-12T17:05:28.469 DEBUG:teuthology.orchestra.run.smithi052:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-12T17:05:28.598 DEBUG:teuthology.orchestra.run.smithi052:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-12T17:05:28.686 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-12T17:05:28.780 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-12T17:05:28.780 INFO:tasks.ceph:role: osd.1 2024-03-12T17:05:28.780 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi052.front.sepia.ceph.com 2024-03-12T17:05:28.780 DEBUG:teuthology.orchestra.run.smithi052:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-12T17:05:28.872 INFO:teuthology.orchestra.run.smithi052.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-12T17:05:28.873 INFO:teuthology.orchestra.run.smithi052.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-12T17:05:28.873 INFO:teuthology.orchestra.run.smithi052.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-12T17:05:28.873 INFO:teuthology.orchestra.run.smithi052.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-12T17:05:28.873 INFO:teuthology.orchestra.run.smithi052.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-12T17:05:28.873 INFO:teuthology.orchestra.run.smithi052.stdout: = sunit=0 swidth=0 blks 2024-03-12T17:05:28.873 INFO:teuthology.orchestra.run.smithi052.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-12T17:05:28.873 INFO:teuthology.orchestra.run.smithi052.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-12T17:05:28.873 INFO:teuthology.orchestra.run.smithi052.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-12T17:05:28.873 INFO:teuthology.orchestra.run.smithi052.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-12T17:05:28.906 INFO:teuthology.orchestra.run.smithi052.stdout:Discarding blocks...Done. 2024-03-12T17:05:28.912 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi052.front.sepia.ceph.com -o noatime 2024-03-12T17:05:28.912 DEBUG:teuthology.orchestra.run.smithi052:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-12T17:05:29.025 DEBUG:teuthology.orchestra.run.smithi052:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-12T17:05:29.116 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-12T17:05:29.212 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-12T17:05:29.212 INFO:tasks.ceph:role: osd.2 2024-03-12T17:05:29.212 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi052.front.sepia.ceph.com 2024-03-12T17:05:29.212 DEBUG:teuthology.orchestra.run.smithi052:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-12T17:05:29.309 INFO:teuthology.orchestra.run.smithi052.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-12T17:05:29.309 INFO:teuthology.orchestra.run.smithi052.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-12T17:05:29.309 INFO:teuthology.orchestra.run.smithi052.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-12T17:05:29.309 INFO:teuthology.orchestra.run.smithi052.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-12T17:05:29.309 INFO:teuthology.orchestra.run.smithi052.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-12T17:05:29.309 INFO:teuthology.orchestra.run.smithi052.stdout: = sunit=0 swidth=0 blks 2024-03-12T17:05:29.309 INFO:teuthology.orchestra.run.smithi052.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-12T17:05:29.309 INFO:teuthology.orchestra.run.smithi052.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-12T17:05:29.309 INFO:teuthology.orchestra.run.smithi052.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-12T17:05:29.309 INFO:teuthology.orchestra.run.smithi052.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-12T17:05:29.342 INFO:teuthology.orchestra.run.smithi052.stdout:Discarding blocks...Done. 2024-03-12T17:05:29.348 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi052.front.sepia.ceph.com -o noatime 2024-03-12T17:05:29.348 DEBUG:teuthology.orchestra.run.smithi052:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-12T17:05:29.465 DEBUG:teuthology.orchestra.run.smithi052:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-12T17:05:29.549 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-12T17:05:29.643 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-12T17:05:29.643 INFO:tasks.ceph:role: osd.3 2024-03-12T17:05:29.643 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi052.front.sepia.ceph.com 2024-03-12T17:05:29.643 DEBUG:teuthology.orchestra.run.smithi052:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-12T17:05:29.741 INFO:teuthology.orchestra.run.smithi052.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-12T17:05:29.741 INFO:teuthology.orchestra.run.smithi052.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-12T17:05:29.741 INFO:teuthology.orchestra.run.smithi052.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-12T17:05:29.742 INFO:teuthology.orchestra.run.smithi052.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-12T17:05:29.742 INFO:teuthology.orchestra.run.smithi052.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-12T17:05:29.742 INFO:teuthology.orchestra.run.smithi052.stdout: = sunit=0 swidth=0 blks 2024-03-12T17:05:29.742 INFO:teuthology.orchestra.run.smithi052.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-12T17:05:29.742 INFO:teuthology.orchestra.run.smithi052.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-12T17:05:29.742 INFO:teuthology.orchestra.run.smithi052.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-12T17:05:29.742 INFO:teuthology.orchestra.run.smithi052.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-12T17:05:29.775 INFO:teuthology.orchestra.run.smithi052.stdout:Discarding blocks...Done. 2024-03-12T17:05:29.781 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi052.front.sepia.ceph.com -o noatime 2024-03-12T17:05:29.782 DEBUG:teuthology.orchestra.run.smithi052:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-12T17:05:29.895 DEBUG:teuthology.orchestra.run.smithi052:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-12T17:05:29.980 DEBUG:teuthology.orchestra.run.smithi052:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-12T17:05:30.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:05:30.130+0000 7f18a5ec9500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-12T17:05:30.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:05:30.131+0000 7f18a5ec9500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-12T17:05:30.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:05:30.131+0000 7f18a5ec9500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-12T17:05:30.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:05:30.131+0000 7f18a5ec9500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-12T17:05:30.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:05:30.131+0000 7f18a5ec9500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-12T17:05:30.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:05:30.134+0000 7f18a5ec9500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-12T17:05:33.311 DEBUG:teuthology.orchestra.run.smithi052:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-12T17:05:33.411 DEBUG:teuthology.orchestra.run.smithi052:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-12T17:05:33.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:05:33.562+0000 7f11ad410500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-12T17:05:33.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:05:33.563+0000 7f11ad410500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-12T17:05:33.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:05:33.563+0000 7f11ad410500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-12T17:05:33.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:05:33.563+0000 7f11ad410500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-12T17:05:33.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:05:33.563+0000 7f11ad410500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-12T17:05:33.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:05:33.565+0000 7f11ad410500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-12T17:05:36.491 DEBUG:teuthology.orchestra.run.smithi052:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-12T17:05:36.595 DEBUG:teuthology.orchestra.run.smithi052:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-12T17:05:36.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:05:36.752+0000 7fec4080c500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-12T17:05:36.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:05:36.752+0000 7fec4080c500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-12T17:05:36.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:05:36.752+0000 7fec4080c500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-12T17:05:36.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:05:36.752+0000 7fec4080c500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-12T17:05:36.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:05:36.752+0000 7fec4080c500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-12T17:05:36.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:05:36.755+0000 7fec4080c500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-12T17:05:39.930 DEBUG:teuthology.orchestra.run.smithi052:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-12T17:05:40.030 DEBUG:teuthology.orchestra.run.smithi052:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-12T17:05:40.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:05:40.177+0000 7f4552411500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-12T17:05:40.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:05:40.178+0000 7f4552411500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-12T17:05:40.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:05:40.178+0000 7f4552411500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-12T17:05:40.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:05:40.178+0000 7f4552411500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-12T17:05:40.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:05:40.178+0000 7f4552411500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-12T17:05:40.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:05:40.180+0000 7f4552411500 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-12T17:05:43.105 DEBUG:teuthology.orchestra.run.smithi052:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-12T17:05:43.209 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-12T17:05:43.312 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-12T17:05:43.312 INFO:tasks.ceph:role: osd.4 2024-03-12T17:05:43.312 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi102.front.sepia.ceph.com 2024-03-12T17:05:43.312 DEBUG:teuthology.orchestra.run.smithi102:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-12T17:05:43.409 INFO:teuthology.orchestra.run.smithi102.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-12T17:05:43.409 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-12T17:05:43.410 INFO:teuthology.orchestra.run.smithi102.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-12T17:05:43.410 INFO:teuthology.orchestra.run.smithi102.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-12T17:05:43.410 INFO:teuthology.orchestra.run.smithi102.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-12T17:05:43.410 INFO:teuthology.orchestra.run.smithi102.stdout: = sunit=0 swidth=0 blks 2024-03-12T17:05:43.410 INFO:teuthology.orchestra.run.smithi102.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-12T17:05:43.410 INFO:teuthology.orchestra.run.smithi102.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-12T17:05:43.410 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-12T17:05:43.410 INFO:teuthology.orchestra.run.smithi102.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-12T17:05:43.914 INFO:teuthology.orchestra.run.smithi102.stdout:Discarding blocks...Done. 2024-03-12T17:05:43.921 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi102.front.sepia.ceph.com -o noatime 2024-03-12T17:05:43.922 DEBUG:teuthology.orchestra.run.smithi102:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-12T17:05:44.104 DEBUG:teuthology.orchestra.run.smithi102:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-12T17:05:44.209 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-12T17:05:44.301 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-12T17:05:44.301 INFO:tasks.ceph:role: osd.5 2024-03-12T17:05:44.302 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi102.front.sepia.ceph.com 2024-03-12T17:05:44.302 DEBUG:teuthology.orchestra.run.smithi102:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-12T17:05:44.399 INFO:teuthology.orchestra.run.smithi102.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-12T17:05:44.399 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-12T17:05:44.399 INFO:teuthology.orchestra.run.smithi102.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-12T17:05:44.399 INFO:teuthology.orchestra.run.smithi102.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-12T17:05:44.399 INFO:teuthology.orchestra.run.smithi102.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-12T17:05:44.400 INFO:teuthology.orchestra.run.smithi102.stdout: = sunit=0 swidth=0 blks 2024-03-12T17:05:44.400 INFO:teuthology.orchestra.run.smithi102.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-12T17:05:44.400 INFO:teuthology.orchestra.run.smithi102.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-12T17:05:44.400 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-12T17:05:44.400 INFO:teuthology.orchestra.run.smithi102.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-12T17:05:44.874 INFO:teuthology.orchestra.run.smithi102.stdout:Discarding blocks...Done. 2024-03-12T17:05:44.880 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi102.front.sepia.ceph.com -o noatime 2024-03-12T17:05:44.880 DEBUG:teuthology.orchestra.run.smithi102:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-12T17:05:45.012 DEBUG:teuthology.orchestra.run.smithi102:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-12T17:05:45.108 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-12T17:05:45.203 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-12T17:05:45.203 INFO:tasks.ceph:role: osd.6 2024-03-12T17:05:45.203 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi102.front.sepia.ceph.com 2024-03-12T17:05:45.203 DEBUG:teuthology.orchestra.run.smithi102:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-12T17:05:45.300 INFO:teuthology.orchestra.run.smithi102.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-12T17:05:45.300 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-12T17:05:45.300 INFO:teuthology.orchestra.run.smithi102.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-12T17:05:45.301 INFO:teuthology.orchestra.run.smithi102.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-12T17:05:45.301 INFO:teuthology.orchestra.run.smithi102.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-12T17:05:45.301 INFO:teuthology.orchestra.run.smithi102.stdout: = sunit=0 swidth=0 blks 2024-03-12T17:05:45.301 INFO:teuthology.orchestra.run.smithi102.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-12T17:05:45.301 INFO:teuthology.orchestra.run.smithi102.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-12T17:05:45.301 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-12T17:05:45.301 INFO:teuthology.orchestra.run.smithi102.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-12T17:05:45.789 INFO:teuthology.orchestra.run.smithi102.stdout:Discarding blocks...Done. 2024-03-12T17:05:45.795 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi102.front.sepia.ceph.com -o noatime 2024-03-12T17:05:45.796 DEBUG:teuthology.orchestra.run.smithi102:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-12T17:05:45.989 DEBUG:teuthology.orchestra.run.smithi102:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-12T17:05:46.086 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-12T17:05:46.176 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-12T17:05:46.176 INFO:tasks.ceph:role: osd.7 2024-03-12T17:05:46.176 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi102.front.sepia.ceph.com 2024-03-12T17:05:46.177 DEBUG:teuthology.orchestra.run.smithi102:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-12T17:05:46.275 INFO:teuthology.orchestra.run.smithi102.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-12T17:05:46.275 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-12T17:05:46.275 INFO:teuthology.orchestra.run.smithi102.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-12T17:05:46.275 INFO:teuthology.orchestra.run.smithi102.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-12T17:05:46.275 INFO:teuthology.orchestra.run.smithi102.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-12T17:05:46.275 INFO:teuthology.orchestra.run.smithi102.stdout: = sunit=0 swidth=0 blks 2024-03-12T17:05:46.276 INFO:teuthology.orchestra.run.smithi102.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-12T17:05:46.276 INFO:teuthology.orchestra.run.smithi102.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-12T17:05:46.276 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-12T17:05:46.276 INFO:teuthology.orchestra.run.smithi102.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-12T17:05:46.732 INFO:teuthology.orchestra.run.smithi102.stdout:Discarding blocks...Done. 2024-03-12T17:05:46.739 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi102.front.sepia.ceph.com -o noatime 2024-03-12T17:05:46.739 DEBUG:teuthology.orchestra.run.smithi102:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-12T17:05:46.884 DEBUG:teuthology.orchestra.run.smithi102:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-12T17:05:46.981 DEBUG:teuthology.orchestra.run.smithi102:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-12T17:05:47.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:05:47.167+0000 7ff916ada500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-12T17:05:47.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:05:47.168+0000 7ff916ada500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-12T17:05:47.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:05:47.168+0000 7ff916ada500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-12T17:05:47.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:05:47.168+0000 7ff916ada500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-12T17:05:47.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:05:47.168+0000 7ff916ada500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-12T17:05:47.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:05:47.172+0000 7ff916ada500 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-12T17:05:50.342 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-12T17:05:50.438 DEBUG:teuthology.orchestra.run.smithi102:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-12T17:05:50.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:05:50.576+0000 7f3d2ff6c500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-12T17:05:50.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:05:50.578+0000 7f3d2ff6c500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-12T17:05:50.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:05:50.578+0000 7f3d2ff6c500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-12T17:05:50.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:05:50.578+0000 7f3d2ff6c500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-12T17:05:50.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:05:50.578+0000 7f3d2ff6c500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-12T17:05:50.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:05:50.580+0000 7f3d2ff6c500 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-12T17:05:52.136 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-12T17:05:52.143 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-12T17:05:53.503 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-12T17:05:53.600 DEBUG:teuthology.orchestra.run.smithi102:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-12T17:05:53.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:05:53.739+0000 7f47cc211500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-12T17:05:53.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:05:53.740+0000 7f47cc211500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-12T17:05:53.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:05:53.740+0000 7f47cc211500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-12T17:05:53.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:05:53.740+0000 7f47cc211500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-12T17:05:53.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:05:53.740+0000 7f47cc211500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-12T17:05:53.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:05:53.742+0000 7f47cc211500 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-12T17:05:56.670 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-12T17:05:56.763 DEBUG:teuthology.orchestra.run.smithi102:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-12T17:05:56.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:05:56.900+0000 7fd32af72500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-12T17:05:56.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:05:56.901+0000 7fd32af72500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-12T17:05:56.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:05:56.901+0000 7fd32af72500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-12T17:05:56.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:05:56.901+0000 7fd32af72500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-12T17:05:56.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:05:56.901+0000 7fd32af72500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-12T17:05:56.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:05:56.903+0000 7fd32af72500 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-12T17:06:00.073 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-12T17:06:00.167 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-12T17:06:00.168 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:06:00.168 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-12T17:06:00.276 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:06:00.276 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-12T17:06:00.373 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:06:00.373 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-03-12T17:06:00.477 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:06:00.477 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-12T17:06:00.580 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:06:00.580 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-12T17:06:00.681 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:06:00.681 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-12T17:06:00.783 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:06:00.783 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-12T17:06:00.886 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T17:06:00.887 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-12T17:06:00.988 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T17:06:00.988 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-12T17:06:01.070 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T17:06:01.070 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-03-12T17:06:01.201 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T17:06:01.201 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-12T17:06:01.293 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T17:06:01.293 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-12T17:06:01.386 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T17:06:01.386 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-12T17:06:01.474 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T17:06:01.474 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-12T17:06:01.613 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:06:01.613 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-12T17:06:01.673 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:06:01.673 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-03-12T17:06:01.732 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T17:06:01.732 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-12T17:06:01.789 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T17:06:01.789 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-03-12T17:06:01.842 INFO:tasks.ceph:Adding keys to all mons... 2024-03-12T17:06:01.842 DEBUG:teuthology.orchestra.run.smithi052:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-12T17:06:01.849 DEBUG:teuthology.orchestra.run.smithi102:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-12T17:06:01.938 INFO:teuthology.orchestra.run.smithi102.stdout:[mgr.y] 2024-03-12T17:06:01.938 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQBWi/BlQIy0BxAAaeyxH1BvjKvpfEgvE41CnA== 2024-03-12T17:06:01.938 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.a] 2024-03-12T17:06:01.938 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQBWi/Bl4S66IRAAwl2dMUzSrsczo+luzg/UDg== 2024-03-12T17:06:01.938 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.c] 2024-03-12T17:06:01.939 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQBWi/Bllf0XNBAAcd9lZCjmGesX1rT6Nac5gQ== 2024-03-12T17:06:01.939 INFO:teuthology.orchestra.run.smithi102.stdout:[osd.0] 2024-03-12T17:06:01.939 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQBai/BlarXWBxAAfIQgbiJtEi3/J8gN4e0TGw== 2024-03-12T17:06:01.939 INFO:teuthology.orchestra.run.smithi102.stdout:[osd.1] 2024-03-12T17:06:01.939 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQBdi/BlzrueIRAApKYnmLqMVcuRhV/a4gFTLg== 2024-03-12T17:06:01.939 INFO:teuthology.orchestra.run.smithi102.stdout:[osd.2] 2024-03-12T17:06:01.939 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQBgi/BlDY3sLBAAmhhTmkfpCXbUWEKVNMsQ1w== 2024-03-12T17:06:01.939 INFO:teuthology.orchestra.run.smithi102.stdout:[osd.3] 2024-03-12T17:06:01.939 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQBki/BlaoypChAAZ8lV0XQxqrbfnZrYRE7XEw== 2024-03-12T17:06:01.939 INFO:teuthology.orchestra.run.smithi102.stdout:[mgr.x] 2024-03-12T17:06:01.939 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQBWi/BlKiFAFRAAwlsqCR8ar9YEpwpYQ1AcAQ== 2024-03-12T17:06:01.939 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.b] 2024-03-12T17:06:01.939 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQBXi/Bla8vvCBAAnvXNXGDljXRihZONHSKF/w== 2024-03-12T17:06:01.939 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.d] 2024-03-12T17:06:01.939 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQBXi/BlGHK2FxAAvywOwM5ZkXa16lyfAYB+NQ== 2024-03-12T17:06:01.940 INFO:teuthology.orchestra.run.smithi102.stdout:[osd.4] 2024-03-12T17:06:01.940 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQBri/Blle0NChAAB7AFUJK+mykQVMak16bp5A== 2024-03-12T17:06:01.940 INFO:teuthology.orchestra.run.smithi102.stdout:[osd.5] 2024-03-12T17:06:01.940 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQBui/Bl/xV3IhAAm44sVuUu6/Zqko1CHnOXkQ== 2024-03-12T17:06:01.940 INFO:teuthology.orchestra.run.smithi102.stdout:[osd.6] 2024-03-12T17:06:01.940 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQBxi/BloF8uLBAAiIYC2menNPYe+GZ/yH/Ozg== 2024-03-12T17:06:01.940 INFO:teuthology.orchestra.run.smithi102.stdout:[osd.7] 2024-03-12T17:06:01.940 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQB0i/BlBNXDNRAAaQuAdjotgiiroWuymS9noQ== 2024-03-12T17:06:01.940 INFO:teuthology.orchestra.run.smithi102.stdout:[client.0] 2024-03-12T17:06:01.940 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQBXi/BlAy3lJBAAI6pQDgbz22pC+i7V7j7TxA== 2024-03-12T17:06:01.940 INFO:teuthology.orchestra.run.smithi102.stdout:[client.2] 2024-03-12T17:06:01.940 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQBXi/Bl7W/5MBAAeI8bJMJRCkzAnki/Sxod7A== 2024-03-12T17:06:01.940 INFO:teuthology.orchestra.run.smithi102.stdout:[client.1] 2024-03-12T17:06:01.940 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQBXi/Bl+OJ1OxAALw+b2TNswYnjzIlx1uoiZA== 2024-03-12T17:06:01.940 INFO:teuthology.orchestra.run.smithi102.stdout:[client.3] 2024-03-12T17:06:01.941 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQBYi/Bl7fdOChAA5ExYqh/p4m1mRyJ5Xd3M8Q== 2024-03-12T17:06:01.943 INFO:teuthology.orchestra.run.smithi052.stdout:[mgr.y] 2024-03-12T17:06:01.943 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBWi/BlQIy0BxAAaeyxH1BvjKvpfEgvE41CnA== 2024-03-12T17:06:01.943 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.a] 2024-03-12T17:06:01.943 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBWi/Bl4S66IRAAwl2dMUzSrsczo+luzg/UDg== 2024-03-12T17:06:01.943 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.c] 2024-03-12T17:06:01.944 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBWi/Bllf0XNBAAcd9lZCjmGesX1rT6Nac5gQ== 2024-03-12T17:06:01.944 INFO:teuthology.orchestra.run.smithi052.stdout:[osd.0] 2024-03-12T17:06:01.944 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBai/BlarXWBxAAfIQgbiJtEi3/J8gN4e0TGw== 2024-03-12T17:06:01.944 INFO:teuthology.orchestra.run.smithi052.stdout:[osd.1] 2024-03-12T17:06:01.944 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBdi/BlzrueIRAApKYnmLqMVcuRhV/a4gFTLg== 2024-03-12T17:06:01.944 INFO:teuthology.orchestra.run.smithi052.stdout:[osd.2] 2024-03-12T17:06:01.944 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBgi/BlDY3sLBAAmhhTmkfpCXbUWEKVNMsQ1w== 2024-03-12T17:06:01.944 INFO:teuthology.orchestra.run.smithi052.stdout:[osd.3] 2024-03-12T17:06:01.944 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBki/BlaoypChAAZ8lV0XQxqrbfnZrYRE7XEw== 2024-03-12T17:06:01.944 INFO:teuthology.orchestra.run.smithi052.stdout:[mgr.x] 2024-03-12T17:06:01.944 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBWi/BlKiFAFRAAwlsqCR8ar9YEpwpYQ1AcAQ== 2024-03-12T17:06:01.944 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.b] 2024-03-12T17:06:01.944 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBXi/Bla8vvCBAAnvXNXGDljXRihZONHSKF/w== 2024-03-12T17:06:01.944 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.d] 2024-03-12T17:06:01.944 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBXi/BlGHK2FxAAvywOwM5ZkXa16lyfAYB+NQ== 2024-03-12T17:06:01.945 INFO:teuthology.orchestra.run.smithi052.stdout:[osd.4] 2024-03-12T17:06:01.945 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBri/Blle0NChAAB7AFUJK+mykQVMak16bp5A== 2024-03-12T17:06:01.945 INFO:teuthology.orchestra.run.smithi052.stdout:[osd.5] 2024-03-12T17:06:01.945 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBui/Bl/xV3IhAAm44sVuUu6/Zqko1CHnOXkQ== 2024-03-12T17:06:01.945 INFO:teuthology.orchestra.run.smithi052.stdout:[osd.6] 2024-03-12T17:06:01.945 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBxi/BloF8uLBAAiIYC2menNPYe+GZ/yH/Ozg== 2024-03-12T17:06:01.945 INFO:teuthology.orchestra.run.smithi052.stdout:[osd.7] 2024-03-12T17:06:01.945 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQB0i/BlBNXDNRAAaQuAdjotgiiroWuymS9noQ== 2024-03-12T17:06:01.945 INFO:teuthology.orchestra.run.smithi052.stdout:[client.0] 2024-03-12T17:06:01.945 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBXi/BlAy3lJBAAI6pQDgbz22pC+i7V7j7TxA== 2024-03-12T17:06:01.945 INFO:teuthology.orchestra.run.smithi052.stdout:[client.2] 2024-03-12T17:06:01.945 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBXi/Bl7W/5MBAAeI8bJMJRCkzAnki/Sxod7A== 2024-03-12T17:06:01.945 INFO:teuthology.orchestra.run.smithi052.stdout:[client.1] 2024-03-12T17:06:01.945 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBXi/Bl+OJ1OxAALw+b2TNswYnjzIlx1uoiZA== 2024-03-12T17:06:01.946 INFO:teuthology.orchestra.run.smithi052.stdout:[client.3] 2024-03-12T17:06:01.946 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBYi/Bl7fdOChAA5ExYqh/p4m1mRyJ5Xd3M8Q== 2024-03-12T17:06:01.948 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-12T17:06:01.955 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-12T17:06:02.098 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-12T17:06:02.106 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-12T17:06:02.254 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-12T17:06:02.258 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-12T17:06:02.405 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-12T17:06:02.410 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-12T17:06:02.548 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-12T17:06:02.554 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-12T17:06:02.693 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-12T17:06:02.699 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-12T17:06:02.844 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-12T17:06:02.849 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-12T17:06:02.995 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-12T17:06:03.002 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-12T17:06:03.143 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-12T17:06:03.151 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-12T17:06:03.302 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-12T17:06:03.307 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-12T17:06:03.457 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-12T17:06:03.461 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-12T17:06:03.604 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-12T17:06:03.612 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-12T17:06:03.757 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-12T17:06:03.762 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-12T17:06:03.907 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-12T17:06:03.915 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-12T17:06:04.060 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-12T17:06:04.065 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-12T17:06:04.213 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-12T17:06:04.220 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-12T17:06:04.369 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-12T17:06:04.376 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-12T17:06:04.520 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-12T17:06:04.527 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-12T17:06:04.670 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-12T17:06:04.670 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-12T17:06:04.773 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-12T17:06:04.934 DEBUG:teuthology.orchestra.run.smithi052:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-12T17:06:05.023 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-12T17:06:05.120 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-12T17:06:05.271 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-12T17:06:05.352 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-12T17:06:05.443 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-12T17:06:05.595 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-12T17:06:05.678 DEBUG:teuthology.orchestra.run.smithi052:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-12T17:06:05.685 DEBUG:teuthology.orchestra.run.smithi102:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-12T17:06:05.737 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-12T17:06:05.737 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-12T17:06:05.738 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-12T17:06:05.743 INFO:tasks.ceph.mon.a:Started 2024-03-12T17:06:05.743 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-12T17:06:05.743 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-12T17:06:05.748 INFO:tasks.ceph.mon.b:Started 2024-03-12T17:06:05.748 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-12T17:06:05.748 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-12T17:06:05.752 INFO:tasks.ceph.mon.c:Started 2024-03-12T17:06:05.752 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-12T17:06:05.752 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-12T17:06:05.753 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-12T17:06:05.759 INFO:tasks.ceph.mgr.y:Started 2024-03-12T17:06:05.760 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-12T17:06:05.760 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-12T17:06:05.764 INFO:tasks.ceph.mgr.x:Started 2024-03-12T17:06:05.764 INFO:tasks.ceph:Setting crush tunables to default 2024-03-12T17:06:05.765 DEBUG:teuthology.orchestra.run.smithi052:> sudo ceph --cluster ceph osd crush tunables default 2024-03-12T17:06:05.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.956+0000 7f6dc9400640 1 Processor -- start 2024-03-12T17:06:05.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.956+0000 7f6dc9400640 1 -- start start 2024-03-12T17:06:05.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.957+0000 7f6dc9400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6dc415c8d0 0x7f6dc40b8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:05.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.957+0000 7f6dc9400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6dc4160530 0x7f6dc40cd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:05.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.957+0000 7f6dc9400640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dc40d0b00 con 0x7f6dc40cd9d0 2024-03-12T17:06:05.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.957+0000 7f6dc9400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dc40cde40 con 0x7f6dc415c8d0 2024-03-12T17:06:05.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.957+0000 7f6dc9400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc4160cf0 con 0x7f6dc4160530 2024-03-12T17:06:05.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.958+0000 7f6dc3e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6dc415c8d0 0x7f6dc40b8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:05.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.958+0000 7f6dbbe00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6dc4160530 0x7f6dc40cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:05.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.958+0000 7f6dc3e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6dc415c8d0 0x7f6dc40b8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:36068/0 (socket says 172.21.15.52:36068) 2024-03-12T17:06:05.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.958+0000 7f6dc3e00640 1 -- 172.21.15.52:0/873428293 learned_addr learned my addr 172.21.15.52:0/873428293 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:05.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.958+0000 7f6dbbe00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6dc4160530 0x7f6dc40cd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.52:57804/0 (socket says 172.21.15.52:57804) 2024-03-12T17:06:05.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.958+0000 7f6dc3e00640 1 -- 172.21.15.52:0/873428293 >> v2:172.21.15.102:3300/0 conn(0x7f6dc415c8d0 msgr2=0x7f6dc40b8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-12T17:06:05.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.958+0000 7f6dc3e00640 1 -- 172.21.15.52:0/873428293 >> v2:172.21.15.102:3300/0 conn(0x7f6dc415c8d0 msgr2=0x7f6dc40b8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-12T17:06:05.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.958+0000 7f6dc3e00640 1 --2- 172.21.15.52:0/873428293 >> v2:172.21.15.102:3300/0 conn(0x7f6dc415c8d0 0x7f6dc40b8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-12T17:06:05.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.958+0000 7f6dc3e00640 1 --2- 172.21.15.52:0/873428293 >> v2:172.21.15.102:3300/0 conn(0x7f6dc415c8d0 0x7f6dc40b8280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-12T17:06:05.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.958+0000 7f6dbbe00640 1 -- 172.21.15.52:0/873428293 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6dc4160530 msgr2=0x7f6dc40cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-12T17:06:05.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.958+0000 7f6dbbe00640 1 -- 172.21.15.52:0/873428293 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6dc4160530 msgr2=0x7f6dc40cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-12T17:06:05.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.958+0000 7f6dbbe00640 1 --2- 172.21.15.52:0/873428293 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6dc4160530 0x7f6dc40cd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-12T17:06:05.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.958+0000 7f6dbbe00640 1 --2- 172.21.15.52:0/873428293 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6dc4160530 0x7f6dc40cd490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-12T17:06:05.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.967+0000 7f6dc3400640 1 -- 172.21.15.52:0/873428293 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1581458084 0 0) 0x7f6dc40d0b00 con 0x7f6dc40cd9d0 2024-03-12T17:06:05.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.967+0000 7f6dc3400640 1 -- 172.21.15.52:0/873428293 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6da8003520 con 0x7f6dc40cd9d0 2024-03-12T17:06:05.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.967+0000 7f6dc3400640 1 -- 172.21.15.52:0/873428293 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2059960196 0 0) 0x7f6da8003520 con 0x7f6dc40cd9d0 2024-03-12T17:06:05.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.967+0000 7f6dc3400640 1 -- 172.21.15.52:0/873428293 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6dc4160530 msgr2=0x7f6dc40cd490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-12T17:06:05.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.967+0000 7f6dc3400640 1 --2- 172.21.15.52:0/873428293 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6dc4160530 0x7f6dc40cd490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:05.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.967+0000 7f6dc3400640 1 -- 172.21.15.52:0/873428293 >> v2:172.21.15.102:3300/0 conn(0x7f6dc415c8d0 msgr2=0x7f6dc40b8280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-12T17:06:05.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.967+0000 7f6dc3400640 1 --2- 172.21.15.52:0/873428293 >> v2:172.21.15.102:3300/0 conn(0x7f6dc415c8d0 0x7f6dc40b8280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:05.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.967+0000 7f6dc3400640 1 -- 172.21.15.52:0/873428293 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc4160fd0 con 0x7f6dc40cd9d0 2024-03-12T17:06:05.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.967+0000 7f6dc3400640 1 -- 172.21.15.52:0/873428293 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f6dbc0019f0 con 0x7f6dc40cd9d0 2024-03-12T17:06:05.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.967+0000 7f6dc3400640 1 -- 172.21.15.52:0/873428293 >> v1:172.21.15.52:6789/0 conn(0x7f6dc40cd9d0 legacy=0x7f6dc40b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:05.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.967+0000 7f6dc3400640 1 --2- 172.21.15.52:0/873428293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6da8003970 0x7f6da8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:05.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.967+0000 7f6dc3400640 1 --2- 172.21.15.52:0/873428293 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6dc4160530 0x7f6da80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:05.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.967+0000 7f6dc3400640 1 --2- 172.21.15.52:0/873428293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6da8004d00 0x7f6da8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:05.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.967+0000 7f6dc3400640 1 -- 172.21.15.52:0/873428293 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc40cde40 con 0x7f6da8004d00 2024-03-12T17:06:05.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.967+0000 7f6dc3400640 1 -- 172.21.15.52:0/873428293 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc4160cf0 con 0x7f6da8003970 2024-03-12T17:06:05.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.967+0000 7f6dc3400640 1 -- 172.21.15.52:0/873428293 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da800ba30 con 0x7f6dc4160530 2024-03-12T17:06:05.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.968+0000 7f6dc8a00640 1 --2- 172.21.15.52:0/873428293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6da8004d00 0x7f6da8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:05.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.968+0000 7f6dbbe00640 1 --2- 172.21.15.52:0/873428293 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6dc4160530 0x7f6da80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:05.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.968+0000 7f6dc3e00640 1 --2- 172.21.15.52:0/873428293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6da8003970 0x7f6da8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:05.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.968+0000 7f6dc8a00640 1 -- 172.21.15.52:0/873428293 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6dc4160530 msgr2=0x7f6da80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:05.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.968+0000 7f6dc8a00640 1 --2- 172.21.15.52:0/873428293 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6dc4160530 0x7f6da80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:05.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.968+0000 7f6dc8a00640 1 -- 172.21.15.52:0/873428293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6da8003970 msgr2=0x7f6da8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:05.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.968+0000 7f6dc8a00640 1 --2- 172.21.15.52:0/873428293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6da8003970 0x7f6da8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:05.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.968+0000 7f6dc8a00640 1 -- 172.21.15.52:0/873428293 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da8003520 con 0x7f6da8004d00 2024-03-12T17:06:05.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.969+0000 7f6dc8a00640 1 --2- 172.21.15.52:0/873428293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6da8004d00 0x7f6da8005180 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc00c4f0 tx=0x7f6da80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a22f81973132d85 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:05.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.970+0000 7f6dc3400640 1 -- 172.21.15.52:0/873428293 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc02cd80 con 0x7f6da8004d00 2024-03-12T17:06:05.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.970+0000 7f6dc3400640 1 -- 172.21.15.52:0/873428293 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dbc034440 con 0x7f6da8004d00 2024-03-12T17:06:05.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.970+0000 7f6dc3400640 1 -- 172.21.15.52:0/873428293 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc0347f0 con 0x7f6da8004d00 2024-03-12T17:06:05.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.970+0000 7f6dc9400640 1 -- 172.21.15.52:0/873428293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6da8004d00 msgr2=0x7f6da8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:05.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.970+0000 7f6dc9400640 1 --2- 172.21.15.52:0/873428293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6da8004d00 0x7f6da8005180 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc00c4f0 tx=0x7f6da80057f0 comp rx=0 tx=0).stop 2024-03-12T17:06:05.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.970+0000 7f6dc8a00640 1 -- 172.21.15.52:0/873428293 reap_dead start 2024-03-12T17:06:05.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.971+0000 7f6dc9400640 1 -- 172.21.15.52:0/873428293 shutdown_connections 2024-03-12T17:06:05.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.971+0000 7f6dc9400640 1 -- 172.21.15.52:0/873428293 >> 172.21.15.52:0/873428293 conn(0x7f6dc406ae90 msgr2=0x7f6dc406b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:05.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.971+0000 7f6dc9400640 1 -- 172.21.15.52:0/873428293 shutdown_connections 2024-03-12T17:06:05.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.971+0000 7f6dc9400640 1 -- 172.21.15.52:0/873428293 wait complete. 2024-03-12T17:06:05.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.972+0000 7f6dc9400640 1 Processor -- start 2024-03-12T17:06:05.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.973+0000 7f6dc9400640 1 -- start start 2024-03-12T17:06:05.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.973+0000 7f6dc9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6da8004d00 0x7f6dc41f2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:05.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.973+0000 7f6dc9400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6dc415ccb0 0x7f6dc40cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:05.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.973+0000 7f6dc9400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dc40b3730 0x7f6dc40b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:05.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.974+0000 7f6dc9400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc40b4120 con 0x7f6dc40b3730 2024-03-12T17:06:05.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.974+0000 7f6dc9400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc41f3100 con 0x7f6da8004d00 2024-03-12T17:06:05.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.974+0000 7f6dc9400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc415d220 con 0x7f6dc415ccb0 2024-03-12T17:06:05.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.974+0000 7f6dc8a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dc40b3730 0x7f6dc40b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:05.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.974+0000 7f6dc8a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dc40b3730 0x7f6dc40b3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50456/0 (socket says 172.21.15.52:50456) 2024-03-12T17:06:05.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.974+0000 7f6dc8a00640 1 -- 172.21.15.52:0/2604663499 learned_addr learned my addr 172.21.15.52:0/2604663499 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:05.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.974+0000 7f6dbbe00640 1 --2- 172.21.15.52:0/2604663499 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6dc415ccb0 0x7f6dc40cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:05.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.974+0000 7f6dc3e00640 1 --2- 172.21.15.52:0/2604663499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6da8004d00 0x7f6dc41f2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:05.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.974+0000 7f6dc8a00640 1 -- 172.21.15.52:0/2604663499 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6dc415ccb0 msgr2=0x7f6dc40cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:05.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.974+0000 7f6dc8a00640 1 --2- 172.21.15.52:0/2604663499 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6dc415ccb0 0x7f6dc40cc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:05.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.974+0000 7f6dc8a00640 1 -- 172.21.15.52:0/2604663499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6da8004d00 msgr2=0x7f6dc41f2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:05.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.974+0000 7f6dc8a00640 1 --2- 172.21.15.52:0/2604663499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6da8004d00 0x7f6dc41f2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:05.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.974+0000 7f6dc8a00640 1 -- 172.21.15.52:0/2604663499 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc415d500 con 0x7f6dc40b3730 2024-03-12T17:06:05.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.974+0000 7f6dc3e00640 1 --2- 172.21.15.52:0/2604663499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6da8004d00 0x7f6dc41f2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:06:05.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.975+0000 7f6dc8a00640 1 --2- 172.21.15.52:0/2604663499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dc40b3730 0x7f6dc40b3be0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc02c720 tx=0x7f6dbc001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:05.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.975+0000 7f6dc0c00640 1 -- 172.21.15.52:0/2604663499 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc003d80 con 0x7f6dc40b3730 2024-03-12T17:06:05.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.975+0000 7f6dc0c00640 1 -- 172.21.15.52:0/2604663499 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dbc002020 con 0x7f6dc40b3730 2024-03-12T17:06:05.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.975+0000 7f6dc0c00640 1 -- 172.21.15.52:0/2604663499 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc034aa0 con 0x7f6dc40b3730 2024-03-12T17:06:05.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.975+0000 7f6dc9400640 1 -- 172.21.15.52:0/2604663499 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dc415d7e0 con 0x7f6dc40b3730 2024-03-12T17:06:05.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.975+0000 7f6dc9400640 1 -- 172.21.15.52:0/2604663499 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc415dda0 con 0x7f6dc40b3730 2024-03-12T17:06:05.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.976+0000 7f6dc0c00640 1 -- 172.21.15.52:0/2604663499 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6dbc03b050 con 0x7f6dc40b3730 2024-03-12T17:06:05.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.976+0000 7f6dc0c00640 1 -- 172.21.15.52:0/2604663499 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6dbc047450 con 0x7f6dc40b3730 2024-03-12T17:06:05.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.976+0000 7f6dc9400640 1 -- 172.21.15.52:0/2604663499 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dc415df90 con 0x7f6dc40b3730 2024-03-12T17:06:05.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:05.980+0000 7f6dc0c00640 1 -- 172.21.15.52:0/2604663499 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f6dbc04c060 con 0x7f6dc40b3730 2024-03-12T17:06:06.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:06.052+0000 7f6dc9400640 1 -- 172.21.15.52:0/2604663499 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f6dc404e030 con 0x7f6dc40b3730 2024-03-12T17:06:06.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:06.058+0000 7f6dc0c00640 1 -- 172.21.15.52:0/2604663499 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f6dbc042070 con 0x7f6dc40b3730 2024-03-12T17:06:06.051 INFO:teuthology.orchestra.run.smithi052.stderr:adjusted tunables profile to default 2024-03-12T17:06:06.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:06.059+0000 7f6dc9400640 1 -- 172.21.15.52:0/2604663499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dc40b3730 msgr2=0x7f6dc40b3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:06.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:06.059+0000 7f6dc9400640 1 --2- 172.21.15.52:0/2604663499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dc40b3730 0x7f6dc40b3be0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc02c720 tx=0x7f6dbc001360 comp rx=0 tx=0).stop 2024-03-12T17:06:06.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:06.059+0000 7f6dc9400640 1 -- 172.21.15.52:0/2604663499 shutdown_connections 2024-03-12T17:06:06.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:06.059+0000 7f6dc9400640 1 --2- 172.21.15.52:0/2604663499 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6dc415ccb0 0x7f6dc40cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:06.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:06.059+0000 7f6dc9400640 1 --2- 172.21.15.52:0/2604663499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dc40b3730 0x7f6dc40b3be0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:06.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:06.059+0000 7f6dc9400640 1 --2- 172.21.15.52:0/2604663499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6da8004d00 0x7f6dc41f2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:06.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:06.059+0000 7f6dc9400640 1 -- 172.21.15.52:0/2604663499 >> 172.21.15.52:0/2604663499 conn(0x7f6dc406ae90 msgr2=0x7f6dc415a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:06.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:06.059+0000 7f6dc9400640 1 -- 172.21.15.52:0/2604663499 shutdown_connections 2024-03-12T17:06:06.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:06.059+0000 7f6dc9400640 1 -- 172.21.15.52:0/2604663499 wait complete. 2024-03-12T17:06:06.070 INFO:tasks.ceph:check_enable_crimson: False 2024-03-12T17:06:06.070 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-12T17:06:06.070 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:06:06.070 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-12T17:06:06.153 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:06:06.154 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-12T17:06:06.273 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:06:06.273 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-12T17:06:06.404 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:06:06.404 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-12T17:06:06.491 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T17:06:06.491 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-12T17:06:06.576 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T17:06:06.576 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-12T17:06:06.697 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T17:06:06.697 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-12T17:06:06.736 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-03-12T17:06:06.742+0000 7fb63200d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-12T17:06:06.745 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-03-12T17:06:06.749+0000 7f9048e13040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-12T17:06:06.801 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-03-12T17:06:06.808+0000 7fb63200d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-12T17:06:06.811 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-03-12T17:06:06.817+0000 7f9048e13040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-12T17:06:06.825 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T17:06:06.825 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-12T17:06:06.953 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new ce602581-c510-46f1-9792-691ff2a839a6 0 2024-03-12T17:06:07.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.189+0000 7f1eeae00640 1 Processor -- start 2024-03-12T17:06:07.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.189+0000 7f1eeae00640 1 -- start start 2024-03-12T17:06:07.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.191+0000 7f1eeae00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1ee406ffb0 0x7f1ee406e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:07.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.191+0000 7f1eeae00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ee40996f0 con 0x7f1ee415a610 2024-03-12T17:06:07.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.191+0000 7f1eeae00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ee4098ae0 con 0x7f1ee406eb40 2024-03-12T17:06:07.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.191+0000 7f1eeae00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ee406eff0 con 0x7f1ee406ffb0 2024-03-12T17:06:07.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.191+0000 7f1ee9000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1ee406ffb0 0x7f1ee406e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:07.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.191+0000 7f1ee9000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1ee406ffb0 0x7f1ee406e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52974/0 (socket says 172.21.15.102:52974) 2024-03-12T17:06:07.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.191+0000 7f1ee9000640 1 -- 172.21.15.102:0/1964895059 learned_addr learned my addr 172.21.15.102:0/1964895059 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-03-12T17:06:07.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.192+0000 7f1ee9000640 1 -- 172.21.15.102:0/1964895059 >> v1:172.21.15.102:6789/0 conn(0x7f1ee406eb40 legacy=0x7f1ee40caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:07.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.192+0000 7f1ee9000640 1 -- 172.21.15.102:0/1964895059 >> v1:172.21.15.52:6789/0 conn(0x7f1ee415a610 legacy=0x7f1ee40ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:07.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.192+0000 7f1ee9000640 1 -- 172.21.15.102:0/1964895059 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee4098dc0 con 0x7f1ee406ffb0 2024-03-12T17:06:07.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.193+0000 7f1ee9000640 1 --2- 172.21.15.102:0/1964895059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ee406ffb0 0x7f1ee406e600 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0009870 tx=0x7f1ed002b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfafb5e092e02bfa server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:07.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.193+0000 7f1ee3e00640 1 -- 172.21.15.102:0/1964895059 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0030860 con 0x7f1ee406ffb0 2024-03-12T17:06:07.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.193+0000 7f1ee3e00640 1 -- 172.21.15.102:0/1964895059 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed00309c0 con 0x7f1ee406ffb0 2024-03-12T17:06:07.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.193+0000 7f1ee3e00640 1 -- 172.21.15.102:0/1964895059 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0030d50 con 0x7f1ee406ffb0 2024-03-12T17:06:07.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.193+0000 7f1eeae00640 1 -- 172.21.15.102:0/1964895059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ee406ffb0 msgr2=0x7f1ee406e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:07.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.193+0000 7f1eeae00640 1 --2- 172.21.15.102:0/1964895059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ee406ffb0 0x7f1ee406e600 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0009870 tx=0x7f1ed002b9a0 comp rx=0 tx=0).stop 2024-03-12T17:06:07.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.194+0000 7f1eeae00640 1 -- 172.21.15.102:0/1964895059 shutdown_connections 2024-03-12T17:06:07.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.194+0000 7f1eeae00640 1 --2- 172.21.15.102:0/1964895059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ee406ffb0 0x7f1ee406e600 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:07.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.194+0000 7f1eeae00640 1 -- 172.21.15.102:0/1964895059 >> 172.21.15.102:0/1964895059 conn(0x7f1ee40d0520 msgr2=0x7f1ee40d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:07.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.194+0000 7f1eeae00640 1 -- 172.21.15.102:0/1964895059 shutdown_connections 2024-03-12T17:06:07.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.194+0000 7f1eeae00640 1 -- 172.21.15.102:0/1964895059 wait complete. 2024-03-12T17:06:07.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.195+0000 7f1eeae00640 1 Processor -- start 2024-03-12T17:06:07.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.195+0000 7f1eeae00640 1 -- start start 2024-03-12T17:06:07.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.196+0000 7f1eeae00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ee406eb40 0x7f1ee4145a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:07.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.196+0000 7f1eeae00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ee406ffb0 0x7f1ee40b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:07.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.196+0000 7f1eeae00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ee415a610 0x7f1ee40b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:07.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.196+0000 7f1eeae00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee40c83b0 con 0x7f1ee406eb40 2024-03-12T17:06:07.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.196+0000 7f1eeae00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee4159cd0 con 0x7f1ee415a610 2024-03-12T17:06:07.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.196+0000 7f1eeae00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee415d3c0 con 0x7f1ee406ffb0 2024-03-12T17:06:07.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.197+0000 7f1eea400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ee415a610 0x7f1ee40b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:07.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.197+0000 7f1ee9000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ee406ffb0 0x7f1ee40b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:07.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.197+0000 7f1eea400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ee415a610 0x7f1ee40b38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52986/0 (socket says 172.21.15.102:52986) 2024-03-12T17:06:07.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.197+0000 7f1eea400640 1 -- 172.21.15.102:0/1816527577 learned_addr learned my addr 172.21.15.102:0/1816527577 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-03-12T17:06:07.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.197+0000 7f1ee9a00640 1 --2- 172.21.15.102:0/1816527577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ee406eb40 0x7f1ee4145a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:07.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.197+0000 7f1eea400640 1 -- 172.21.15.102:0/1816527577 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ee406ffb0 msgr2=0x7f1ee40b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:07.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.197+0000 7f1eea400640 1 --2- 172.21.15.102:0/1816527577 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ee406ffb0 0x7f1ee40b3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:07.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.197+0000 7f1eea400640 1 -- 172.21.15.102:0/1816527577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ee406eb40 msgr2=0x7f1ee4145a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:07.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.197+0000 7f1eea400640 1 --2- 172.21.15.102:0/1816527577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ee406eb40 0x7f1ee4145a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:07.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.197+0000 7f1eea400640 1 -- 172.21.15.102:0/1816527577 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed00094d0 con 0x7f1ee415a610 2024-03-12T17:06:07.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.197+0000 7f1eea400640 1 --2- 172.21.15.102:0/1816527577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ee415a610 0x7f1ee40b38b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1edc00c7a0 tx=0x7f1edc00cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:07.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.197+0000 7f1ee1600640 1 -- 172.21.15.102:0/1816527577 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc003b70 con 0x7f1ee415a610 2024-03-12T17:06:07.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.198+0000 7f1ee1600640 1 -- 172.21.15.102:0/1816527577 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1edc003cf0 con 0x7f1ee415a610 2024-03-12T17:06:07.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.198+0000 7f1ee1600640 1 -- 172.21.15.102:0/1816527577 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc00aa90 con 0x7f1ee415a610 2024-03-12T17:06:07.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.198+0000 7f1eeae00640 1 -- 172.21.15.102:0/1816527577 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ee415d6a0 con 0x7f1ee415a610 2024-03-12T17:06:07.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.198+0000 7f1eeae00640 1 -- 172.21.15.102:0/1816527577 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee40c7fd0 con 0x7f1ee415a610 2024-03-12T17:06:07.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.198+0000 7f1ee1600640 1 -- 172.21.15.102:0/1816527577 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1edc0050d0 con 0x7f1ee415a610 2024-03-12T17:06:07.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.198+0000 7f1ee1600640 1 -- 172.21.15.102:0/1816527577 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1edc012830 con 0x7f1ee415a610 2024-03-12T17:06:07.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.199+0000 7f1eeae00640 1 -- 172.21.15.102:0/1816527577 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee40c84f0 con 0x7f1ee415a610 2024-03-12T17:06:07.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.202+0000 7f1ee1600640 1 -- 172.21.15.102:0/1816527577 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f1ee40c84f0 con 0x7f1ee415a610 2024-03-12T17:06:07.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.279+0000 7f1eeae00640 1 -- 172.21.15.102:0/1816527577 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ce602581-c510-46f1-9792-691ff2a839a6", "id": 0} v 0) v1 -- 0x7f1ee404e030 con 0x7f1ee415a610 2024-03-12T17:06:07.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.285+0000 7f1ee1600640 1 -- 172.21.15.102:0/1816527577 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ce602581-c510-46f1-9792-691ff2a839a6", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1edc02a0b0 con 0x7f1ee415a610 2024-03-12T17:06:07.279 INFO:teuthology.orchestra.run.smithi102.stdout:0 2024-03-12T17:06:07.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.287+0000 7f1eeae00640 1 -- 172.21.15.102:0/1816527577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ee415a610 msgr2=0x7f1ee40b38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:07.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.287+0000 7f1eeae00640 1 --2- 172.21.15.102:0/1816527577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ee415a610 0x7f1ee40b38b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1edc00c7a0 tx=0x7f1edc00cc60 comp rx=0 tx=0).stop 2024-03-12T17:06:07.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.287+0000 7f1eeae00640 1 -- 172.21.15.102:0/1816527577 shutdown_connections 2024-03-12T17:06:07.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.287+0000 7f1eeae00640 1 --2- 172.21.15.102:0/1816527577 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ee406ffb0 0x7f1ee40b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:07.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.287+0000 7f1eeae00640 1 --2- 172.21.15.102:0/1816527577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ee415a610 0x7f1ee40b38b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:07.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.287+0000 7f1eeae00640 1 --2- 172.21.15.102:0/1816527577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ee406eb40 0x7f1ee4145a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:07.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.287+0000 7f1eeae00640 1 -- 172.21.15.102:0/1816527577 >> 172.21.15.102:0/1816527577 conn(0x7f1ee40d0520 msgr2=0x7f1ee406b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:07.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.287+0000 7f1eeae00640 1 -- 172.21.15.102:0/1816527577 shutdown_connections 2024-03-12T17:06:07.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.287+0000 7f1eeae00640 1 -- 172.21.15.102:0/1816527577 wait complete. 2024-03-12T17:06:07.297 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new 91ba0742-065a-4c84-8000-bbda30993c50 1 2024-03-12T17:06:07.437 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-03-12T17:06:07.443+0000 7fb63200d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-12T17:06:07.477 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-03-12T17:06:07.484+0000 7f9048e13040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-12T17:06:07.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.493+0000 7fe24ba00640 1 Processor -- start 2024-03-12T17:06:07.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.493+0000 7fe24ba00640 1 -- start start 2024-03-12T17:06:07.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.494+0000 7fe24ba00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe2440b47d0 0x7fe2440b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:07.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.494+0000 7fe24ba00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2441617d0 con 0x7fe2440d0f20 2024-03-12T17:06:07.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.494+0000 7fe24ba00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2440b5150 con 0x7fe24415cd10 2024-03-12T17:06:07.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.494+0000 7fe24ba00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe244160c50 con 0x7fe2440b47d0 2024-03-12T17:06:07.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.495+0000 7fe249000640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fe24415cd10 0x7fe2440cdc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:52270/0 (socket says 172.21.15.102:52270) 2024-03-12T17:06:07.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.495+0000 7fe249000640 1 -- 172.21.15.102:0/247098657 learned_addr learned my addr 172.21.15.102:0/247098657 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-03-12T17:06:07.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.495+0000 7fe243e00640 1 --2- 172.21.15.102:0/247098657 >> v2:172.21.15.52:3300/0 conn(0x7fe2440b47d0 0x7fe2440b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:07.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.495+0000 7fe243400640 1 -- 172.21.15.102:0/247098657 <== mon.1 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2233787486 0 0) 0x7fe2440b5150 con 0x7fe24415cd10 2024-03-12T17:06:07.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.495+0000 7fe243400640 1 -- 172.21.15.102:0/247098657 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe22c003520 con 0x7fe24415cd10 2024-03-12T17:06:07.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.495+0000 7fe243400640 1 -- 172.21.15.102:0/247098657 <== mon.1 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3175437272 0 0) 0x7fe22c003520 con 0x7fe24415cd10 2024-03-12T17:06:07.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.495+0000 7fe243400640 1 -- 172.21.15.102:0/247098657 >> v2:172.21.15.52:3300/0 conn(0x7fe2440b47d0 msgr2=0x7fe2440b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:07.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.495+0000 7fe243400640 1 --2- 172.21.15.102:0/247098657 >> v2:172.21.15.52:3300/0 conn(0x7fe2440b47d0 0x7fe2440b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:07.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.495+0000 7fe243400640 1 -- 172.21.15.102:0/247098657 >> v1:172.21.15.52:6789/0 conn(0x7fe2440d0f20 legacy=0x7fe2440d1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:07.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.495+0000 7fe243400640 1 -- 172.21.15.102:0/247098657 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe244160f30 con 0x7fe24415cd10 2024-03-12T17:06:07.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.495+0000 7fe243400640 1 -- 172.21.15.102:0/247098657 <== mon.1 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7fe238001670 con 0x7fe24415cd10 2024-03-12T17:06:07.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.496+0000 7fe243400640 1 -- 172.21.15.102:0/247098657 >> v1:172.21.15.102:6789/0 conn(0x7fe24415cd10 legacy=0x7fe2440cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:07.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.496+0000 7fe243400640 1 --2- 172.21.15.102:0/247098657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe22c003970 0x7fe22c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:07.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.496+0000 7fe243400640 1 --2- 172.21.15.102:0/247098657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe22c0047c0 0x7fe22c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:07.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.496+0000 7fe243400640 1 --2- 172.21.15.102:0/247098657 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe22c005180 0x7fe22c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:07.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.496+0000 7fe243400640 1 -- 172.21.15.102:0/247098657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe244160c50 con 0x7fe22c0047c0 2024-03-12T17:06:07.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.496+0000 7fe243400640 1 -- 172.21.15.102:0/247098657 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c00c020 con 0x7fe22c003970 2024-03-12T17:06:07.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.496+0000 7fe243400640 1 -- 172.21.15.102:0/247098657 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c00c1e0 con 0x7fe22c005180 2024-03-12T17:06:07.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.496+0000 7fe249000640 1 --2- 172.21.15.102:0/247098657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe22c003970 0x7fe22c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:07.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.496+0000 7fe243e00640 1 --2- 172.21.15.102:0/247098657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe22c0047c0 0x7fe22c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:07.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.496+0000 7fe249a00640 1 --2- 172.21.15.102:0/247098657 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe22c005180 0x7fe22c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:07.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.496+0000 7fe249000640 1 -- 172.21.15.102:0/247098657 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe22c005180 msgr2=0x7fe22c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:07.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.496+0000 7fe249000640 1 --2- 172.21.15.102:0/247098657 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe22c005180 0x7fe22c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:07.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.496+0000 7fe249000640 1 -- 172.21.15.102:0/247098657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe22c0047c0 msgr2=0x7fe22c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:07.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.496+0000 7fe249000640 1 --2- 172.21.15.102:0/247098657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe22c0047c0 0x7fe22c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:07.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.496+0000 7fe249000640 1 -- 172.21.15.102:0/247098657 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22c003520 con 0x7fe22c003970 2024-03-12T17:06:07.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.496+0000 7fe243e00640 1 --2- 172.21.15.102:0/247098657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe22c0047c0 0x7fe22c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:06:07.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.496+0000 7fe249a00640 1 --2- 172.21.15.102:0/247098657 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe22c005180 0x7fe22c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:07.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.497+0000 7fe249a00640 1 -- 172.21.15.102:0/247098657 reap_dead start 2024-03-12T17:06:07.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.498+0000 7fe249000640 1 --2- 172.21.15.102:0/247098657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe22c003970 0x7fe22c003d90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe23800c240 tx=0x7fe22c0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dbe41f374bf0b0c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:07.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.498+0000 7fe243400640 1 -- 172.21.15.102:0/247098657 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe238005ad0 con 0x7fe22c003970 2024-03-12T17:06:07.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.498+0000 7fe243400640 1 -- 172.21.15.102:0/247098657 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe238005c50 con 0x7fe22c003970 2024-03-12T17:06:07.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.498+0000 7fe243400640 1 -- 172.21.15.102:0/247098657 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23802c8b0 con 0x7fe22c003970 2024-03-12T17:06:07.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.499+0000 7fe24ba00640 1 -- 172.21.15.102:0/247098657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe22c003970 msgr2=0x7fe22c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:07.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.499+0000 7fe24ba00640 1 --2- 172.21.15.102:0/247098657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe22c003970 0x7fe22c003d90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe23800c240 tx=0x7fe22c0057f0 comp rx=0 tx=0).stop 2024-03-12T17:06:07.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.499+0000 7fe24ba00640 1 -- 172.21.15.102:0/247098657 shutdown_connections 2024-03-12T17:06:07.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.499+0000 7fe24ba00640 1 --2- 172.21.15.102:0/247098657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe22c003970 0x7fe22c003d90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:07.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.499+0000 7fe24ba00640 1 -- 172.21.15.102:0/247098657 >> 172.21.15.102:0/247098657 conn(0x7fe2440cf830 msgr2=0x7fe2440cfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:07.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.499+0000 7fe24ba00640 1 -- 172.21.15.102:0/247098657 shutdown_connections 2024-03-12T17:06:07.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.499+0000 7fe24ba00640 1 -- 172.21.15.102:0/247098657 wait complete. 2024-03-12T17:06:07.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.501+0000 7fe24ba00640 1 Processor -- start 2024-03-12T17:06:07.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.501+0000 7fe24ba00640 1 -- start start 2024-03-12T17:06:07.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.502+0000 7fe24ba00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe22c003970 0x7fe2441489d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:07.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.502+0000 7fe24ba00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe244149080 0x7fe2440cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:07.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.502+0000 7fe24ba00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2440cdb00 0x7fe2441f5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:07.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.502+0000 7fe24ba00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2440cdf80 con 0x7fe22c003970 2024-03-12T17:06:07.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.502+0000 7fe24ba00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24414b490 con 0x7fe2440cdb00 2024-03-12T17:06:07.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.502+0000 7fe24ba00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2441f6550 con 0x7fe244149080 2024-03-12T17:06:07.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.502+0000 7fe243e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe244149080 0x7fe2440cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:07.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.502+0000 7fe249a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2440cdb00 0x7fe2441f5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:07.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.502+0000 7fe243e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe244149080 0x7fe2440cd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:36880/0 (socket says 172.21.15.102:36880) 2024-03-12T17:06:07.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.502+0000 7fe243e00640 1 -- 172.21.15.102:0/2001826456 learned_addr learned my addr 172.21.15.102:0/2001826456 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-03-12T17:06:07.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.502+0000 7fe249000640 1 --2- 172.21.15.102:0/2001826456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe22c003970 0x7fe2441489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:07.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.503+0000 7fe243e00640 1 -- 172.21.15.102:0/2001826456 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2440cdb00 msgr2=0x7fe2441f5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:07.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.503+0000 7fe243e00640 1 --2- 172.21.15.102:0/2001826456 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2440cdb00 0x7fe2441f5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:07.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.503+0000 7fe243e00640 1 -- 172.21.15.102:0/2001826456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe22c003970 msgr2=0x7fe2441489d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:07.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.503+0000 7fe243e00640 1 --2- 172.21.15.102:0/2001826456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe22c003970 0x7fe2441489d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:07.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.503+0000 7fe243e00640 1 -- 172.21.15.102:0/2001826456 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2441f6830 con 0x7fe244149080 2024-03-12T17:06:07.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.503+0000 7fe243e00640 1 --2- 172.21.15.102:0/2001826456 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe244149080 0x7fe2440cd5c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe230011700 tx=0x7fe230011bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:07.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.503+0000 7fe240c00640 1 -- 172.21.15.102:0/2001826456 <== mon.2 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23000f9c0 con 0x7fe244149080 2024-03-12T17:06:07.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.503+0000 7fe240c00640 1 -- 172.21.15.102:0/2001826456 <== mon.2 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23000fb20 con 0x7fe244149080 2024-03-12T17:06:07.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.503+0000 7fe240c00640 1 -- 172.21.15.102:0/2001826456 <== mon.2 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23000fe90 con 0x7fe244149080 2024-03-12T17:06:07.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.503+0000 7fe24ba00640 1 -- 172.21.15.102:0/2001826456 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2440c7d00 con 0x7fe244149080 2024-03-12T17:06:07.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.503+0000 7fe24ba00640 1 -- 172.21.15.102:0/2001826456 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2440c82c0 con 0x7fe244149080 2024-03-12T17:06:07.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.504+0000 7fe240c00640 1 -- 172.21.15.102:0/2001826456 <== mon.2 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe230005990 con 0x7fe244149080 2024-03-12T17:06:07.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.504+0000 7fe240c00640 1 -- 172.21.15.102:0/2001826456 <== mon.2 v2:172.21.15.102:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fe230015460 con 0x7fe244149080 2024-03-12T17:06:07.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.504+0000 7fe24ba00640 1 -- 172.21.15.102:0/2001826456 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2440c7b60 con 0x7fe244149080 2024-03-12T17:06:07.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.507+0000 7fe240c00640 1 -- 172.21.15.102:0/2001826456 <== mon.2 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fe2440c7b60 con 0x7fe244149080 2024-03-12T17:06:07.502 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-03-12T17:06:07.509+0000 7fb63200d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-12T17:06:07.542 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-03-12T17:06:07.549+0000 7f9048e13040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-12T17:06:07.556 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-03-12T17:06:07.562+0000 7fb63200d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-12T17:06:07.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.585+0000 7fe24ba00640 1 -- 172.21.15.102:0/2001826456 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd new", "uuid": "91ba0742-065a-4c84-8000-bbda30993c50", "id": 1} v 0) v1 -- 0x7fe244066bb0 con 0x7fe244149080 2024-03-12T17:06:07.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.592+0000 7fe240c00640 1 -- 172.21.15.102:0/2001826456 <== mon.2 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "91ba0742-065a-4c84-8000-bbda30993c50", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe230022020 con 0x7fe244149080 2024-03-12T17:06:07.586 INFO:teuthology.orchestra.run.smithi102.stdout:1 2024-03-12T17:06:07.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.593+0000 7fe24ba00640 1 -- 172.21.15.102:0/2001826456 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe244149080 msgr2=0x7fe2440cd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:07.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.593+0000 7fe24ba00640 1 --2- 172.21.15.102:0/2001826456 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe244149080 0x7fe2440cd5c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe230011700 tx=0x7fe230011bc0 comp rx=0 tx=0).stop 2024-03-12T17:06:07.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.593+0000 7fe24ba00640 1 -- 172.21.15.102:0/2001826456 shutdown_connections 2024-03-12T17:06:07.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.593+0000 7fe24ba00640 1 --2- 172.21.15.102:0/2001826456 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe244149080 0x7fe2440cd5c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:07.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.593+0000 7fe24ba00640 1 --2- 172.21.15.102:0/2001826456 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2440cdb00 0x7fe2441f5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:07.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.593+0000 7fe24ba00640 1 --2- 172.21.15.102:0/2001826456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe22c003970 0x7fe2441489d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:07.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.594+0000 7fe24ba00640 1 -- 172.21.15.102:0/2001826456 >> 172.21.15.102:0/2001826456 conn(0x7fe2440cf830 msgr2=0x7fe2440b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:07.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.594+0000 7fe24ba00640 1 -- 172.21.15.102:0/2001826456 shutdown_connections 2024-03-12T17:06:07.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.594+0000 7fe24ba00640 1 -- 172.21.15.102:0/2001826456 wait complete. 2024-03-12T17:06:07.595 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-03-12T17:06:07.601+0000 7f9048e13040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-12T17:06:07.603 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new c166769b-d6f9-4435-a0ae-203fb1fb8743 2 2024-03-12T17:06:07.623 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-03-12T17:06:07.629+0000 7fb63200d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-12T17:06:07.661 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-03-12T17:06:07.667+0000 7f9048e13040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-12T17:06:07.675 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-03-12T17:06:07.682+0000 7fb63200d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-12T17:06:07.713 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-03-12T17:06:07.719+0000 7f9048e13040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-12T17:06:07.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.792+0000 7fcb1e800640 1 Processor -- start 2024-03-12T17:06:07.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.792+0000 7fcb1e800640 1 -- start start 2024-03-12T17:06:07.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.793+0000 7fcb1e800640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fcb1815c780 0x7fcb180cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:07.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.794+0000 7fcb1e800640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb180cd130 con 0x7fcb180d0a40 2024-03-12T17:06:07.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.794+0000 7fcb1e800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb180cf9a0 con 0x7fcb180d02d0 2024-03-12T17:06:07.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.794+0000 7fcb1e800640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb180cfc80 con 0x7fcb1815c780 2024-03-12T17:06:07.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.794+0000 7fcb1ca00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fcb180d02d0 0x7fcb180cf0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:52284/0 (socket says 172.21.15.102:52284) 2024-03-12T17:06:07.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.794+0000 7fcb1ca00640 1 -- 172.21.15.102:0/1946560314 learned_addr learned my addr 172.21.15.102:0/1946560314 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-03-12T17:06:07.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.794+0000 7fcb1d400640 1 --2- 172.21.15.102:0/1946560314 >> v2:172.21.15.52:3300/0 conn(0x7fcb1815c780 0x7fcb180cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:07.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.794+0000 7fcb17e00640 1 -- 172.21.15.102:0/1946560314 <== mon.1 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 122994432 0 0) 0x7fcb180cf9a0 con 0x7fcb180d02d0 2024-03-12T17:06:07.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.794+0000 7fcb17e00640 1 -- 172.21.15.102:0/1946560314 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb00003520 con 0x7fcb180d02d0 2024-03-12T17:06:07.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.795+0000 7fcb17e00640 1 -- 172.21.15.102:0/1946560314 <== mon.1 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1274082765 0 0) 0x7fcb00003520 con 0x7fcb180d02d0 2024-03-12T17:06:07.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.795+0000 7fcb17e00640 1 -- 172.21.15.102:0/1946560314 >> v2:172.21.15.52:3300/0 conn(0x7fcb1815c780 msgr2=0x7fcb180cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:07.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.795+0000 7fcb17e00640 1 --2- 172.21.15.102:0/1946560314 >> v2:172.21.15.52:3300/0 conn(0x7fcb1815c780 0x7fcb180cdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:07.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.795+0000 7fcb17e00640 1 -- 172.21.15.102:0/1946560314 >> v1:172.21.15.52:6789/0 conn(0x7fcb180d0a40 legacy=0x7fcb180b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:07.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.795+0000 7fcb17e00640 1 -- 172.21.15.102:0/1946560314 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb180cd3d0 con 0x7fcb180d02d0 2024-03-12T17:06:07.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.795+0000 7fcb17e00640 1 -- 172.21.15.102:0/1946560314 <== mon.1 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7fcb04001670 con 0x7fcb180d02d0 2024-03-12T17:06:07.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.795+0000 7fcb17e00640 1 -- 172.21.15.102:0/1946560314 >> v1:172.21.15.102:6789/0 conn(0x7fcb180d02d0 legacy=0x7fcb180cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:07.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.795+0000 7fcb17e00640 1 --2- 172.21.15.102:0/1946560314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb00003970 0x7fcb00003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:07.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.795+0000 7fcb17e00640 1 --2- 172.21.15.102:0/1946560314 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcb000047c0 0x7fcb00004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:07.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.795+0000 7fcb17e00640 1 --2- 172.21.15.102:0/1946560314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb00005180 0x7fcb0000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:07.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.795+0000 7fcb17e00640 1 -- 172.21.15.102:0/1946560314 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb180cfc80 con 0x7fcb00005180 2024-03-12T17:06:07.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.795+0000 7fcb17e00640 1 -- 172.21.15.102:0/1946560314 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0000c120 con 0x7fcb00003970 2024-03-12T17:06:07.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.795+0000 7fcb17e00640 1 -- 172.21.15.102:0/1946560314 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0000c2e0 con 0x7fcb000047c0 2024-03-12T17:06:07.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.795+0000 7fcb1ca00640 1 --2- 172.21.15.102:0/1946560314 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcb000047c0 0x7fcb00004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:07.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.795+0000 7fcb1d400640 1 --2- 172.21.15.102:0/1946560314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb00003970 0x7fcb00003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:07.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.795+0000 7fcb1de00640 1 --2- 172.21.15.102:0/1946560314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb00005180 0x7fcb0000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:07.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.795+0000 7fcb1d400640 1 -- 172.21.15.102:0/1946560314 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcb000047c0 msgr2=0x7fcb00004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:07.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.795+0000 7fcb1d400640 1 --2- 172.21.15.102:0/1946560314 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcb000047c0 0x7fcb00004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:07.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.795+0000 7fcb1d400640 1 -- 172.21.15.102:0/1946560314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb00005180 msgr2=0x7fcb0000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:07.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.795+0000 7fcb1d400640 1 --2- 172.21.15.102:0/1946560314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb00005180 0x7fcb0000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:07.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.795+0000 7fcb1d400640 1 -- 172.21.15.102:0/1946560314 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb00003520 con 0x7fcb00003970 2024-03-12T17:06:07.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.795+0000 7fcb1de00640 1 -- 172.21.15.102:0/1946560314 reap_dead start 2024-03-12T17:06:07.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.795+0000 7fcb1ca00640 1 --2- 172.21.15.102:0/1946560314 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcb000047c0 0x7fcb00004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T17:06:07.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.796+0000 7fcb1d400640 1 --2- 172.21.15.102:0/1946560314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb00003970 0x7fcb00003d90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c009ac0 tx=0x7fcb000057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5be9ba92ac0a8c5f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:07.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.797+0000 7fcb17e00640 1 -- 172.21.15.102:0/1946560314 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c02c6d0 con 0x7fcb00003970 2024-03-12T17:06:07.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.797+0000 7fcb17e00640 1 -- 172.21.15.102:0/1946560314 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb0c02c830 con 0x7fcb00003970 2024-03-12T17:06:07.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.797+0000 7fcb17e00640 1 -- 172.21.15.102:0/1946560314 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c02cbc0 con 0x7fcb00003970 2024-03-12T17:06:07.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.797+0000 7fcb1e800640 1 -- 172.21.15.102:0/1946560314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb00003970 msgr2=0x7fcb00003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:07.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.797+0000 7fcb1e800640 1 --2- 172.21.15.102:0/1946560314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb00003970 0x7fcb00003d90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c009ac0 tx=0x7fcb000057f0 comp rx=0 tx=0).stop 2024-03-12T17:06:07.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.797+0000 7fcb1e800640 1 -- 172.21.15.102:0/1946560314 shutdown_connections 2024-03-12T17:06:07.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.797+0000 7fcb1e800640 1 --2- 172.21.15.102:0/1946560314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb00003970 0x7fcb00003d90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:07.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.797+0000 7fcb1e800640 1 -- 172.21.15.102:0/1946560314 >> 172.21.15.102:0/1946560314 conn(0x7fcb18089fd0 msgr2=0x7fcb180d2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:07.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.798+0000 7fcb1e800640 1 -- 172.21.15.102:0/1946560314 shutdown_connections 2024-03-12T17:06:07.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.798+0000 7fcb1e800640 1 -- 172.21.15.102:0/1946560314 wait complete. 2024-03-12T17:06:07.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.799+0000 7fcb1e800640 1 Processor -- start 2024-03-12T17:06:07.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.799+0000 7fcb1e800640 1 -- start start 2024-03-12T17:06:07.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.800+0000 7fcb1e800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcb00003970 0x7fcb18157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:07.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.800+0000 7fcb1e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb180c8d40 0x7fcb180b5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:07.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.800+0000 7fcb1e800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb180b64f0 0x7fcb180ca980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:07.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.800+0000 7fcb1e800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb180b75a0 con 0x7fcb180b64f0 2024-03-12T17:06:07.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.800+0000 7fcb1e800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb180b6990 con 0x7fcb180c8d40 2024-03-12T17:06:07.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.800+0000 7fcb1e800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb180b6c70 con 0x7fcb00003970 2024-03-12T17:06:07.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.800+0000 7fcb1ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb180c8d40 0x7fcb180b5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:07.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.800+0000 7fcb1ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb180c8d40 0x7fcb180b5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53034/0 (socket says 172.21.15.102:53034) 2024-03-12T17:06:07.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.800+0000 7fcb1ca00640 1 -- 172.21.15.102:0/3695442278 learned_addr learned my addr 172.21.15.102:0/3695442278 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-03-12T17:06:07.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.800+0000 7fcb1d400640 1 --2- 172.21.15.102:0/3695442278 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcb00003970 0x7fcb18157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:07.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.801+0000 7fcb1de00640 1 --2- 172.21.15.102:0/3695442278 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb180b64f0 0x7fcb180ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:07.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.801+0000 7fcb1ca00640 1 -- 172.21.15.102:0/3695442278 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcb00003970 msgr2=0x7fcb18157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:07.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.801+0000 7fcb1ca00640 1 --2- 172.21.15.102:0/3695442278 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcb00003970 0x7fcb18157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:07.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.801+0000 7fcb1ca00640 1 -- 172.21.15.102:0/3695442278 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb180b64f0 msgr2=0x7fcb180ca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:07.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.801+0000 7fcb1ca00640 1 --2- 172.21.15.102:0/3695442278 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb180b64f0 0x7fcb180ca980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:07.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.801+0000 7fcb1ca00640 1 -- 172.21.15.102:0/3695442278 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb180b6f50 con 0x7fcb180c8d40 2024-03-12T17:06:07.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.801+0000 7fcb1ca00640 1 --2- 172.21.15.102:0/3695442278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb180c8d40 0x7fcb180b5fb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcb04007cb0 tx=0x7fcb0400f5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:07.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.801+0000 7fcb15600640 1 -- 172.21.15.102:0/3695442278 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0400fc00 con 0x7fcb180c8d40 2024-03-12T17:06:07.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.801+0000 7fcb15600640 1 -- 172.21.15.102:0/3695442278 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb0400fd60 con 0x7fcb180c8d40 2024-03-12T17:06:07.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.801+0000 7fcb15600640 1 -- 172.21.15.102:0/3695442278 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb040135f0 con 0x7fcb180c8d40 2024-03-12T17:06:07.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.801+0000 7fcb1e800640 1 -- 172.21.15.102:0/3695442278 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb180b7230 con 0x7fcb180c8d40 2024-03-12T17:06:07.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.802+0000 7fcb1e800640 1 -- 172.21.15.102:0/3695442278 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb180cb0a0 con 0x7fcb180c8d40 2024-03-12T17:06:07.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.802+0000 7fcb15600640 1 -- 172.21.15.102:0/3695442278 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fcb04013a50 con 0x7fcb180c8d40 2024-03-12T17:06:07.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.802+0000 7fcb15600640 1 -- 172.21.15.102:0/3695442278 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fcb0401d460 con 0x7fcb180c8d40 2024-03-12T17:06:07.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.803+0000 7fcb1e800640 1 -- 172.21.15.102:0/3695442278 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb180b7710 con 0x7fcb180c8d40 2024-03-12T17:06:07.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.805+0000 7fcb15600640 1 -- 172.21.15.102:0/3695442278 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fcb04007ce0 con 0x7fcb180c8d40 2024-03-12T17:06:07.811 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-03-12T17:06:07.818+0000 7fb63200d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-12T17:06:07.848 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-03-12T17:06:07.855+0000 7f9048e13040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-12T17:06:07.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.882+0000 7fcb1e800640 1 -- 172.21.15.102:0/3695442278 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c166769b-d6f9-4435-a0ae-203fb1fb8743", "id": 2} v 0) v1 -- 0x7fcb180cb2c0 con 0x7fcb180c8d40 2024-03-12T17:06:07.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.890+0000 7fcb15600640 1 -- 172.21.15.102:0/3695442278 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c166769b-d6f9-4435-a0ae-203fb1fb8743", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcb04013d00 con 0x7fcb180c8d40 2024-03-12T17:06:07.883 INFO:teuthology.orchestra.run.smithi102.stdout:2 2024-03-12T17:06:07.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.891+0000 7fcb1e800640 1 -- 172.21.15.102:0/3695442278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb180c8d40 msgr2=0x7fcb180b5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:07.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.891+0000 7fcb1e800640 1 --2- 172.21.15.102:0/3695442278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb180c8d40 0x7fcb180b5fb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcb04007cb0 tx=0x7fcb0400f5a0 comp rx=0 tx=0).stop 2024-03-12T17:06:07.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.891+0000 7fcb1e800640 1 -- 172.21.15.102:0/3695442278 shutdown_connections 2024-03-12T17:06:07.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.891+0000 7fcb1e800640 1 --2- 172.21.15.102:0/3695442278 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb180b64f0 0x7fcb180ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:07.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.891+0000 7fcb1e800640 1 --2- 172.21.15.102:0/3695442278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb180c8d40 0x7fcb180b5fb0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:07.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.891+0000 7fcb1e800640 1 --2- 172.21.15.102:0/3695442278 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcb00003970 0x7fcb18157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:07.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.891+0000 7fcb1e800640 1 -- 172.21.15.102:0/3695442278 >> 172.21.15.102:0/3695442278 conn(0x7fcb18089fd0 msgr2=0x7fcb180d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:07.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.891+0000 7fcb1e800640 1 -- 172.21.15.102:0/3695442278 shutdown_connections 2024-03-12T17:06:07.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:07.891+0000 7fcb1e800640 1 -- 172.21.15.102:0/3695442278 wait complete. 2024-03-12T17:06:07.896 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-03-12T17:06:07.903+0000 7fb63200d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-12T17:06:07.899 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new 179df347-56fa-409a-a308-ba2a64d598be 3 2024-03-12T17:06:07.933 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-03-12T17:06:07.940+0000 7f9048e13040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-12T17:06:07.953 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-03-12T17:06:07.959+0000 7fb63200d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-12T17:06:07.992 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-03-12T17:06:07.998+0000 7f9048e13040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-12T17:06:08.018 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-03-12T17:06:08.024+0000 7fb63200d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-12T17:06:08.053 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-03-12T17:06:08.060+0000 7f9048e13040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-12T17:06:08.075 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-03-12T17:06:08.081+0000 7fb63200d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-12T17:06:08.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.092+0000 7f6efc400640 1 Processor -- start 2024-03-12T17:06:08.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.092+0000 7f6efc400640 1 -- start start 2024-03-12T17:06:08.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.093+0000 7f6efc400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6ef415c980 0x7f6ef40cf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:08.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.093+0000 7f6efc400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6ef40cdd00 0x7f6ef40d1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:08.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.093+0000 7f6efc400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ef40cf730 con 0x7f6ef4161010 2024-03-12T17:06:08.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.093+0000 7f6efc400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ef40cfa10 con 0x7f6ef415c980 2024-03-12T17:06:08.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.093+0000 7f6efc400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef40cfcf0 con 0x7f6ef40cdd00 2024-03-12T17:06:08.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.093+0000 7f6ef9a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6ef415c980 0x7f6ef40cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:08.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.093+0000 7f6ef9a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6ef415c980 0x7f6ef40cf130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51202/0 (socket says 172.21.15.102:51202) 2024-03-12T17:06:08.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.093+0000 7f6ef9a00640 1 -- 172.21.15.102:0/1379138728 learned_addr learned my addr 172.21.15.102:0/1379138728 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-03-12T17:06:08.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.093+0000 7f6efa400640 1 --2- 172.21.15.102:0/1379138728 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6ef40cdd00 0x7f6ef40d1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:08.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.093+0000 7f6ef9000640 1 --1- 172.21.15.102:0/1379138728 >> v1:172.21.15.102:6789/0 conn(0x7f6ef4161010 0x7f6ef40cd5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:39266/0 (socket says 172.21.15.102:39266) 2024-03-12T17:06:08.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.094+0000 7f6ef9a00640 1 -- 172.21.15.102:0/1379138728 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6ef40cdd00 msgr2=0x7f6ef40d1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:08.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.094+0000 7f6ef9a00640 1 --2- 172.21.15.102:0/1379138728 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6ef40cdd00 0x7f6ef40d1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:08.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.094+0000 7f6ef9a00640 1 -- 172.21.15.102:0/1379138728 >> v1:172.21.15.102:6789/0 conn(0x7f6ef4161010 legacy=0x7f6ef40cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:08.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.094+0000 7f6ef9a00640 1 -- 172.21.15.102:0/1379138728 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef40cffd0 con 0x7f6ef415c980 2024-03-12T17:06:08.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.095+0000 7f6ef9a00640 1 --2- 172.21.15.102:0/1379138728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ef415c980 0x7f6ef40cf130 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8009870 tx=0x7f6ee802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7ac40093c932f6f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:08.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.095+0000 7f6ef3e00640 1 -- 172.21.15.102:0/1379138728 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8030840 con 0x7f6ef415c980 2024-03-12T17:06:08.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.096+0000 7f6ef3e00640 1 -- 172.21.15.102:0/1379138728 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee80309a0 con 0x7f6ef415c980 2024-03-12T17:06:08.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.096+0000 7f6ef3e00640 1 -- 172.21.15.102:0/1379138728 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8030d30 con 0x7f6ef415c980 2024-03-12T17:06:08.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.096+0000 7f6efc400640 1 -- 172.21.15.102:0/1379138728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ef415c980 msgr2=0x7f6ef40cf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:08.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.096+0000 7f6efc400640 1 --2- 172.21.15.102:0/1379138728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ef415c980 0x7f6ef40cf130 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8009870 tx=0x7f6ee802b980 comp rx=0 tx=0).stop 2024-03-12T17:06:08.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.096+0000 7f6efc400640 1 -- 172.21.15.102:0/1379138728 shutdown_connections 2024-03-12T17:06:08.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.096+0000 7f6efc400640 1 --2- 172.21.15.102:0/1379138728 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6ef40cdd00 0x7f6ef40d1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:08.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.096+0000 7f6efc400640 1 --2- 172.21.15.102:0/1379138728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ef415c980 0x7f6ef40cf130 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:08.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.096+0000 7f6efc400640 1 -- 172.21.15.102:0/1379138728 >> 172.21.15.102:0/1379138728 conn(0x7f6ef40713c0 msgr2=0x7f6ef40717d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:08.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.096+0000 7f6efc400640 1 -- 172.21.15.102:0/1379138728 shutdown_connections 2024-03-12T17:06:08.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.096+0000 7f6efc400640 1 -- 172.21.15.102:0/1379138728 wait complete. 2024-03-12T17:06:08.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.098+0000 7f6efc400640 1 Processor -- start 2024-03-12T17:06:08.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.098+0000 7f6efc400640 1 -- start start 2024-03-12T17:06:08.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.099+0000 7f6efc400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6ef40cdd00 0x7f6ef4151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:08.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.099+0000 7f6efc400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ef415c980 0x7f6ef40b78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:08.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.099+0000 7f6efc400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ef4161010 0x7f6ef40b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:08.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.099+0000 7f6efc400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef41f76e0 con 0x7f6ef4161010 2024-03-12T17:06:08.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.099+0000 7f6efc400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef4152360 con 0x7f6ef415c980 2024-03-12T17:06:08.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.099+0000 7f6efc400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef41f6a60 con 0x7f6ef40cdd00 2024-03-12T17:06:08.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.099+0000 7f6ef9000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ef415c980 0x7f6ef40b78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:08.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.099+0000 7f6ef9a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6ef40cdd00 0x7f6ef4151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:08.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.099+0000 7f6ef9000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ef415c980 0x7f6ef40b78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51210/0 (socket says 172.21.15.102:51210) 2024-03-12T17:06:08.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.099+0000 7f6ef9000640 1 -- 172.21.15.102:0/2981687337 learned_addr learned my addr 172.21.15.102:0/2981687337 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-03-12T17:06:08.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.100+0000 7f6efa400640 1 --2- 172.21.15.102:0/2981687337 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ef4161010 0x7f6ef40b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:08.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.100+0000 7f6ef9000640 1 -- 172.21.15.102:0/2981687337 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6ef40cdd00 msgr2=0x7f6ef4151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:08.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.100+0000 7f6ef9000640 1 --2- 172.21.15.102:0/2981687337 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6ef40cdd00 0x7f6ef4151e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:08.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.100+0000 7f6ef9000640 1 -- 172.21.15.102:0/2981687337 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ef4161010 msgr2=0x7f6ef40b7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:08.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.100+0000 7f6ef9000640 1 --2- 172.21.15.102:0/2981687337 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ef4161010 0x7f6ef40b7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:08.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.100+0000 7f6ef9000640 1 -- 172.21.15.102:0/2981687337 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee80094d0 con 0x7f6ef415c980 2024-03-12T17:06:08.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.100+0000 7f6ef9000640 1 --2- 172.21.15.102:0/2981687337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ef415c980 0x7f6ef40b78c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000c3e0 tx=0x7f6ee000c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:08.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.100+0000 7f6ef1600640 1 -- 172.21.15.102:0/2981687337 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee00074e0 con 0x7f6ef415c980 2024-03-12T17:06:08.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.100+0000 7f6ef1600640 1 -- 172.21.15.102:0/2981687337 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee0007640 con 0x7f6ef415c980 2024-03-12T17:06:08.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.100+0000 7f6ef1600640 1 -- 172.21.15.102:0/2981687337 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee00049e0 con 0x7f6ef415c980 2024-03-12T17:06:08.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.100+0000 7f6efc400640 1 -- 172.21.15.102:0/2981687337 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef41f6d40 con 0x7f6ef415c980 2024-03-12T17:06:08.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.100+0000 7f6efc400640 1 -- 172.21.15.102:0/2981687337 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef41f7300 con 0x7f6ef415c980 2024-03-12T17:06:08.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.100+0000 7f6ef1600640 1 -- 172.21.15.102:0/2981687337 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6ee0013400 con 0x7f6ef415c980 2024-03-12T17:06:08.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.101+0000 7f6ef1600640 1 -- 172.21.15.102:0/2981687337 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f6ee0013b40 con 0x7f6ef415c980 2024-03-12T17:06:08.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.101+0000 7f6efc400640 1 -- 172.21.15.102:0/2981687337 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ef4073390 con 0x7f6ef415c980 2024-03-12T17:06:08.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.104+0000 7f6ef1600640 1 -- 172.21.15.102:0/2981687337 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f6ee0013d40 con 0x7f6ef415c980 2024-03-12T17:06:08.112 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-03-12T17:06:08.118+0000 7f9048e13040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-12T17:06:08.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.182+0000 7f6efc400640 1 -- 172.21.15.102:0/2981687337 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd new", "uuid": "179df347-56fa-409a-a308-ba2a64d598be", "id": 3} v 0) v1 -- 0x7f6ef40d0550 con 0x7f6ef415c980 2024-03-12T17:06:08.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.190+0000 7f6ef1600640 1 -- 172.21.15.102:0/2981687337 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "179df347-56fa-409a-a308-ba2a64d598be", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6ee0013780 con 0x7f6ef415c980 2024-03-12T17:06:08.183 INFO:teuthology.orchestra.run.smithi102.stdout:3 2024-03-12T17:06:08.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.191+0000 7f6efc400640 1 -- 172.21.15.102:0/2981687337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ef415c980 msgr2=0x7f6ef40b78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:08.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.191+0000 7f6efc400640 1 --2- 172.21.15.102:0/2981687337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ef415c980 0x7f6ef40b78c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000c3e0 tx=0x7f6ee000c8a0 comp rx=0 tx=0).stop 2024-03-12T17:06:08.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.191+0000 7f6efc400640 1 -- 172.21.15.102:0/2981687337 shutdown_connections 2024-03-12T17:06:08.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.191+0000 7f6efc400640 1 --2- 172.21.15.102:0/2981687337 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ef4161010 0x7f6ef40b7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:08.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.191+0000 7f6efc400640 1 --2- 172.21.15.102:0/2981687337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ef415c980 0x7f6ef40b78c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:08.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.191+0000 7f6efc400640 1 --2- 172.21.15.102:0/2981687337 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6ef40cdd00 0x7f6ef4151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:08.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.191+0000 7f6efc400640 1 -- 172.21.15.102:0/2981687337 >> 172.21.15.102:0/2981687337 conn(0x7f6ef40713c0 msgr2=0x7f6ef40d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:08.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.191+0000 7f6efc400640 1 -- 172.21.15.102:0/2981687337 shutdown_connections 2024-03-12T17:06:08.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.191+0000 7f6efc400640 1 -- 172.21.15.102:0/2981687337 wait complete. 2024-03-12T17:06:08.200 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c 4 2024-03-12T17:06:08.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.435+0000 7ff5b8200640 1 Processor -- start 2024-03-12T17:06:08.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.436+0000 7ff5b8200640 1 -- start start 2024-03-12T17:06:08.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.437+0000 7ff5b8200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff5b00d0d40 0x7ff5b00b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:08.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.437+0000 7ff5b8200640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5b00d20c0 con 0x7ff5b00b84b0 2024-03-12T17:06:08.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.437+0000 7ff5b8200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5b00b8940 con 0x7ff5b00cd740 2024-03-12T17:06:08.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.437+0000 7ff5b8200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5b00d14b0 con 0x7ff5b00d0d40 2024-03-12T17:06:08.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.437+0000 7ff5b5800640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7ff5b00cd740 0x7ff5b00cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:39276/0 (socket says 172.21.15.102:39276) 2024-03-12T17:06:08.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.437+0000 7ff5b5800640 1 -- 172.21.15.102:0/1856712908 learned_addr learned my addr 172.21.15.102:0/1856712908 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-03-12T17:06:08.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.438+0000 7ff5b6200640 1 --2- 172.21.15.102:0/1856712908 >> v2:172.21.15.52:3300/0 conn(0x7ff5b00d0d40 0x7ff5b00b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:08.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.438+0000 7ff5afe00640 1 -- 172.21.15.102:0/1856712908 <== mon.1 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1018933358 0 0) 0x7ff5b00b8940 con 0x7ff5b00cd740 2024-03-12T17:06:08.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.438+0000 7ff5afe00640 1 -- 172.21.15.102:0/1856712908 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff598003520 con 0x7ff5b00cd740 2024-03-12T17:06:08.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.438+0000 7ff5afe00640 1 -- 172.21.15.102:0/1856712908 <== mon.1 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2581460449 0 0) 0x7ff598003520 con 0x7ff5b00cd740 2024-03-12T17:06:08.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.438+0000 7ff5afe00640 1 -- 172.21.15.102:0/1856712908 >> v2:172.21.15.52:3300/0 conn(0x7ff5b00d0d40 msgr2=0x7ff5b00b61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:08.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.438+0000 7ff5afe00640 1 --2- 172.21.15.102:0/1856712908 >> v2:172.21.15.52:3300/0 conn(0x7ff5b00d0d40 0x7ff5b00b61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:08.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.438+0000 7ff5afe00640 1 -- 172.21.15.102:0/1856712908 >> v1:172.21.15.52:6789/0 conn(0x7ff5b00b84b0 legacy=0x7ff5b00b5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:08.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.438+0000 7ff5afe00640 1 -- 172.21.15.102:0/1856712908 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b00d1790 con 0x7ff5b00cd740 2024-03-12T17:06:08.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.438+0000 7ff5afe00640 1 -- 172.21.15.102:0/1856712908 <== mon.1 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7ff5a4001670 con 0x7ff5b00cd740 2024-03-12T17:06:08.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.438+0000 7ff5afe00640 1 -- 172.21.15.102:0/1856712908 >> v1:172.21.15.102:6789/0 conn(0x7ff5b00cd740 legacy=0x7ff5b00cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:08.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.438+0000 7ff5afe00640 1 --2- 172.21.15.102:0/1856712908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff598003970 0x7ff598003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:08.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.438+0000 7ff5afe00640 1 --2- 172.21.15.102:0/1856712908 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5980047c0 0x7ff598004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:08.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.438+0000 7ff5afe00640 1 --2- 172.21.15.102:0/1856712908 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff598005180 0x7ff59800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:08.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.438+0000 7ff5afe00640 1 -- 172.21.15.102:0/1856712908 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b00d14b0 con 0x7ff598003970 2024-03-12T17:06:08.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.438+0000 7ff5afe00640 1 -- 172.21.15.102:0/1856712908 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59800c0d0 con 0x7ff5980047c0 2024-03-12T17:06:08.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.438+0000 7ff5afe00640 1 -- 172.21.15.102:0/1856712908 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff59800c290 con 0x7ff598005180 2024-03-12T17:06:08.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.438+0000 7ff5b4e00640 1 --2- 172.21.15.102:0/1856712908 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5980047c0 0x7ff598004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:08.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.439+0000 7ff5b6200640 1 --2- 172.21.15.102:0/1856712908 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff598005180 0x7ff59800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:08.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.439+0000 7ff5b5800640 1 --2- 172.21.15.102:0/1856712908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff598003970 0x7ff598003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:08.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.439+0000 7ff5b4e00640 1 -- 172.21.15.102:0/1856712908 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff598005180 msgr2=0x7ff59800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:08.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.439+0000 7ff5b4e00640 1 --2- 172.21.15.102:0/1856712908 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff598005180 0x7ff59800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:08.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.439+0000 7ff5b4e00640 1 -- 172.21.15.102:0/1856712908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff598003970 msgr2=0x7ff598003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:08.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.439+0000 7ff5b4e00640 1 --2- 172.21.15.102:0/1856712908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff598003970 0x7ff598003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:08.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.439+0000 7ff5b4e00640 1 -- 172.21.15.102:0/1856712908 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff598003520 con 0x7ff5980047c0 2024-03-12T17:06:08.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.439+0000 7ff5b6200640 1 -- 172.21.15.102:0/1856712908 reap_dead start 2024-03-12T17:06:08.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.440+0000 7ff5b4e00640 1 --2- 172.21.15.102:0/1856712908 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5980047c0 0x7ff598004c40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff59c001750 tx=0x7ff5980057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f29bd50edf160e54 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:08.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.440+0000 7ff5afe00640 1 -- 172.21.15.102:0/1856712908 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c0052d0 con 0x7ff5980047c0 2024-03-12T17:06:08.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.440+0000 7ff5afe00640 1 -- 172.21.15.102:0/1856712908 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59c02e050 con 0x7ff5980047c0 2024-03-12T17:06:08.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.440+0000 7ff5afe00640 1 -- 172.21.15.102:0/1856712908 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c003830 con 0x7ff5980047c0 2024-03-12T17:06:08.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.441+0000 7ff5b8200640 1 -- 172.21.15.102:0/1856712908 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5980047c0 msgr2=0x7ff598004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:08.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.441+0000 7ff5b8200640 1 --2- 172.21.15.102:0/1856712908 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5980047c0 0x7ff598004c40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff59c001750 tx=0x7ff5980057f0 comp rx=0 tx=0).stop 2024-03-12T17:06:08.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.441+0000 7ff5b8200640 1 -- 172.21.15.102:0/1856712908 shutdown_connections 2024-03-12T17:06:08.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.441+0000 7ff5b8200640 1 --2- 172.21.15.102:0/1856712908 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5980047c0 0x7ff598004c40 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:08.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.441+0000 7ff5b8200640 1 -- 172.21.15.102:0/1856712908 >> 172.21.15.102:0/1856712908 conn(0x7ff5b00cf5d0 msgr2=0x7ff5b00cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:08.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.441+0000 7ff5b8200640 1 -- 172.21.15.102:0/1856712908 shutdown_connections 2024-03-12T17:06:08.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.441+0000 7ff5b8200640 1 -- 172.21.15.102:0/1856712908 wait complete. 2024-03-12T17:06:08.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.444+0000 7ff5b8200640 1 Processor -- start 2024-03-12T17:06:08.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.444+0000 7ff5b8200640 1 -- start start 2024-03-12T17:06:08.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.445+0000 7ff5b8200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff5980047c0 0x7ff5b0159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:08.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.445+0000 7ff5b8200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5b00c68d0 0x7ff5b00cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:08.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.446+0000 7ff5b8200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5b00c7a00 0x7ff5b00c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:08.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.446+0000 7ff5b8200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b00c7730 con 0x7ff5b00c68d0 2024-03-12T17:06:08.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.446+0000 7ff5b8200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b00c83c0 con 0x7ff5b00c7a00 2024-03-12T17:06:08.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.446+0000 7ff5b8200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b015a430 con 0x7ff5980047c0 2024-03-12T17:06:08.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.446+0000 7ff5b6200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5b00c7a00 0x7ff5b00c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:08.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.446+0000 7ff5b5800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff5980047c0 0x7ff5b0159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:08.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.446+0000 7ff5b6200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5b00c7a00 0x7ff5b00c7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51238/0 (socket says 172.21.15.102:51238) 2024-03-12T17:06:08.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.446+0000 7ff5b6200640 1 -- 172.21.15.102:0/1731303727 learned_addr learned my addr 172.21.15.102:0/1731303727 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-03-12T17:06:08.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.446+0000 7ff5b4e00640 1 --2- 172.21.15.102:0/1731303727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5b00c68d0 0x7ff5b00cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:08.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.446+0000 7ff5b5800640 1 -- 172.21.15.102:0/1731303727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5b00c7a00 msgr2=0x7ff5b00c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:08.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.446+0000 7ff5b5800640 1 --2- 172.21.15.102:0/1731303727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5b00c7a00 0x7ff5b00c7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:08.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.446+0000 7ff5b5800640 1 -- 172.21.15.102:0/1731303727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5b00c68d0 msgr2=0x7ff5b00cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:08.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.446+0000 7ff5b5800640 1 --2- 172.21.15.102:0/1731303727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5b00c68d0 0x7ff5b00cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:08.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.446+0000 7ff5b5800640 1 -- 172.21.15.102:0/1731303727 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b00cc330 con 0x7ff5980047c0 2024-03-12T17:06:08.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.446+0000 7ff5b6200640 1 --2- 172.21.15.102:0/1731303727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5b00c7a00 0x7ff5b00c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T17:06:08.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.447+0000 7ff5b5800640 1 --2- 172.21.15.102:0/1731303727 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff5980047c0 0x7ff5b0159ef0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4004ef0 tx=0x7ff5a4007430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:08.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.447+0000 7ff5ad600640 1 -- 172.21.15.102:0/1731303727 <== mon.2 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a400f7b0 con 0x7ff5980047c0 2024-03-12T17:06:08.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.447+0000 7ff5ad600640 1 -- 172.21.15.102:0/1731303727 <== mon.2 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5a400f910 con 0x7ff5980047c0 2024-03-12T17:06:08.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.447+0000 7ff5b8200640 1 -- 172.21.15.102:0/1731303727 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5b00c6e90 con 0x7ff5980047c0 2024-03-12T17:06:08.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.447+0000 7ff5ad600640 1 -- 172.21.15.102:0/1731303727 <== mon.2 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a400fc80 con 0x7ff5980047c0 2024-03-12T17:06:08.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.447+0000 7ff5b8200640 1 -- 172.21.15.102:0/1731303727 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5b00c7450 con 0x7ff5980047c0 2024-03-12T17:06:08.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.447+0000 7ff5ad600640 1 -- 172.21.15.102:0/1731303727 <== mon.2 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff5a401d760 con 0x7ff5980047c0 2024-03-12T17:06:08.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.447+0000 7ff5ad600640 1 -- 172.21.15.102:0/1731303727 <== mon.2 v2:172.21.15.102:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7ff5a4013a90 con 0x7ff5980047c0 2024-03-12T17:06:08.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.448+0000 7ff5b8200640 1 -- 172.21.15.102:0/1731303727 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5b0073390 con 0x7ff5980047c0 2024-03-12T17:06:08.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.452+0000 7ff5ad600640 1 -- 172.21.15.102:0/1731303727 <== mon.2 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7ff5a4013d20 con 0x7ff5980047c0 2024-03-12T17:06:08.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.529+0000 7ff5b8200640 1 -- 172.21.15.102:0/1731303727 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c", "id": 4} v 0) v1 -- 0x7ff5b00b20b0 con 0x7ff5980047c0 2024-03-12T17:06:08.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.536+0000 7ff5ad600640 1 -- 172.21.15.102:0/1731303727 <== mon.2 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff5a4013580 con 0x7ff5980047c0 2024-03-12T17:06:08.529 INFO:teuthology.orchestra.run.smithi102.stdout:4 2024-03-12T17:06:08.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.537+0000 7ff5b8200640 1 -- 172.21.15.102:0/1731303727 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff5980047c0 msgr2=0x7ff5b0159ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:08.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.537+0000 7ff5b8200640 1 --2- 172.21.15.102:0/1731303727 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff5980047c0 0x7ff5b0159ef0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4004ef0 tx=0x7ff5a4007430 comp rx=0 tx=0).stop 2024-03-12T17:06:08.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.537+0000 7ff5b8200640 1 -- 172.21.15.102:0/1731303727 shutdown_connections 2024-03-12T17:06:08.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.537+0000 7ff5b8200640 1 --2- 172.21.15.102:0/1731303727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5b00c7a00 0x7ff5b00c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:08.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.537+0000 7ff5b8200640 1 --2- 172.21.15.102:0/1731303727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5b00c68d0 0x7ff5b00cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:08.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.537+0000 7ff5b8200640 1 --2- 172.21.15.102:0/1731303727 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff5980047c0 0x7ff5b0159ef0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:08.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.537+0000 7ff5b8200640 1 -- 172.21.15.102:0/1731303727 >> 172.21.15.102:0/1731303727 conn(0x7ff5b00cf5d0 msgr2=0x7ff5b00b7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:08.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.537+0000 7ff5b8200640 1 -- 172.21.15.102:0/1731303727 shutdown_connections 2024-03-12T17:06:08.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.537+0000 7ff5b8200640 1 -- 172.21.15.102:0/1731303727 wait complete. 2024-03-12T17:06:08.545 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new a0270815-1af1-4a1a-b74d-3e9241179739 5 2024-03-12T17:06:08.551 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-03-12T17:06:08.557+0000 7fb63200d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-12T17:06:08.588 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-03-12T17:06:08.594+0000 7f9048e13040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-12T17:06:08.718 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-03-12T17:06:08.724+0000 7fb63200d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-12T17:06:08.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.735+0000 7fa4d7200640 1 Processor -- start 2024-03-12T17:06:08.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.735+0000 7fa4d7200640 1 -- start start 2024-03-12T17:06:08.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.737+0000 7fa4d7200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa4d8159820 0x7fa4d80afa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:08.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.737+0000 7fa4d7200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa4d80ad900 0x7fa4d80b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:08.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.737+0000 7fa4d7200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4d80aea90 con 0x7fa4d8159c00 2024-03-12T17:06:08.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.737+0000 7fa4d7200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4d80aecc0 con 0x7fa4d8159820 2024-03-12T17:06:08.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.737+0000 7fa4d7200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d80aefa0 con 0x7fa4d80ad900 2024-03-12T17:06:08.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.737+0000 7fa4d6800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa4d80ad900 0x7fa4d80b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:08.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.737+0000 7fa4d5e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa4d8159820 0x7fa4d80afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:08.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.737+0000 7fa4d6800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa4d80ad900 0x7fa4d80b06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:38436/0 (socket says 172.21.15.102:38436) 2024-03-12T17:06:08.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.737+0000 7fa4d6800640 1 -- 172.21.15.102:0/1650147828 learned_addr learned my addr 172.21.15.102:0/1650147828 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-03-12T17:06:08.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.738+0000 7fa4d5e00640 1 -- 172.21.15.102:0/1650147828 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa4d80ad900 msgr2=0x7fa4d80b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:08.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.738+0000 7fa4d5e00640 1 --2- 172.21.15.102:0/1650147828 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa4d80ad900 0x7fa4d80b06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:08.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.738+0000 7fa4d5e00640 1 -- 172.21.15.102:0/1650147828 >> v1:172.21.15.102:6789/0 conn(0x7fa4d8159c00 legacy=0x7fa4d80affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:08.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.738+0000 7fa4d5e00640 1 -- 172.21.15.102:0/1650147828 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d80b0c10 con 0x7fa4d8159820 2024-03-12T17:06:08.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.738+0000 7fa4d6800640 1 --2- 172.21.15.102:0/1650147828 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa4d80ad900 0x7fa4d80b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T17:06:08.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.739+0000 7fa4d5e00640 1 --2- 172.21.15.102:0/1650147828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4d8159820 0x7fa4d80afa80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa4c80097a0 tx=0x7fa4c802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=148d5ec2485dd30 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:08.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.739+0000 7fa4d4a00640 1 -- 172.21.15.102:0/1650147828 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8031780 con 0x7fa4d8159820 2024-03-12T17:06:08.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.739+0000 7fa4d4a00640 1 -- 172.21.15.102:0/1650147828 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c80318e0 con 0x7fa4d8159820 2024-03-12T17:06:08.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.740+0000 7fa4d4a00640 1 -- 172.21.15.102:0/1650147828 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8031c70 con 0x7fa4d8159820 2024-03-12T17:06:08.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.740+0000 7fa4d7200640 1 -- 172.21.15.102:0/1650147828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4d8159820 msgr2=0x7fa4d80afa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:08.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.740+0000 7fa4d7200640 1 --2- 172.21.15.102:0/1650147828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4d8159820 0x7fa4d80afa80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa4c80097a0 tx=0x7fa4c802b820 comp rx=0 tx=0).stop 2024-03-12T17:06:08.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.740+0000 7fa4d7200640 1 -- 172.21.15.102:0/1650147828 shutdown_connections 2024-03-12T17:06:08.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.740+0000 7fa4d7200640 1 --2- 172.21.15.102:0/1650147828 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa4d80ad900 0x7fa4d80b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:08.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.740+0000 7fa4d7200640 1 --2- 172.21.15.102:0/1650147828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4d8159820 0x7fa4d80afa80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:08.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.740+0000 7fa4d7200640 1 -- 172.21.15.102:0/1650147828 >> 172.21.15.102:0/1650147828 conn(0x7fa4d8098a40 msgr2=0x7fa4d80b47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:08.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.740+0000 7fa4d7200640 1 -- 172.21.15.102:0/1650147828 shutdown_connections 2024-03-12T17:06:08.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.740+0000 7fa4d7200640 1 -- 172.21.15.102:0/1650147828 wait complete. 2024-03-12T17:06:08.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.742+0000 7fa4d7200640 1 Processor -- start 2024-03-12T17:06:08.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.742+0000 7fa4d7200640 1 -- start start 2024-03-12T17:06:08.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.743+0000 7fa4d7200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4d8159820 0x7fa4d8159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:08.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.743+0000 7fa4d7200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa4d8141090 0x7fa4d8148970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:08.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.743+0000 7fa4d7200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4d813fd60 0x7fa4d8144dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:08.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.743+0000 7fa4d7200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d81472f0 con 0x7fa4d813fd60 2024-03-12T17:06:08.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.743+0000 7fa4d7200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d815a5d0 con 0x7fa4d8159820 2024-03-12T17:06:08.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.743+0000 7fa4d7200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d815a8b0 con 0x7fa4d8141090 2024-03-12T17:06:08.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.743+0000 7fa4d5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4d8159820 0x7fa4d8159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:08.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.743+0000 7fa4d5400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa4d8141090 0x7fa4d8148970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:08.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.743+0000 7fa4d5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4d8159820 0x7fa4d8159ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51254/0 (socket says 172.21.15.102:51254) 2024-03-12T17:06:08.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.743+0000 7fa4d5e00640 1 -- 172.21.15.102:0/4104473564 learned_addr learned my addr 172.21.15.102:0/4104473564 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-03-12T17:06:08.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.744+0000 7fa4d6800640 1 --2- 172.21.15.102:0/4104473564 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4d813fd60 0x7fa4d8144dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:08.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.744+0000 7fa4d5e00640 1 -- 172.21.15.102:0/4104473564 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa4d8141090 msgr2=0x7fa4d8148970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:08.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.744+0000 7fa4d5e00640 1 --2- 172.21.15.102:0/4104473564 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa4d8141090 0x7fa4d8148970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:08.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.744+0000 7fa4d5e00640 1 -- 172.21.15.102:0/4104473564 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4d813fd60 msgr2=0x7fa4d8144dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:08.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.744+0000 7fa4d5e00640 1 --2- 172.21.15.102:0/4104473564 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4d813fd60 0x7fa4d8144dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:08.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.744+0000 7fa4d5e00640 1 -- 172.21.15.102:0/4104473564 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d0009b30 con 0x7fa4d8159820 2024-03-12T17:06:08.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.744+0000 7fa4d5400640 1 --2- 172.21.15.102:0/4104473564 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa4d8141090 0x7fa4d8148970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T17:06:08.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.744+0000 7fa4d6800640 1 --2- 172.21.15.102:0/4104473564 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4d813fd60 0x7fa4d8144dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:08.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.744+0000 7fa4d5e00640 1 --2- 172.21.15.102:0/4104473564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4d8159820 0x7fa4d8159ff0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8034040 tx=0x7fa4c80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:08.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.744+0000 7fa4ce000640 1 -- 172.21.15.102:0/4104473564 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8032820 con 0x7fa4d8159820 2024-03-12T17:06:08.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.744+0000 7fa4ce000640 1 -- 172.21.15.102:0/4104473564 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c8032980 con 0x7fa4d8159820 2024-03-12T17:06:08.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.744+0000 7fa4ce000640 1 -- 172.21.15.102:0/4104473564 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8032d30 con 0x7fa4d8159820 2024-03-12T17:06:08.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.744+0000 7fa4d7200640 1 -- 172.21.15.102:0/4104473564 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4c8009400 con 0x7fa4d8159820 2024-03-12T17:06:08.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.744+0000 7fa4d7200640 1 -- 172.21.15.102:0/4104473564 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4d815ae70 con 0x7fa4d8159820 2024-03-12T17:06:08.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.745+0000 7fa4ce000640 1 -- 172.21.15.102:0/4104473564 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa4c804e440 con 0x7fa4d8159820 2024-03-12T17:06:08.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.745+0000 7fa4ce000640 1 -- 172.21.15.102:0/4104473564 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fa4c804ec70 con 0x7fa4d8159820 2024-03-12T17:06:08.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.745+0000 7fa4d7200640 1 -- 172.21.15.102:0/4104473564 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4d80b1340 con 0x7fa4d8159820 2024-03-12T17:06:08.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.748+0000 7fa4ce000640 1 -- 172.21.15.102:0/4104473564 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fa4c804e6f0 con 0x7fa4d8159820 2024-03-12T17:06:08.755 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-03-12T17:06:08.761+0000 7f9048e13040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-12T17:06:08.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.825+0000 7fa4d7200640 1 -- 172.21.15.102:0/4104473564 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a0270815-1af1-4a1a-b74d-3e9241179739", "id": 5} v 0) v1 -- 0x7fa4d804e030 con 0x7fa4d8159820 2024-03-12T17:06:08.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.832+0000 7fa4ce000640 1 -- 172.21.15.102:0/4104473564 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a0270815-1af1-4a1a-b74d-3e9241179739", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa4c804e890 con 0x7fa4d8159820 2024-03-12T17:06:08.825 INFO:teuthology.orchestra.run.smithi102.stdout:5 2024-03-12T17:06:08.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.833+0000 7fa4d7200640 1 -- 172.21.15.102:0/4104473564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4d8159820 msgr2=0x7fa4d8159ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:08.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.833+0000 7fa4d7200640 1 --2- 172.21.15.102:0/4104473564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4d8159820 0x7fa4d8159ff0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8034040 tx=0x7fa4c80046c0 comp rx=0 tx=0).stop 2024-03-12T17:06:08.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.833+0000 7fa4d7200640 1 -- 172.21.15.102:0/4104473564 shutdown_connections 2024-03-12T17:06:08.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.833+0000 7fa4d7200640 1 --2- 172.21.15.102:0/4104473564 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa4d8141090 0x7fa4d8148970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:08.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.833+0000 7fa4d7200640 1 --2- 172.21.15.102:0/4104473564 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4d813fd60 0x7fa4d8144dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:08.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.833+0000 7fa4d7200640 1 --2- 172.21.15.102:0/4104473564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4d8159820 0x7fa4d8159ff0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:08.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.833+0000 7fa4d7200640 1 -- 172.21.15.102:0/4104473564 >> 172.21.15.102:0/4104473564 conn(0x7fa4d8098a40 msgr2=0x7fa4d80b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:08.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.834+0000 7fa4d7200640 1 -- 172.21.15.102:0/4104473564 shutdown_connections 2024-03-12T17:06:08.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:08.834+0000 7fa4d7200640 1 -- 172.21.15.102:0/4104473564 wait complete. 2024-03-12T17:06:08.843 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new 7992e177-6c68-400b-b7d7-5be8d226812e 6 2024-03-12T17:06:08.849 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-03-12T17:06:08.856+0000 7fb63200d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-12T17:06:08.891 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-03-12T17:06:08.898+0000 7f9048e13040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-12T17:06:08.905 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-03-12T17:06:08.911+0000 7fb63200d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-12T17:06:08.947 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-03-12T17:06:08.954+0000 7f9048e13040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-12T17:06:09.038 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-03-12T17:06:09.044+0000 7fb63200d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-12T17:06:09.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.080+0000 7efd7e400640 1 Processor -- start 2024-03-12T17:06:09.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.080+0000 7efd7e400640 1 -- start start 2024-03-12T17:06:09.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.081+0000 7efd7e400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd780b84b0 0x7efd780d0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:09.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.081+0000 7efd7e400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7efd780b5aa0 0x7efd780b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:09.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.081+0000 7efd7e400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd780d1280 con 0x7efd780cd740 2024-03-12T17:06:09.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.081+0000 7efd7e400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd780d20c0 con 0x7efd780b5aa0 2024-03-12T17:06:09.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.081+0000 7efd7e400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd780b8910 con 0x7efd780b84b0 2024-03-12T17:06:09.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.081+0000 7efd77e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7efd780b5aa0 0x7efd780b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:09.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.082+0000 7efd76a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd780b84b0 0x7efd780d0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:09.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.082+0000 7efd77e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7efd780b5aa0 0x7efd780b5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51260/0 (socket says 172.21.15.102:51260) 2024-03-12T17:06:09.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.082+0000 7efd77e00640 1 -- 172.21.15.102:0/2306337663 learned_addr learned my addr 172.21.15.102:0/2306337663 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-03-12T17:06:09.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.082+0000 7efd76a00640 1 -- 172.21.15.102:0/2306337663 >> v2:172.21.15.102:3300/0 conn(0x7efd780b5aa0 msgr2=0x7efd780b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:09.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.082+0000 7efd76a00640 1 --2- 172.21.15.102:0/2306337663 >> v2:172.21.15.102:3300/0 conn(0x7efd780b5aa0 0x7efd780b5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:09.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.082+0000 7efd76a00640 1 -- 172.21.15.102:0/2306337663 >> v1:172.21.15.102:6789/0 conn(0x7efd780cd740 legacy=0x7efd780cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:09.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.082+0000 7efd76a00640 1 -- 172.21.15.102:0/2306337663 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd780d14b0 con 0x7efd780b84b0 2024-03-12T17:06:09.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.082+0000 7efd77e00640 1 --2- 172.21.15.102:0/2306337663 >> v2:172.21.15.102:3300/0 conn(0x7efd780b5aa0 0x7efd780b5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T17:06:09.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.083+0000 7efd76a00640 1 --2- 172.21.15.102:0/2306337663 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd780b84b0 0x7efd780d0d40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efd640097a0 tx=0x7efd6402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=724d9b4d40e94a65 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:09.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.083+0000 7efd76000640 1 -- 172.21.15.102:0/2306337663 <== mon.2 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6402d040 con 0x7efd780b84b0 2024-03-12T17:06:09.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.083+0000 7efd76000640 1 -- 172.21.15.102:0/2306337663 <== mon.2 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd64031810 con 0x7efd780b84b0 2024-03-12T17:06:09.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.083+0000 7efd76000640 1 -- 172.21.15.102:0/2306337663 <== mon.2 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd64031ba0 con 0x7efd780b84b0 2024-03-12T17:06:09.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.084+0000 7efd7e400640 1 -- 172.21.15.102:0/2306337663 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd780b84b0 msgr2=0x7efd780d0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:09.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.084+0000 7efd7e400640 1 --2- 172.21.15.102:0/2306337663 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd780b84b0 0x7efd780d0d40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efd640097a0 tx=0x7efd6402b8b0 comp rx=0 tx=0).stop 2024-03-12T17:06:09.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.084+0000 7efd7e400640 1 -- 172.21.15.102:0/2306337663 shutdown_connections 2024-03-12T17:06:09.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.084+0000 7efd7e400640 1 --2- 172.21.15.102:0/2306337663 >> v2:172.21.15.102:3300/0 conn(0x7efd780b5aa0 0x7efd780b5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:09.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.084+0000 7efd7e400640 1 --2- 172.21.15.102:0/2306337663 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd780b84b0 0x7efd780d0d40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:09.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.084+0000 7efd7e400640 1 -- 172.21.15.102:0/2306337663 >> 172.21.15.102:0/2306337663 conn(0x7efd780cf5d0 msgr2=0x7efd780cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:09.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.084+0000 7efd7e400640 1 -- 172.21.15.102:0/2306337663 shutdown_connections 2024-03-12T17:06:09.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.084+0000 7efd7e400640 1 -- 172.21.15.102:0/2306337663 wait complete. 2024-03-12T17:06:09.079 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-03-12T17:06:09.085+0000 7f9048e13040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-12T17:06:09.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.086+0000 7efd7e400640 1 Processor -- start 2024-03-12T17:06:09.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.086+0000 7efd7e400640 1 -- start start 2024-03-12T17:06:09.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.087+0000 7efd7e400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd780b5aa0 0x7efd780c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:09.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.087+0000 7efd7e400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd780b84b0 0x7efd78154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:09.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.087+0000 7efd7e400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd780cd740 0x7efd781f2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:09.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.087+0000 7efd7e400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd78154580 con 0x7efd780b84b0 2024-03-12T17:06:09.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.087+0000 7efd7e400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd781f32d0 con 0x7efd780cd740 2024-03-12T17:06:09.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.087+0000 7efd7e400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd781f5cc0 con 0x7efd780b5aa0 2024-03-12T17:06:09.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.087+0000 7efd77400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd780b5aa0 0x7efd780c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:09.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.087+0000 7efd77400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd780b5aa0 0x7efd780c6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:38454/0 (socket says 172.21.15.102:38454) 2024-03-12T17:06:09.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.087+0000 7efd77e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd780cd740 0x7efd781f2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:09.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.087+0000 7efd77400640 1 -- 172.21.15.102:0/2975557085 learned_addr learned my addr 172.21.15.102:0/2975557085 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-03-12T17:06:09.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.087+0000 7efd76a00640 1 --2- 172.21.15.102:0/2975557085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd780b84b0 0x7efd78154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:09.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.088+0000 7efd77e00640 1 -- 172.21.15.102:0/2975557085 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd780b5aa0 msgr2=0x7efd780c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:09.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.088+0000 7efd77e00640 1 --2- 172.21.15.102:0/2975557085 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd780b5aa0 0x7efd780c6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:09.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.088+0000 7efd77e00640 1 -- 172.21.15.102:0/2975557085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd780b84b0 msgr2=0x7efd78154040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:09.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.088+0000 7efd77e00640 1 --2- 172.21.15.102:0/2975557085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd780b84b0 0x7efd78154040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:09.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.088+0000 7efd77e00640 1 -- 172.21.15.102:0/2975557085 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd70009b30 con 0x7efd780cd740 2024-03-12T17:06:09.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.088+0000 7efd77400640 1 --2- 172.21.15.102:0/2975557085 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd780b5aa0 0x7efd780c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T17:06:09.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.088+0000 7efd76a00640 1 --2- 172.21.15.102:0/2975557085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd780b84b0 0x7efd78154040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:09.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.088+0000 7efd77e00640 1 --2- 172.21.15.102:0/2975557085 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd780cd740 0x7efd781f2d90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7efd7000efc0 tx=0x7efd7000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:09.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.088+0000 7efd6f400640 1 -- 172.21.15.102:0/2975557085 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd70018070 con 0x7efd780cd740 2024-03-12T17:06:09.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.088+0000 7efd6f400640 1 -- 172.21.15.102:0/2975557085 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7000f040 con 0x7efd780cd740 2024-03-12T17:06:09.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.088+0000 7efd7e400640 1 -- 172.21.15.102:0/2975557085 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd64009400 con 0x7efd780cd740 2024-03-12T17:06:09.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.088+0000 7efd6f400640 1 -- 172.21.15.102:0/2975557085 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7000cd40 con 0x7efd780cd740 2024-03-12T17:06:09.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.088+0000 7efd7e400640 1 -- 172.21.15.102:0/2975557085 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd781f6280 con 0x7efd780cd740 2024-03-12T17:06:09.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.088+0000 7efd6f400640 1 -- 172.21.15.102:0/2975557085 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7efd70003c90 con 0x7efd780cd740 2024-03-12T17:06:09.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.088+0000 7efd6f400640 1 -- 172.21.15.102:0/2975557085 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7efd70014a70 con 0x7efd780cd740 2024-03-12T17:06:09.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.089+0000 7efd7e400640 1 -- 172.21.15.102:0/2975557085 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd780c82f0 con 0x7efd780cd740 2024-03-12T17:06:09.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.092+0000 7efd6f400640 1 -- 172.21.15.102:0/2975557085 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7efd780c82f0 con 0x7efd780cd740 2024-03-12T17:06:09.154 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-03-12T17:06:09.160+0000 7fb63200d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-12T17:06:09.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.169+0000 7efd7e400640 1 -- 172.21.15.102:0/2975557085 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7992e177-6c68-400b-b7d7-5be8d226812e", "id": 6} v 0) v1 -- 0x7efd78073390 con 0x7efd780cd740 2024-03-12T17:06:09.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.176+0000 7efd6f400640 1 -- 172.21.15.102:0/2975557085 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7992e177-6c68-400b-b7d7-5be8d226812e", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7efd7002b0b0 con 0x7efd780cd740 2024-03-12T17:06:09.169 INFO:teuthology.orchestra.run.smithi102.stdout:6 2024-03-12T17:06:09.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.177+0000 7efd7e400640 1 -- 172.21.15.102:0/2975557085 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd780cd740 msgr2=0x7efd781f2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:09.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.177+0000 7efd7e400640 1 --2- 172.21.15.102:0/2975557085 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd780cd740 0x7efd781f2d90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7efd7000efc0 tx=0x7efd7000c490 comp rx=0 tx=0).stop 2024-03-12T17:06:09.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.177+0000 7efd7e400640 1 -- 172.21.15.102:0/2975557085 shutdown_connections 2024-03-12T17:06:09.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.177+0000 7efd7e400640 1 --2- 172.21.15.102:0/2975557085 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd780cd740 0x7efd781f2d90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:09.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.177+0000 7efd7e400640 1 --2- 172.21.15.102:0/2975557085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd780b84b0 0x7efd78154040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:09.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.177+0000 7efd7e400640 1 --2- 172.21.15.102:0/2975557085 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd780b5aa0 0x7efd780c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:09.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.177+0000 7efd7e400640 1 -- 172.21.15.102:0/2975557085 >> 172.21.15.102:0/2975557085 conn(0x7efd780cf5d0 msgr2=0x7efd780d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:09.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.177+0000 7efd7e400640 1 -- 172.21.15.102:0/2975557085 shutdown_connections 2024-03-12T17:06:09.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.177+0000 7efd7e400640 1 -- 172.21.15.102:0/2975557085 wait complete. 2024-03-12T17:06:09.188 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b 7 2024-03-12T17:06:09.195 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-03-12T17:06:09.201+0000 7f9048e13040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-12T17:06:09.339 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-03-12T17:06:09.345+0000 7fb63200d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-12T17:06:09.377 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-03-12T17:06:09.383+0000 7f9048e13040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-12T17:06:09.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.384+0000 7f511d600640 1 Processor -- start 2024-03-12T17:06:09.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.384+0000 7f511d600640 1 -- start start 2024-03-12T17:06:09.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.386+0000 7f511d600640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f511815c7c0 0x7f51180cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:09.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.386+0000 7f511d600640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51180b5780 con 0x7f51180cf7c0 2024-03-12T17:06:09.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.386+0000 7f511d600640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51180d1f10 con 0x7f51180d0780 2024-03-12T17:06:09.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.386+0000 7f511d600640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51180d18c0 con 0x7f511815c7c0 2024-03-12T17:06:09.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.387+0000 7f510fe00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f51180d0780 0x7f51180cf0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:39308/0 (socket says 172.21.15.102:39308) 2024-03-12T17:06:09.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.387+0000 7f510fe00640 1 -- 172.21.15.102:0/511103491 learned_addr learned my addr 172.21.15.102:0/511103491 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-03-12T17:06:09.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.387+0000 7f5117e00640 1 --2- 172.21.15.102:0/511103491 >> v2:172.21.15.52:3300/0 conn(0x7f511815c7c0 0x7f51180cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:09.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.387+0000 7f5117400640 1 -- 172.21.15.102:0/511103491 <== mon.1 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3296577298 0 0) 0x7f51180d1f10 con 0x7f51180d0780 2024-03-12T17:06:09.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.387+0000 7f5117400640 1 -- 172.21.15.102:0/511103491 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50fc003520 con 0x7f51180d0780 2024-03-12T17:06:09.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.388+0000 7f5117400640 1 -- 172.21.15.102:0/511103491 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3510099485 0 0) 0x7f51180b5780 con 0x7f51180cf7c0 2024-03-12T17:06:09.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.388+0000 7f5117400640 1 -- 172.21.15.102:0/511103491 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51180d1f10 con 0x7f51180cf7c0 2024-03-12T17:06:09.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.388+0000 7f5117400640 1 -- 172.21.15.102:0/511103491 <== mon.1 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1979661778 0 0) 0x7f50fc003520 con 0x7f51180d0780 2024-03-12T17:06:09.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.388+0000 7f5117400640 1 -- 172.21.15.102:0/511103491 >> v2:172.21.15.52:3300/0 conn(0x7f511815c7c0 msgr2=0x7f51180cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:09.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.388+0000 7f5117400640 1 --2- 172.21.15.102:0/511103491 >> v2:172.21.15.52:3300/0 conn(0x7f511815c7c0 0x7f51180cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:09.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.388+0000 7f5117400640 1 -- 172.21.15.102:0/511103491 >> v1:172.21.15.52:6789/0 conn(0x7f51180cf7c0 legacy=0x7f51180b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:09.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.388+0000 7f5117400640 1 -- 172.21.15.102:0/511103491 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f511815cba0 con 0x7f51180d0780 2024-03-12T17:06:09.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.388+0000 7f5117400640 1 -- 172.21.15.102:0/511103491 <== mon.1 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f51000017f0 con 0x7f51180d0780 2024-03-12T17:06:09.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.388+0000 7f5117400640 1 -- 172.21.15.102:0/511103491 >> v1:172.21.15.102:6789/0 conn(0x7f51180d0780 legacy=0x7f51180cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:09.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.388+0000 7f5117400640 1 --2- 172.21.15.102:0/511103491 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f50fc003b50 0x7f50fc003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:09.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.388+0000 7f5117400640 1 --2- 172.21.15.102:0/511103491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50fc0047c0 0x7f50fc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:09.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.388+0000 7f5117400640 1 --2- 172.21.15.102:0/511103491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50fc005180 0x7f50fc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:09.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.388+0000 7f5117400640 1 -- 172.21.15.102:0/511103491 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51180d18c0 con 0x7f50fc005180 2024-03-12T17:06:09.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.388+0000 7f5117400640 1 -- 172.21.15.102:0/511103491 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc00c050 con 0x7f50fc0047c0 2024-03-12T17:06:09.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.388+0000 7f5117400640 1 -- 172.21.15.102:0/511103491 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc00c210 con 0x7f50fc003b50 2024-03-12T17:06:09.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.388+0000 7f510fe00640 1 --2- 172.21.15.102:0/511103491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50fc0047c0 0x7f50fc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:09.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.388+0000 7f5117e00640 1 --2- 172.21.15.102:0/511103491 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f50fc003b50 0x7f50fc003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:09.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.389+0000 7f511cc00640 1 --2- 172.21.15.102:0/511103491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50fc005180 0x7f50fc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:09.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.389+0000 7f5117e00640 1 -- 172.21.15.102:0/511103491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50fc0047c0 msgr2=0x7f50fc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:09.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.389+0000 7f5117e00640 1 --2- 172.21.15.102:0/511103491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50fc0047c0 0x7f50fc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:09.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.389+0000 7f5117e00640 1 -- 172.21.15.102:0/511103491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50fc005180 msgr2=0x7f50fc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:09.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.389+0000 7f5117e00640 1 --2- 172.21.15.102:0/511103491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50fc005180 0x7f50fc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:09.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.389+0000 7f5117e00640 1 -- 172.21.15.102:0/511103491 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50fc003520 con 0x7f50fc003b50 2024-03-12T17:06:09.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.389+0000 7f511cc00640 1 -- 172.21.15.102:0/511103491 reap_dead start 2024-03-12T17:06:09.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.389+0000 7f510fe00640 1 --2- 172.21.15.102:0/511103491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50fc0047c0 0x7f50fc004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T17:06:09.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.390+0000 7f5117e00640 1 --2- 172.21.15.102:0/511103491 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f50fc003b50 0x7f50fc003f50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5108009ac0 tx=0x7f50fc0057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16c3800c9f09c21d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:09.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.390+0000 7f5117400640 1 -- 172.21.15.102:0/511103491 <== mon.2 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510802c6f0 con 0x7f50fc003b50 2024-03-12T17:06:09.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.390+0000 7f5117400640 1 -- 172.21.15.102:0/511103491 <== mon.2 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f510802c850 con 0x7f50fc003b50 2024-03-12T17:06:09.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.390+0000 7f5117400640 1 -- 172.21.15.102:0/511103491 <== mon.2 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510802cbe0 con 0x7f50fc003b50 2024-03-12T17:06:09.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.391+0000 7f511d600640 1 -- 172.21.15.102:0/511103491 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f50fc003b50 msgr2=0x7f50fc003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:09.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.391+0000 7f511d600640 1 --2- 172.21.15.102:0/511103491 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f50fc003b50 0x7f50fc003f50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5108009ac0 tx=0x7f50fc0057f0 comp rx=0 tx=0).stop 2024-03-12T17:06:09.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.391+0000 7f511d600640 1 -- 172.21.15.102:0/511103491 shutdown_connections 2024-03-12T17:06:09.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.391+0000 7f511d600640 1 --2- 172.21.15.102:0/511103491 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f50fc003b50 0x7f50fc003f50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:09.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.391+0000 7f511d600640 1 -- 172.21.15.102:0/511103491 >> 172.21.15.102:0/511103491 conn(0x7f51180719f0 msgr2=0x7f511806aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:09.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.391+0000 7f511d600640 1 -- 172.21.15.102:0/511103491 shutdown_connections 2024-03-12T17:06:09.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.391+0000 7f511d600640 1 -- 172.21.15.102:0/511103491 wait complete. 2024-03-12T17:06:09.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.392+0000 7f511d600640 1 Processor -- start 2024-03-12T17:06:09.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.393+0000 7f511d600640 1 -- start start 2024-03-12T17:06:09.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.393+0000 7f511d600640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f50fc003b50 0x7f5118159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:09.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.393+0000 7f511d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51180cb620 0x7f51180b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:09.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.393+0000 7f511d600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51180b38c0 0x7f51180c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:09.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.394+0000 7f511d600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5118159550 con 0x7f51180b38c0 2024-03-12T17:06:09.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.394+0000 7f511d600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51180cbbd0 con 0x7f51180cb620 2024-03-12T17:06:09.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.394+0000 7f511d600640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51180cbeb0 con 0x7f50fc003b50 2024-03-12T17:06:09.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.394+0000 7f5117e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f50fc003b50 0x7f5118159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:09.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.394+0000 7f510fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51180cb620 0x7f51180b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:09.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.394+0000 7f5117e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f50fc003b50 0x7f5118159010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:38478/0 (socket says 172.21.15.102:38478) 2024-03-12T17:06:09.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.394+0000 7f5117e00640 1 -- 172.21.15.102:0/3131953151 learned_addr learned my addr 172.21.15.102:0/3131953151 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-03-12T17:06:09.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.394+0000 7f511cc00640 1 --2- 172.21.15.102:0/3131953151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51180b38c0 0x7f51180c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:09.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.394+0000 7f5117e00640 1 -- 172.21.15.102:0/3131953151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51180cb620 msgr2=0x7f51180b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:09.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.394+0000 7f5117e00640 1 --2- 172.21.15.102:0/3131953151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51180cb620 0x7f51180b3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:09.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.394+0000 7f5117e00640 1 -- 172.21.15.102:0/3131953151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51180b38c0 msgr2=0x7f51180c6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:09.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.394+0000 7f5117e00640 1 --2- 172.21.15.102:0/3131953151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51180b38c0 0x7f51180c6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:09.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.394+0000 7f5117e00640 1 -- 172.21.15.102:0/3131953151 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51180cc190 con 0x7f50fc003b50 2024-03-12T17:06:09.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.394+0000 7f510fe00640 1 --2- 172.21.15.102:0/3131953151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51180cb620 0x7f51180b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T17:06:09.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.394+0000 7f5117e00640 1 --2- 172.21.15.102:0/3131953151 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f50fc003b50 0x7f5118159010 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5108037040 tx=0x7f5108004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:09.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.395+0000 7f5114c00640 1 -- 172.21.15.102:0/3131953151 <== mon.2 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5108003810 con 0x7f50fc003b50 2024-03-12T17:06:09.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.395+0000 7f5114c00640 1 -- 172.21.15.102:0/3131953151 <== mon.2 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5108003970 con 0x7f50fc003b50 2024-03-12T17:06:09.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.395+0000 7f5114c00640 1 -- 172.21.15.102:0/3131953151 <== mon.2 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5108003d00 con 0x7f50fc003b50 2024-03-12T17:06:09.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.395+0000 7f511d600640 1 -- 172.21.15.102:0/3131953151 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51180cc470 con 0x7f50fc003b50 2024-03-12T17:06:09.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.395+0000 7f511d600640 1 -- 172.21.15.102:0/3131953151 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51180c7170 con 0x7f50fc003b50 2024-03-12T17:06:09.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.395+0000 7f5114c00640 1 -- 172.21.15.102:0/3131953151 <== mon.2 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5108005c90 con 0x7f50fc003b50 2024-03-12T17:06:09.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.395+0000 7f5114c00640 1 -- 172.21.15.102:0/3131953151 <== mon.2 v2:172.21.15.102:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f51080066b0 con 0x7f50fc003b50 2024-03-12T17:06:09.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.396+0000 7f511d600640 1 -- 172.21.15.102:0/3131953151 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f511804e030 con 0x7f50fc003b50 2024-03-12T17:06:09.392 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-03-12T17:06:09.398+0000 7fb63200d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-12T17:06:09.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.400+0000 7f5114c00640 1 -- 172.21.15.102:0/3131953151 <== mon.2 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f5108044020 con 0x7f50fc003b50 2024-03-12T17:06:09.432 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-03-12T17:06:09.438+0000 7f9048e13040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-12T17:06:09.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.477+0000 7f511d600640 1 -- 172.21.15.102:0/3131953151 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b", "id": 7} v 0) v1 -- 0x7f51181f2040 con 0x7f50fc003b50 2024-03-12T17:06:09.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.483+0000 7f5114c00640 1 -- 172.21.15.102:0/3131953151 <== mon.2 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5108039070 con 0x7f50fc003b50 2024-03-12T17:06:09.477 INFO:teuthology.orchestra.run.smithi102.stdout:7 2024-03-12T17:06:09.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.484+0000 7f511d600640 1 -- 172.21.15.102:0/3131953151 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f50fc003b50 msgr2=0x7f5118159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:09.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.484+0000 7f511d600640 1 --2- 172.21.15.102:0/3131953151 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f50fc003b50 0x7f5118159010 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5108037040 tx=0x7f5108004730 comp rx=0 tx=0).stop 2024-03-12T17:06:09.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.485+0000 7f511d600640 1 -- 172.21.15.102:0/3131953151 shutdown_connections 2024-03-12T17:06:09.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.485+0000 7f511d600640 1 --2- 172.21.15.102:0/3131953151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51180b38c0 0x7f51180c6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:09.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.485+0000 7f511d600640 1 --2- 172.21.15.102:0/3131953151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51180cb620 0x7f51180b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:09.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.485+0000 7f511d600640 1 --2- 172.21.15.102:0/3131953151 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f50fc003b50 0x7f5118159010 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:09.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.485+0000 7f511d600640 1 -- 172.21.15.102:0/3131953151 >> 172.21.15.102:0/3131953151 conn(0x7f51180719f0 msgr2=0x7f51180ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:09.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.485+0000 7f511d600640 1 -- 172.21.15.102:0/3131953151 shutdown_connections 2024-03-12T17:06:09.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-12T17:06:09.485+0000 7f511d600640 1 -- 172.21.15.102:0/3131953151 wait complete. 2024-03-12T17:06:09.493 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-12T17:06:09.494 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-12T17:06:09.501 INFO:tasks.ceph.osd.0:Started 2024-03-12T17:06:09.501 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-12T17:06:09.501 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-12T17:06:09.508 INFO:tasks.ceph.osd.1:Started 2024-03-12T17:06:09.508 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-12T17:06:09.508 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-12T17:06:09.515 INFO:tasks.ceph.osd.2:Started 2024-03-12T17:06:09.516 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-12T17:06:09.516 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-12T17:06:09.522 INFO:tasks.ceph.osd.3:Started 2024-03-12T17:06:09.522 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-12T17:06:09.522 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-12T17:06:09.529 INFO:tasks.ceph.osd.4:Started 2024-03-12T17:06:09.529 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-12T17:06:09.530 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-12T17:06:09.536 INFO:tasks.ceph.osd.5:Started 2024-03-12T17:06:09.536 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-12T17:06:09.536 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-12T17:06:09.543 INFO:tasks.ceph.osd.6:Started 2024-03-12T17:06:09.543 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-12T17:06:09.544 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-12T17:06:09.550 INFO:tasks.ceph.osd.7:Started 2024-03-12T17:06:09.550 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:06:09.597 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-03-12T17:06:09.604+0000 7fb63200d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-12T17:06:09.628 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-03-12T17:06:09.634+0000 7f9048e13040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-12T17:06:09.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.790+0000 7fb1eec00640 1 Processor -- start 2024-03-12T17:06:09.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.790+0000 7fb1eec00640 1 -- start start 2024-03-12T17:06:09.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.791+0000 7fb1eec00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb1e815cd70 0x7fb1e80d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:09.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.791+0000 7fb1eec00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb1e80d1330 0x7fb1e80b4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:09.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.791+0000 7fb1eec00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb1e80b4880 0x7fb1e806b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:09.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.791+0000 7fb1eec00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1e815d150 con 0x7fb1e80d1330 2024-03-12T17:06:09.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.791+0000 7fb1eec00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1e80b4c80 con 0x7fb1e80b4880 2024-03-12T17:06:09.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.791+0000 7fb1eec00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e80d17d0 con 0x7fb1e815cd70 2024-03-12T17:06:09.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.791+0000 7fb1dfe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb1e80d1330 0x7fb1e80b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:09.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.791+0000 7fb1dfe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb1e80d1330 0x7fb1e80b4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50498/0 (socket says 172.21.15.52:50498) 2024-03-12T17:06:09.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.791+0000 7fb1dfe00640 1 -- 172.21.15.52:0/1656605953 learned_addr learned my addr 172.21.15.52:0/1656605953 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:09.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.791+0000 7fb1e7e00640 1 --2- 172.21.15.52:0/1656605953 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb1e815cd70 0x7fb1e80d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:09.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.791+0000 7fb1ecc00640 1 --2- 172.21.15.52:0/1656605953 >> v2:172.21.15.102:3300/0 conn(0x7fb1e80b4880 0x7fb1e806b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:09.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.792+0000 7fb1dfe00640 1 -- 172.21.15.52:0/1656605953 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb1e815cd70 msgr2=0x7fb1e80d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:09.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.792+0000 7fb1dfe00640 1 --2- 172.21.15.52:0/1656605953 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb1e815cd70 0x7fb1e80d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:09.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.792+0000 7fb1dfe00640 1 -- 172.21.15.52:0/1656605953 >> v2:172.21.15.102:3300/0 conn(0x7fb1e80b4880 msgr2=0x7fb1e806b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:09.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.792+0000 7fb1dfe00640 1 --2- 172.21.15.52:0/1656605953 >> v2:172.21.15.102:3300/0 conn(0x7fb1e80b4880 0x7fb1e806b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:09.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.792+0000 7fb1dfe00640 1 -- 172.21.15.52:0/1656605953 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e80d1ab0 con 0x7fb1e80d1330 2024-03-12T17:06:09.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.794+0000 7fb1dfe00640 1 --2- 172.21.15.52:0/1656605953 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1e80d1330 0x7fb1e80b4340 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb1e80c9910 tx=0x7fb1d002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e686583f643461 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:09.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.794+0000 7fb1e7400640 1 -- 172.21.15.52:0/1656605953 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d0031870 con 0x7fb1e80d1330 2024-03-12T17:06:09.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.794+0000 7fb1e7400640 1 -- 172.21.15.52:0/1656605953 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d00319d0 con 0x7fb1e80d1330 2024-03-12T17:06:09.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.794+0000 7fb1e7400640 1 -- 172.21.15.52:0/1656605953 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d0031d60 con 0x7fb1e80d1330 2024-03-12T17:06:09.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.795+0000 7fb1eec00640 1 -- 172.21.15.52:0/1656605953 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1e80d1330 msgr2=0x7fb1e80b4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:09.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.795+0000 7fb1eec00640 1 --2- 172.21.15.52:0/1656605953 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1e80d1330 0x7fb1e80b4340 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb1e80c9910 tx=0x7fb1d002b910 comp rx=0 tx=0).stop 2024-03-12T17:06:09.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.795+0000 7fb1eec00640 1 -- 172.21.15.52:0/1656605953 shutdown_connections 2024-03-12T17:06:09.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.795+0000 7fb1eec00640 1 --2- 172.21.15.52:0/1656605953 >> v2:172.21.15.102:3300/0 conn(0x7fb1e80b4880 0x7fb1e806b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:09.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.795+0000 7fb1eec00640 1 --2- 172.21.15.52:0/1656605953 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1e80d1330 0x7fb1e80b4340 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:09.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.795+0000 7fb1eec00640 1 --2- 172.21.15.52:0/1656605953 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb1e815cd70 0x7fb1e80d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:09.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.795+0000 7fb1eec00640 1 -- 172.21.15.52:0/1656605953 >> 172.21.15.52:0/1656605953 conn(0x7fb1e80cf340 msgr2=0x7fb1e80cf750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:09.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.795+0000 7fb1eec00640 1 -- 172.21.15.52:0/1656605953 shutdown_connections 2024-03-12T17:06:09.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.795+0000 7fb1eec00640 1 -- 172.21.15.52:0/1656605953 wait complete. 2024-03-12T17:06:09.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.796+0000 7fb1eec00640 1 Processor -- start 2024-03-12T17:06:09.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.797+0000 7fb1eec00640 1 -- start start 2024-03-12T17:06:09.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.797+0000 7fb1eec00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1e80b4880 0x7fb1e80a9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:09.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.797+0000 7fb1eec00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb1e80d1330 0x7fb1e814c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:09.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.797+0000 7fb1eec00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1e815cd70 0x7fb1e814da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:09.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.797+0000 7fb1eec00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e814c6c0 con 0x7fb1e80b4880 2024-03-12T17:06:09.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.797+0000 7fb1eec00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e814c960 con 0x7fb1e815cd70 2024-03-12T17:06:09.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.798+0000 7fb1eec00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e814cc40 con 0x7fb1e80d1330 2024-03-12T17:06:09.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.798+0000 7fb1e7e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1e80b4880 0x7fb1e80a9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:09.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.798+0000 7fb1e7e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1e80b4880 0x7fb1e80a9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50508/0 (socket says 172.21.15.52:50508) 2024-03-12T17:06:09.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.798+0000 7fb1e7e00640 1 -- 172.21.15.52:0/1192103674 learned_addr learned my addr 172.21.15.52:0/1192103674 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:09.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.798+0000 7fb1dfe00640 1 --2- 172.21.15.52:0/1192103674 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb1e80d1330 0x7fb1e814c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:09.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.798+0000 7fb1ecc00640 1 --2- 172.21.15.52:0/1192103674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1e815cd70 0x7fb1e814da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:09.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.798+0000 7fb1e7e00640 1 -- 172.21.15.52:0/1192103674 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb1e80d1330 msgr2=0x7fb1e814c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:09.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.798+0000 7fb1e7e00640 1 --2- 172.21.15.52:0/1192103674 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb1e80d1330 0x7fb1e814c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:09.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.798+0000 7fb1e7e00640 1 -- 172.21.15.52:0/1192103674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1e815cd70 msgr2=0x7fb1e814da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:09.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.798+0000 7fb1e7e00640 1 --2- 172.21.15.52:0/1192103674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1e815cd70 0x7fb1e814da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:09.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.798+0000 7fb1e7e00640 1 -- 172.21.15.52:0/1192103674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d00094d0 con 0x7fb1e80b4880 2024-03-12T17:06:09.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.799+0000 7fb1e7e00640 1 --2- 172.21.15.52:0/1192103674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1e80b4880 0x7fb1e80a9650 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800e870 tx=0x7fb1d800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:09.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.799+0000 7fb1e4c00640 1 -- 172.21.15.52:0/1192103674 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d80094f0 con 0x7fb1e80b4880 2024-03-12T17:06:09.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.799+0000 7fb1e4c00640 1 -- 172.21.15.52:0/1192103674 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d800c940 con 0x7fb1e80b4880 2024-03-12T17:06:09.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.799+0000 7fb1e4c00640 1 -- 172.21.15.52:0/1192103674 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d800ccf0 con 0x7fb1e80b4880 2024-03-12T17:06:09.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.799+0000 7fb1eec00640 1 -- 172.21.15.52:0/1192103674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e814cf20 con 0x7fb1e80b4880 2024-03-12T17:06:09.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.799+0000 7fb1eec00640 1 -- 172.21.15.52:0/1192103674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e814d4e0 con 0x7fb1e80b4880 2024-03-12T17:06:09.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.799+0000 7fb1e4c00640 1 -- 172.21.15.52:0/1192103674 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb1d8005c90 con 0x7fb1e80b4880 2024-03-12T17:06:09.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.799+0000 7fb1e4c00640 1 -- 172.21.15.52:0/1192103674 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fb1d8013b40 con 0x7fb1e80b4880 2024-03-12T17:06:09.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.800+0000 7fb1eec00640 1 -- 172.21.15.52:0/1192103674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1e8073390 con 0x7fb1e80b4880 2024-03-12T17:06:09.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.805+0000 7fb1e4c00640 1 -- 172.21.15.52:0/1192103674 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fb1d8021060 con 0x7fb1e80b4880 2024-03-12T17:06:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.869+0000 7fb1eec00640 1 -- 172.21.15.52:0/1192103674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1e81600b0 con 0x7fb1e80b4880 2024-03-12T17:06:09.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.870+0000 7fb1e4c00640 1 -- 172.21.15.52:0/1192103674 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fb1d8017070 con 0x7fb1e80b4880 2024-03-12T17:06:09.864 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:06:09.864 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":10,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:06:09.480010+0000","last_up_change":"0.000000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:06:09.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.871+0000 7fb1eec00640 1 -- 172.21.15.52:0/1192103674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1e80b4880 msgr2=0x7fb1e80a9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:09.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.871+0000 7fb1eec00640 1 --2- 172.21.15.52:0/1192103674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1e80b4880 0x7fb1e80a9650 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800e870 tx=0x7fb1d800ed30 comp rx=0 tx=0).stop 2024-03-12T17:06:09.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.871+0000 7fb1eec00640 1 -- 172.21.15.52:0/1192103674 shutdown_connections 2024-03-12T17:06:09.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.871+0000 7fb1eec00640 1 --2- 172.21.15.52:0/1192103674 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb1e80d1330 0x7fb1e814c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:09.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.871+0000 7fb1eec00640 1 --2- 172.21.15.52:0/1192103674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1e815cd70 0x7fb1e814da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:09.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.871+0000 7fb1eec00640 1 --2- 172.21.15.52:0/1192103674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1e80b4880 0x7fb1e80a9650 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:09.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.871+0000 7fb1eec00640 1 -- 172.21.15.52:0/1192103674 >> 172.21.15.52:0/1192103674 conn(0x7fb1e80cf340 msgr2=0x7fb1e80cf720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:09.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.871+0000 7fb1eec00640 1 -- 172.21.15.52:0/1192103674 shutdown_connections 2024-03-12T17:06:09.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:09.871+0000 7fb1eec00640 1 -- 172.21.15.52:0/1192103674 wait complete. 2024-03-12T17:06:09.882 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-12T17:06:09.882 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-12T17:06:10.214 INFO:tasks.ceph.osd.4.smithi102.stderr:2024-03-12T17:06:10.220+0000 7fa7ddc12500 -1 Falling back to public interface 2024-03-12T17:06:10.220 INFO:tasks.ceph.osd.6.smithi102.stderr:2024-03-12T17:06:10.227+0000 7fab3b60e500 -1 Falling back to public interface 2024-03-12T17:06:10.227 INFO:tasks.ceph.osd.0.smithi052.stderr:2024-03-12T17:06:10.234+0000 7f4b2f9ad500 -1 Falling back to public interface 2024-03-12T17:06:10.233 INFO:tasks.ceph.osd.2.smithi052.stderr:2024-03-12T17:06:10.240+0000 7f1f53903500 -1 Falling back to public interface 2024-03-12T17:06:10.245 INFO:tasks.ceph.osd.1.smithi052.stderr:2024-03-12T17:06:10.252+0000 7fc62c00e500 -1 Falling back to public interface 2024-03-12T17:06:10.245 INFO:tasks.ceph.osd.3.smithi052.stderr:2024-03-12T17:06:10.252+0000 7f095ae0c500 -1 Falling back to public interface 2024-03-12T17:06:10.251 INFO:tasks.ceph.osd.5.smithi102.stderr:2024-03-12T17:06:10.258+0000 7f7801110500 -1 Falling back to public interface 2024-03-12T17:06:10.269 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-03-12T17:06:10.275+0000 7f57ce60e500 -1 Falling back to public interface 2024-03-12T17:06:10.617 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-03-12T17:06:10.623+0000 7f9048e13040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-12T17:06:10.637 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-03-12T17:06:10.644+0000 7fb63200d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-12T17:06:10.689 DEBUG:teuthology.orchestra.run.smithi052:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-12T17:06:10.862 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.868+0000 7f7e3da00640 1 Processor -- start 2024-03-12T17:06:10.862 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.868+0000 7f7e3da00640 1 -- start start 2024-03-12T17:06:10.873 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.870+0000 7f7e3da00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e3815ce30 0x7f7e380b83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:10.874 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.870+0000 7f7e3da00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7e380b4af0 0x7f7e380b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:10.874 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.870+0000 7f7e3da00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e3815d210 con 0x7f7e380cd8c0 2024-03-12T17:06:10.874 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.870+0000 7f7e3da00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e380b2180 con 0x7f7e380b4af0 2024-03-12T17:06:10.874 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.870+0000 7f7e3da00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e380b88e0 con 0x7f7e3815ce30 2024-03-12T17:06:10.874 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.870+0000 7f7e37400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7e380b4af0 0x7f7e380b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:10.874 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.870+0000 7f7e37400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7e380b4af0 0x7f7e380b5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50520/0 (socket says 172.21.15.52:50520) 2024-03-12T17:06:10.875 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.871+0000 7f7e37400640 1 -- 172.21.15.52:0/1173819278 learned_addr learned my addr 172.21.15.52:0/1173819278 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:10.875 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.871+0000 7f7e36a00640 1 --2- 172.21.15.52:0/1173819278 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e3815ce30 0x7f7e380b83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:10.875 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.871+0000 7f7e37400640 1 -- 172.21.15.52:0/1173819278 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e3815ce30 msgr2=0x7f7e380b83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:10.875 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.871+0000 7f7e37400640 1 --2- 172.21.15.52:0/1173819278 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e3815ce30 0x7f7e380b83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:10.875 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.871+0000 7f7e37400640 1 -- 172.21.15.52:0/1173819278 >> v1:172.21.15.52:6789/0 conn(0x7f7e380cd8c0 legacy=0x7f7e380b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:10.875 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.871+0000 7f7e37400640 1 -- 172.21.15.52:0/1173819278 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e380cddf0 con 0x7f7e380b4af0 2024-03-12T17:06:10.876 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.872+0000 7f7e37400640 1 --2- 172.21.15.52:0/1173819278 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e380b4af0 0x7f7e380b5a90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7e30009e50 tx=0x7f7e3002bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5774a9aa7a8cab17 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:10.876 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.872+0000 7f7e35600640 1 -- 172.21.15.52:0/1173819278 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3002cb90 con 0x7f7e380b4af0 2024-03-12T17:06:10.876 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.872+0000 7f7e35600640 1 -- 172.21.15.52:0/1173819278 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e3002ccf0 con 0x7f7e380b4af0 2024-03-12T17:06:10.876 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.872+0000 7f7e35600640 1 -- 172.21.15.52:0/1173819278 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e30034610 con 0x7f7e380b4af0 2024-03-12T17:06:10.876 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.873+0000 7f7e3da00640 1 -- 172.21.15.52:0/1173819278 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e380b4af0 msgr2=0x7f7e380b5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:10.876 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.873+0000 7f7e3da00640 1 --2- 172.21.15.52:0/1173819278 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e380b4af0 0x7f7e380b5a90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7e30009e50 tx=0x7f7e3002bff0 comp rx=0 tx=0).stop 2024-03-12T17:06:10.876 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.873+0000 7f7e3da00640 1 -- 172.21.15.52:0/1173819278 shutdown_connections 2024-03-12T17:06:10.877 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.873+0000 7f7e3da00640 1 --2- 172.21.15.52:0/1173819278 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e380b4af0 0x7f7e380b5a90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:10.877 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.873+0000 7f7e3da00640 1 --2- 172.21.15.52:0/1173819278 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e3815ce30 0x7f7e380b83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:10.877 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.873+0000 7f7e3da00640 1 -- 172.21.15.52:0/1173819278 >> 172.21.15.52:0/1173819278 conn(0x7f7e380cf3e0 msgr2=0x7f7e380cf7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:10.877 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.873+0000 7f7e3da00640 1 -- 172.21.15.52:0/1173819278 shutdown_connections 2024-03-12T17:06:10.877 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.873+0000 7f7e3da00640 1 -- 172.21.15.52:0/1173819278 wait complete. 2024-03-12T17:06:10.877 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.875+0000 7f7e3da00640 1 Processor -- start 2024-03-12T17:06:10.878 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.875+0000 7f7e3da00640 1 -- start start 2024-03-12T17:06:10.878 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.876+0000 7f7e3da00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e380b4af0 0x7f7e3815ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:10.878 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.876+0000 7f7e3da00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e380cd8c0 0x7f7e380c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:10.878 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.876+0000 7f7e3da00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e3815ce30 0x7f7e380cb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:10.878 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.876+0000 7f7e3da00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e380c6ff0 con 0x7f7e380cd8c0 2024-03-12T17:06:10.879 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.876+0000 7f7e3da00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3815f110 con 0x7f7e380b4af0 2024-03-12T17:06:10.879 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.876+0000 7f7e3da00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e380c7270 con 0x7f7e3815ce30 2024-03-12T17:06:10.879 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.876+0000 7f7e36000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e380cd8c0 0x7f7e380c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:10.879 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.876+0000 7f7e36000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e380cd8c0 0x7f7e380c8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50534/0 (socket says 172.21.15.52:50534) 2024-03-12T17:06:10.879 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.876+0000 7f7e36000640 1 -- 172.21.15.52:0/2568128584 learned_addr learned my addr 172.21.15.52:0/2568128584 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:10.879 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.876+0000 7f7e37400640 1 --2- 172.21.15.52:0/2568128584 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e3815ce30 0x7f7e380cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:10.879 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.876+0000 7f7e36a00640 1 --2- 172.21.15.52:0/2568128584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e380b4af0 0x7f7e3815ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:10.880 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.877+0000 7f7e36000640 1 -- 172.21.15.52:0/2568128584 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e3815ce30 msgr2=0x7f7e380cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:10.880 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.877+0000 7f7e36000640 1 --2- 172.21.15.52:0/2568128584 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e3815ce30 0x7f7e380cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:10.880 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.877+0000 7f7e36000640 1 -- 172.21.15.52:0/2568128584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e380b4af0 msgr2=0x7f7e3815ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:10.880 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.877+0000 7f7e36000640 1 --2- 172.21.15.52:0/2568128584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e380b4af0 0x7f7e3815ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:10.880 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.877+0000 7f7e36000640 1 -- 172.21.15.52:0/2568128584 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e30009b30 con 0x7f7e380cd8c0 2024-03-12T17:06:10.881 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.877+0000 7f7e36000640 1 --2- 172.21.15.52:0/2568128584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e380cd8c0 0x7f7e380c8d30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400c450 tx=0x7f7e2400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:10.881 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.877+0000 7f7e2ea00640 1 -- 172.21.15.52:0/2568128584 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e240052d0 con 0x7f7e380cd8c0 2024-03-12T17:06:10.881 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.877+0000 7f7e2ea00640 1 -- 172.21.15.52:0/2568128584 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e24005430 con 0x7f7e380cd8c0 2024-03-12T17:06:10.881 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.877+0000 7f7e2ea00640 1 -- 172.21.15.52:0/2568128584 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e24012870 con 0x7f7e380cd8c0 2024-03-12T17:06:10.881 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.877+0000 7f7e3da00640 1 -- 172.21.15.52:0/2568128584 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e380c7530 con 0x7f7e380cd8c0 2024-03-12T17:06:10.881 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.878+0000 7f7e3da00640 1 -- 172.21.15.52:0/2568128584 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e380c7af0 con 0x7f7e380cd8c0 2024-03-12T17:06:10.881 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.878+0000 7f7e2ea00640 1 -- 172.21.15.52:0/2568128584 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7e24012cd0 con 0x7f7e380cd8c0 2024-03-12T17:06:10.882 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.878+0000 7f7e2ea00640 1 -- 172.21.15.52:0/2568128584 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f7e24011380 con 0x7f7e380cd8c0 2024-03-12T17:06:10.882 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.879+0000 7f7e3da00640 1 -- 172.21.15.52:0/2568128584 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e380670d0 con 0x7f7e380cd8c0 2024-03-12T17:06:10.882 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.881+0000 7f7e2ea00640 1 -- 172.21.15.52:0/2568128584 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f7e3815d210 con 0x7f7e380cd8c0 2024-03-12T17:06:10.920 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-03-12T17:06:10.927+0000 7f9048e13040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-12T17:06:10.941 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.948+0000 7f7e3da00640 1 -- 172.21.15.52:0/2568128584 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e3806c420 con 0x7f7e380cd8c0 2024-03-12T17:06:10.943 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.949+0000 7f7e2ea00640 1 -- 172.21.15.52:0/2568128584 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f7e240035f0 con 0x7f7e380cd8c0 2024-03-12T17:06:10.943 INFO:teuthology.misc.health.smithi052.stdout: 2024-03-12T17:06:10.943 INFO:teuthology.misc.health.smithi052.stdout:{"epoch":10,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:06:09.480010+0000","last_up_change":"0.000000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:06:10.944 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.951+0000 7f7e3da00640 1 -- 172.21.15.52:0/2568128584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e380cd8c0 msgr2=0x7f7e380c8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:10.944 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.951+0000 7f7e3da00640 1 --2- 172.21.15.52:0/2568128584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e380cd8c0 0x7f7e380c8d30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400c450 tx=0x7f7e2400c910 comp rx=0 tx=0).stop 2024-03-12T17:06:10.945 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.951+0000 7f7e3da00640 1 -- 172.21.15.52:0/2568128584 shutdown_connections 2024-03-12T17:06:10.945 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.951+0000 7f7e3da00640 1 --2- 172.21.15.52:0/2568128584 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e3815ce30 0x7f7e380cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:10.945 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.951+0000 7f7e3da00640 1 --2- 172.21.15.52:0/2568128584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e380cd8c0 0x7f7e380c8d30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:10.945 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.951+0000 7f7e3da00640 1 --2- 172.21.15.52:0/2568128584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e380b4af0 0x7f7e3815ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:10.945 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.951+0000 7f7e3da00640 1 -- 172.21.15.52:0/2568128584 >> 172.21.15.52:0/2568128584 conn(0x7f7e380cf3e0 msgr2=0x7f7e380b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:10.945 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.951+0000 7f7e3da00640 1 -- 172.21.15.52:0/2568128584 shutdown_connections 2024-03-12T17:06:10.945 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:10.951+0000 7f7e3da00640 1 -- 172.21.15.52:0/2568128584 wait complete. 2024-03-12T17:06:10.957 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-12T17:06:10.964 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-03-12T17:06:10.970+0000 7fb63200d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-12T17:06:10.980 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-03-12T17:06:10.986+0000 7f9048e13040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-12T17:06:11.026 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-03-12T17:06:11.033+0000 7fb63200d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-12T17:06:11.110 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-03-12T17:06:11.116+0000 7f9048e13040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-12T17:06:11.168 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-03-12T17:06:11.174+0000 7fb63200d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-12T17:06:12.152 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-03-12T17:06:12.159+0000 7f8fe9600640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-12T17:06:12.152 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-03-12T17:06:12.159+0000 7f8fe9600640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-12T17:06:13.543 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-03-12T17:06:13.550+0000 7f57ce60e500 -1 osd.7 0 log_to_monitors true 2024-03-12T17:06:13.583 INFO:tasks.ceph.osd.3.smithi052.stderr:2024-03-12T17:06:13.590+0000 7f095ae0c500 -1 osd.3 0 log_to_monitors true 2024-03-12T17:06:13.754 INFO:tasks.ceph.osd.0.smithi052.stderr:2024-03-12T17:06:13.760+0000 7f4b2f9ad500 -1 osd.0 0 log_to_monitors true 2024-03-12T17:06:13.755 INFO:tasks.ceph.osd.4.smithi102.stderr:2024-03-12T17:06:13.763+0000 7fa7ddc12500 -1 osd.4 0 log_to_monitors true 2024-03-12T17:06:13.784 INFO:tasks.ceph.osd.6.smithi102.stderr:2024-03-12T17:06:13.792+0000 7fab3b60e500 -1 osd.6 0 log_to_monitors true 2024-03-12T17:06:13.794 INFO:tasks.ceph.osd.2.smithi052.stderr:2024-03-12T17:06:13.800+0000 7f1f53903500 -1 osd.2 0 log_to_monitors true 2024-03-12T17:06:13.800 INFO:tasks.ceph.osd.5.smithi102.stderr:2024-03-12T17:06:13.808+0000 7f7801110500 -1 osd.5 0 log_to_monitors true 2024-03-12T17:06:13.819 INFO:tasks.ceph.osd.1.smithi052.stderr:2024-03-12T17:06:13.825+0000 7fc62c00e500 -1 osd.1 0 log_to_monitors true 2024-03-12T17:06:16.800 INFO:tasks.ceph.osd.0.smithi052.stderr:2024-03-12T17:06:16.807+0000 7f4b2a800640 -1 osd.0 0 waiting for initial osdmap 2024-03-12T17:06:16.816 INFO:tasks.ceph.osd.0.smithi052.stderr:2024-03-12T17:06:16.822+0000 7f4b20800640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-12T17:06:16.947 INFO:tasks.ceph.osd.2.smithi052.stderr:2024-03-12T17:06:16.952+0000 7f1f4e800640 -1 osd.2 0 waiting for initial osdmap 2024-03-12T17:06:16.954 INFO:tasks.ceph.osd.2.smithi052.stderr:2024-03-12T17:06:16.960+0000 7f1f44800640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-12T17:06:16.956 INFO:tasks.ceph.osd.1.smithi052.stderr:2024-03-12T17:06:16.963+0000 7fc626800640 -1 osd.1 0 waiting for initial osdmap 2024-03-12T17:06:16.962 INFO:tasks.ceph.osd.3.smithi052.stderr:2024-03-12T17:06:16.968+0000 7f0955600640 -1 osd.3 0 waiting for initial osdmap 2024-03-12T17:06:16.963 INFO:tasks.ceph.osd.1.smithi052.stderr:2024-03-12T17:06:16.970+0000 7fc61c800640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-12T17:06:16.966 INFO:tasks.ceph.osd.3.smithi052.stderr:2024-03-12T17:06:16.973+0000 7f094b600640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-12T17:06:17.441 INFO:tasks.ceph.osd.5.smithi102.stderr:2024-03-12T17:06:17.449+0000 7f77fc000640 -1 osd.5 0 waiting for initial osdmap 2024-03-12T17:06:17.452 INFO:tasks.ceph.osd.5.smithi102.stderr:2024-03-12T17:06:17.460+0000 7f77f2000640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-12T17:06:17.493 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-03-12T17:06:17.501+0000 7f57c9800640 -1 osd.7 0 waiting for initial osdmap 2024-03-12T17:06:17.501 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-03-12T17:06:17.509+0000 7f57bee00640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-12T17:06:17.554 INFO:tasks.ceph.osd.6.smithi102.stderr:2024-03-12T17:06:17.561+0000 7fab35e00640 -1 osd.6 0 waiting for initial osdmap 2024-03-12T17:06:17.559 INFO:tasks.ceph.osd.6.smithi102.stderr:2024-03-12T17:06:17.567+0000 7fab2be00640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-12T17:06:17.563 INFO:tasks.ceph.osd.4.smithi102.stderr:2024-03-12T17:06:17.568+0000 7fa7d8600640 -1 osd.4 0 waiting for initial osdmap 2024-03-12T17:06:17.571 INFO:tasks.ceph.osd.4.smithi102.stderr:2024-03-12T17:06:17.578+0000 7fa7ce600640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-12T17:06:17.764 DEBUG:teuthology.orchestra.run.smithi052:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-12T17:06:17.946 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.953+0000 7f67b3200640 1 Processor -- start 2024-03-12T17:06:17.947 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.953+0000 7f67b3200640 1 -- start start 2024-03-12T17:06:17.948 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.954+0000 7f67b3200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f67ac142570 0x7f67ac06cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:17.948 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.954+0000 7f67b3200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f67ac0cd8a0 0x7f67ac06a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:17.949 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.954+0000 7f67b3200640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67ac0cb5a0 con 0x7f67ac06a060 2024-03-12T17:06:17.949 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.955+0000 7f67b3200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67ac06aa30 con 0x7f67ac142570 2024-03-12T17:06:17.949 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.955+0000 7f67b3200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67ac0cc400 con 0x7f67ac0cd8a0 2024-03-12T17:06:17.949 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.955+0000 7f67abe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f67ac142570 0x7f67ac06cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:17.949 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.955+0000 7f67a3e00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f67ac06a060 0x7f67ac15d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:53288/0 (socket says 172.21.15.52:53288) 2024-03-12T17:06:17.949 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.955+0000 7f67abe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f67ac142570 0x7f67ac06cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38012/0 (socket says 172.21.15.52:38012) 2024-03-12T17:06:17.949 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.955+0000 7f67a3e00640 1 -- 172.21.15.52:0/2772852409 learned_addr learned my addr 172.21.15.52:0/2772852409 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:06:17.950 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.955+0000 7f67abe00640 1 -- 172.21.15.52:0/2772852409 >> v2:172.21.15.102:3300/0 conn(0x7f67ac0cd8a0 msgr2=0x7f67ac06a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:17.950 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.955+0000 7f67b1200640 1 --2- 172.21.15.52:0/2772852409 >> v2:172.21.15.102:3300/0 conn(0x7f67ac0cd8a0 0x7f67ac06a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:17.950 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.955+0000 7f67abe00640 1 --2- 172.21.15.52:0/2772852409 >> v2:172.21.15.102:3300/0 conn(0x7f67ac0cd8a0 0x7f67ac06a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:17.950 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.955+0000 7f67abe00640 1 -- 172.21.15.52:0/2772852409 >> v1:172.21.15.52:6789/0 conn(0x7f67ac06a060 legacy=0x7f67ac15d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:17.950 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.955+0000 7f67ab400640 1 -- 172.21.15.52:0/2772852409 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1737438458 0 0) 0x7f67ac0cb5a0 con 0x7f67ac06a060 2024-03-12T17:06:17.950 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.955+0000 7f67abe00640 1 -- 172.21.15.52:0/2772852409 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ac0cb7f0 con 0x7f67ac142570 2024-03-12T17:06:17.950 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.955+0000 7f67b1200640 1 --2- 172.21.15.52:0/2772852409 >> v2:172.21.15.102:3300/0 conn(0x7f67ac0cd8a0 0x7f67ac06a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:17.951 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.956+0000 7f67abe00640 1 --2- 172.21.15.52:0/2772852409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67ac142570 0x7f67ac06cd30 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f67ac0afc90 tx=0x7f679c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c18c87dee0068f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:17.951 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.957+0000 7f67ab400640 1 -- 172.21.15.52:0/2772852409 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679c031830 con 0x7f67ac142570 2024-03-12T17:06:17.951 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.957+0000 7f67ab400640 1 -- 172.21.15.52:0/2772852409 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f679c031990 con 0x7f67ac142570 2024-03-12T17:06:17.951 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.957+0000 7f67ab400640 1 -- 172.21.15.52:0/2772852409 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679c031d20 con 0x7f67ac142570 2024-03-12T17:06:17.952 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.957+0000 7f67b3200640 1 -- 172.21.15.52:0/2772852409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67ac142570 msgr2=0x7f67ac06cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:17.952 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.957+0000 7f67b3200640 1 --2- 172.21.15.52:0/2772852409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67ac142570 0x7f67ac06cd30 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f67ac0afc90 tx=0x7f679c02b8d0 comp rx=0 tx=0).stop 2024-03-12T17:06:17.952 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.957+0000 7f67b3200640 1 -- 172.21.15.52:0/2772852409 shutdown_connections 2024-03-12T17:06:17.952 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.957+0000 7f67b3200640 1 --2- 172.21.15.52:0/2772852409 >> v2:172.21.15.102:3300/0 conn(0x7f67ac0cd8a0 0x7f67ac06a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:17.952 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.957+0000 7f67b3200640 1 --2- 172.21.15.52:0/2772852409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67ac142570 0x7f67ac06cd30 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:17.952 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.957+0000 7f67b3200640 1 -- 172.21.15.52:0/2772852409 >> 172.21.15.52:0/2772852409 conn(0x7f67ac069c80 msgr2=0x7f67ac0b38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:17.952 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.957+0000 7f67b3200640 1 -- 172.21.15.52:0/2772852409 shutdown_connections 2024-03-12T17:06:17.952 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.957+0000 7f67b3200640 1 -- 172.21.15.52:0/2772852409 wait complete. 2024-03-12T17:06:17.953 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.959+0000 7f67b3200640 1 Processor -- start 2024-03-12T17:06:17.953 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.959+0000 7f67b3200640 1 -- start start 2024-03-12T17:06:17.954 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.960+0000 7f67b3200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67ac06a060 0x7f67ac135840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:17.954 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.960+0000 7f67b3200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67ac0cd8a0 0x7f67ac157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:17.954 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.960+0000 7f67b3200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67ac142570 0x7f67ac1583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:17.954 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.960+0000 7f67b3200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac1588f0 con 0x7f67ac142570 2024-03-12T17:06:17.954 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.960+0000 7f67b3200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac1372a0 con 0x7f67ac0cd8a0 2024-03-12T17:06:17.954 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.960+0000 7f67b3200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac133640 con 0x7f67ac06a060 2024-03-12T17:06:17.954 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.960+0000 7f67b1200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67ac142570 0x7f67ac1583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:17.955 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.960+0000 7f67b1200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67ac142570 0x7f67ac1583b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38016/0 (socket says 172.21.15.52:38016) 2024-03-12T17:06:17.955 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.960+0000 7f67b1200640 1 -- 172.21.15.52:0/2986094072 learned_addr learned my addr 172.21.15.52:0/2986094072 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:17.955 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.960+0000 7f67abe00640 1 --2- 172.21.15.52:0/2986094072 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67ac06a060 0x7f67ac135840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:17.955 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.961+0000 7f67a3e00640 1 --2- 172.21.15.52:0/2986094072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67ac0cd8a0 0x7f67ac157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:17.955 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.961+0000 7f67b1200640 1 -- 172.21.15.52:0/2986094072 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67ac06a060 msgr2=0x7f67ac135840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:17.955 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.961+0000 7f67b1200640 1 --2- 172.21.15.52:0/2986094072 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67ac06a060 0x7f67ac135840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:17.955 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.961+0000 7f67b1200640 1 -- 172.21.15.52:0/2986094072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67ac0cd8a0 msgr2=0x7f67ac157e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:17.955 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.961+0000 7f67b1200640 1 --2- 172.21.15.52:0/2986094072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67ac0cd8a0 0x7f67ac157e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:17.955 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.961+0000 7f67b1200640 1 -- 172.21.15.52:0/2986094072 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679c009400 con 0x7f67ac142570 2024-03-12T17:06:17.956 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.961+0000 7f67b1200640 1 --2- 172.21.15.52:0/2986094072 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67ac142570 0x7f67ac1583b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f67a400cf20 tx=0x7f67a400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:17.956 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.961+0000 7f67a8c00640 1 -- 172.21.15.52:0/2986094072 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a400d840 con 0x7f67ac142570 2024-03-12T17:06:17.956 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.961+0000 7f67a8c00640 1 -- 172.21.15.52:0/2986094072 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a400d9a0 con 0x7f67ac142570 2024-03-12T17:06:17.956 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.961+0000 7f67a8c00640 1 -- 172.21.15.52:0/2986094072 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a400dd50 con 0x7f67ac142570 2024-03-12T17:06:17.956 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.961+0000 7f67b3200640 1 -- 172.21.15.52:0/2986094072 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67ac1338c0 con 0x7f67ac142570 2024-03-12T17:06:17.956 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.961+0000 7f67b3200640 1 -- 172.21.15.52:0/2986094072 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67ac133e80 con 0x7f67ac142570 2024-03-12T17:06:17.957 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.962+0000 7f67b3200640 1 -- 172.21.15.52:0/2986094072 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67ac071c00 con 0x7f67ac142570 2024-03-12T17:06:17.957 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.963+0000 7f67a8c00640 1 -- 172.21.15.52:0/2986094072 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 4) v1 ==== 89081+0+0 (secure 0 0 0) 0x7f67a401e3b0 con 0x7f67ac142570 2024-03-12T17:06:17.957 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.964+0000 7f67a8c00640 1 --2- 172.21.15.52:0/2986094072 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f678806aba0 0x7f678806d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:17.958 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.964+0000 7f67a8c00640 1 -- 172.21.15.52:0/2986094072 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f67a4089880 con 0x7f67ac142570 2024-03-12T17:06:17.958 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.965+0000 7f67abe00640 1 --2- 172.21.15.52:0/2986094072 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f678806aba0 0x7f678806d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:17.959 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.966+0000 7f67abe00640 1 --2- 172.21.15.52:0/2986094072 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f678806aba0 0x7f678806d060 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f679c036000 tx=0x7f679c03c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:17.963 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:17.969+0000 7f67a8c00640 1 -- 172.21.15.52:0/2986094072 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f67a4054f70 con 0x7f67ac142570 2024-03-12T17:06:18.157 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:18.163+0000 7f67b3200640 1 -- 172.21.15.52:0/2986094072 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67ac04e110 con 0x7f67ac142570 2024-03-12T17:06:18.159 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:18.165+0000 7f67a8c00640 1 -- 172.21.15.52:0/2986094072 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10440 (secure 0 0 0) 0x7f67a401a020 con 0x7f67ac142570 2024-03-12T17:06:18.159 INFO:teuthology.misc.health.smithi052.stdout: 2024-03-12T17:06:18.159 INFO:teuthology.misc.health.smithi052.stdout:{"epoch":13,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:06:17.807936+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:06:18.162 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:18.168+0000 7f67b3200640 1 -- 172.21.15.52:0/2986094072 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f678806aba0 msgr2=0x7f678806d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:18.162 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:18.168+0000 7f67b3200640 1 --2- 172.21.15.52:0/2986094072 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f678806aba0 0x7f678806d060 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f679c036000 tx=0x7f679c03c040 comp rx=0 tx=0).stop 2024-03-12T17:06:18.162 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:18.168+0000 7f67b3200640 1 -- 172.21.15.52:0/2986094072 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67ac142570 msgr2=0x7f67ac1583b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:18.162 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:18.168+0000 7f67b3200640 1 --2- 172.21.15.52:0/2986094072 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67ac142570 0x7f67ac1583b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f67a400cf20 tx=0x7f67a400a7d0 comp rx=0 tx=0).stop 2024-03-12T17:06:18.162 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:18.168+0000 7f67b3200640 1 -- 172.21.15.52:0/2986094072 shutdown_connections 2024-03-12T17:06:18.162 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:18.168+0000 7f67b3200640 1 --2- 172.21.15.52:0/2986094072 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f678806aba0 0x7f678806d060 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:18.162 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:18.168+0000 7f67b3200640 1 --2- 172.21.15.52:0/2986094072 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67ac142570 0x7f67ac1583b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:18.162 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:18.168+0000 7f67b3200640 1 --2- 172.21.15.52:0/2986094072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67ac0cd8a0 0x7f67ac157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:18.162 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:18.168+0000 7f67b3200640 1 --2- 172.21.15.52:0/2986094072 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67ac06a060 0x7f67ac135840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:18.162 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:18.168+0000 7f67b3200640 1 -- 172.21.15.52:0/2986094072 >> 172.21.15.52:0/2986094072 conn(0x7f67ac069c80 msgr2=0x7f67ac0b3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:18.162 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:18.168+0000 7f67b3200640 1 -- 172.21.15.52:0/2986094072 shutdown_connections 2024-03-12T17:06:18.163 INFO:teuthology.misc.health.smithi052.stderr:2024-03-12T17:06:18.168+0000 7f67b3200640 1 -- 172.21.15.52:0/2986094072 wait complete. 2024-03-12T17:06:18.175 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-12T17:06:18.175 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-12T17:06:18.176 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-12T17:06:18.176 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-12T17:06:18.183 INFO:tasks.ceph.mds.a:Started 2024-03-12T17:06:18.183 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-12T17:06:18.183 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-12T17:06:18.191 INFO:tasks.ceph.mds.c:Started 2024-03-12T17:06:18.192 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-12T17:06:18.192 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-12T17:06:18.199 INFO:tasks.ceph.mds.b:Started 2024-03-12T17:06:18.199 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-12T17:06:18.199 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-12T17:06:18.206 INFO:tasks.ceph.mds.d:Started 2024-03-12T17:06:18.206 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-12T17:06:18.207 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:06:18.332 INFO:tasks.ceph.mds.a.smithi052.stdout:starting mds.a at 2024-03-12T17:06:18.333 INFO:tasks.ceph.mds.c.smithi052.stdout:starting mds.c at 2024-03-12T17:06:18.358 INFO:tasks.ceph.mds.d.smithi102.stdout:starting mds.d at 2024-03-12T17:06:18.359 INFO:tasks.ceph.mds.b.smithi102.stdout:starting mds.b at 2024-03-12T17:06:18.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.391+0000 7f19e3600640 1 Processor -- start 2024-03-12T17:06:18.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.391+0000 7f19e3600640 1 -- start start 2024-03-12T17:06:18.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.392+0000 7f19e3600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f19dc15cda0 0x7f19dc0cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:18.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.393+0000 7f19e3600640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f19dc0b4a60 0x7f19dc0b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:18.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.393+0000 7f19e3600640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19dc0d1830 con 0x7f19dc0b5a30 2024-03-12T17:06:18.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.393+0000 7f19e3600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19dc0b1f20 con 0x7f19dc15cda0 2024-03-12T17:06:18.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.393+0000 7f19e3600640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19dc0d0c20 con 0x7f19dc0b4a60 2024-03-12T17:06:18.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.393+0000 7f19e0c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f19dc15cda0 0x7f19dc0cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:18.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.393+0000 7f19e0c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f19dc15cda0 0x7f19dc0cdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:52542/0 (socket says 172.21.15.52:52542) 2024-03-12T17:06:18.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.393+0000 7f19dbe00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f19dc0b5a30 0x7f19dc0b4350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.52:49900/0 (socket says 172.21.15.52:49900) 2024-03-12T17:06:18.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.393+0000 7f19e0c00640 1 -- 172.21.15.52:0/1158194275 learned_addr learned my addr 172.21.15.52:0/1158194275 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:18.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.393+0000 7f19e1600640 1 --2- 172.21.15.52:0/1158194275 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f19dc0b4a60 0x7f19dc0b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:18.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.393+0000 7f19e0c00640 1 -- 172.21.15.52:0/1158194275 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f19dc0b4a60 msgr2=0x7f19dc0b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:18.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.393+0000 7f19e0c00640 1 --2- 172.21.15.52:0/1158194275 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f19dc0b4a60 0x7f19dc0b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:18.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.393+0000 7f19e0c00640 1 -- 172.21.15.52:0/1158194275 >> v1:172.21.15.102:6789/0 conn(0x7f19dc0b5a30 legacy=0x7f19dc0b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:18.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.393+0000 7f19e0c00640 1 -- 172.21.15.52:0/1158194275 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19dc0d0f00 con 0x7f19dc15cda0 2024-03-12T17:06:18.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.394+0000 7f19e0c00640 1 --2- 172.21.15.52:0/1158194275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19dc15cda0 0x7f19dc0cdb40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f19d0009870 tx=0x7f19d002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90166a6e9ef3aec8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:18.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.394+0000 7f19db400640 1 -- 172.21.15.52:0/1158194275 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d0030840 con 0x7f19dc15cda0 2024-03-12T17:06:18.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.395+0000 7f19db400640 1 -- 172.21.15.52:0/1158194275 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d00309a0 con 0x7f19dc15cda0 2024-03-12T17:06:18.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.395+0000 7f19db400640 1 -- 172.21.15.52:0/1158194275 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d0030d30 con 0x7f19dc15cda0 2024-03-12T17:06:18.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.395+0000 7f19e3600640 1 -- 172.21.15.52:0/1158194275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19dc15cda0 msgr2=0x7f19dc0cdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:18.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.395+0000 7f19e3600640 1 --2- 172.21.15.52:0/1158194275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19dc15cda0 0x7f19dc0cdb40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f19d0009870 tx=0x7f19d002b980 comp rx=0 tx=0).stop 2024-03-12T17:06:18.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.395+0000 7f19e3600640 1 -- 172.21.15.52:0/1158194275 shutdown_connections 2024-03-12T17:06:18.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.395+0000 7f19e3600640 1 --2- 172.21.15.52:0/1158194275 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f19dc0b4a60 0x7f19dc0b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:18.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.395+0000 7f19e3600640 1 --2- 172.21.15.52:0/1158194275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19dc15cda0 0x7f19dc0cdb40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:18.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.395+0000 7f19e3600640 1 -- 172.21.15.52:0/1158194275 >> 172.21.15.52:0/1158194275 conn(0x7f19dc0cf350 msgr2=0x7f19dc0cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:18.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.395+0000 7f19e3600640 1 -- 172.21.15.52:0/1158194275 shutdown_connections 2024-03-12T17:06:18.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.395+0000 7f19e3600640 1 -- 172.21.15.52:0/1158194275 wait complete. 2024-03-12T17:06:18.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.397+0000 7f19e3600640 1 Processor -- start 2024-03-12T17:06:18.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.397+0000 7f19e3600640 1 -- start start 2024-03-12T17:06:18.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.398+0000 7f19e3600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19dc0b4a60 0x7f19dc1f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:18.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.398+0000 7f19e3600640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f19dc0b5a30 0x7f19dc0c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:18.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.398+0000 7f19e3600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19dc15cda0 0x7f19dc0b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:18.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.398+0000 7f19e3600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19dc1f2180 con 0x7f19dc15cda0 2024-03-12T17:06:18.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.398+0000 7f19e3600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19dc0c9090 con 0x7f19dc0b4a60 2024-03-12T17:06:18.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.398+0000 7f19e3600640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19dc0c8480 con 0x7f19dc0b5a30 2024-03-12T17:06:18.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.398+0000 7f19e1600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19dc15cda0 0x7f19dc0b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:18.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.398+0000 7f19e1600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19dc15cda0 0x7f19dc0b3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38060/0 (socket says 172.21.15.52:38060) 2024-03-12T17:06:18.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.398+0000 7f19e1600640 1 -- 172.21.15.52:0/3848690109 learned_addr learned my addr 172.21.15.52:0/3848690109 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:18.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.398+0000 7f19e0c00640 1 --2- 172.21.15.52:0/3848690109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19dc0b4a60 0x7f19dc1f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:18.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.398+0000 7f19dbe00640 1 --2- 172.21.15.52:0/3848690109 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f19dc0b5a30 0x7f19dc0c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:18.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.399+0000 7f19e0c00640 1 -- 172.21.15.52:0/3848690109 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f19dc0b5a30 msgr2=0x7f19dc0c7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:18.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.399+0000 7f19e0c00640 1 --2- 172.21.15.52:0/3848690109 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f19dc0b5a30 0x7f19dc0c7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:18.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.399+0000 7f19e0c00640 1 -- 172.21.15.52:0/3848690109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19dc15cda0 msgr2=0x7f19dc0b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:18.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.399+0000 7f19e0c00640 1 --2- 172.21.15.52:0/3848690109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19dc15cda0 0x7f19dc0b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:18.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.399+0000 7f19e0c00640 1 -- 172.21.15.52:0/3848690109 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d00094d0 con 0x7f19dc0b4a60 2024-03-12T17:06:18.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.399+0000 7f19e0c00640 1 --2- 172.21.15.52:0/3848690109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19dc0b4a60 0x7f19dc1f1c40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f19dc0d39d0 tx=0x7f19d0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:18.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.399+0000 7f19d8c00640 1 -- 172.21.15.52:0/3848690109 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d0039a10 con 0x7f19dc0b4a60 2024-03-12T17:06:18.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.399+0000 7f19d8c00640 1 -- 172.21.15.52:0/3848690109 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d0039b70 con 0x7f19dc0b4a60 2024-03-12T17:06:18.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.399+0000 7f19d8c00640 1 -- 172.21.15.52:0/3848690109 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d0043410 con 0x7f19dc0b4a60 2024-03-12T17:06:18.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.399+0000 7f19e3600640 1 -- 172.21.15.52:0/3848690109 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19dc0c8760 con 0x7f19dc0b4a60 2024-03-12T17:06:18.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.399+0000 7f19e3600640 1 -- 172.21.15.52:0/3848690109 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19dc0c8d20 con 0x7f19dc0b4a60 2024-03-12T17:06:18.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.400+0000 7f19e3600640 1 -- 172.21.15.52:0/3848690109 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19dc073390 con 0x7f19dc0b4a60 2024-03-12T17:06:18.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.401+0000 7f19d8c00640 1 -- 172.21.15.52:0/3848690109 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 4) v1 ==== 89081+0+0 (secure 0 0 0) 0x7f19d002f390 con 0x7f19dc0b4a60 2024-03-12T17:06:18.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.402+0000 7f19d8c00640 1 --2- 172.21.15.52:0/3848690109 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f19bc06ab70 0x7f19bc06d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:18.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.402+0000 7f19d8c00640 1 -- 172.21.15.52:0/3848690109 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f19d00aad40 con 0x7f19dc0b4a60 2024-03-12T17:06:18.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.402+0000 7f19dbe00640 1 --2- 172.21.15.52:0/3848690109 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f19bc06ab70 0x7f19bc06d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:18.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.403+0000 7f19dbe00640 1 --2- 172.21.15.52:0/3848690109 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f19bc06ab70 0x7f19bc06d030 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f19c8000f60 tx=0x7f19c8000ed0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:18.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.409+0000 7f19d8c00640 1 -- 172.21.15.52:0/3848690109 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f19d0076570 con 0x7f19dc0b4a60 2024-03-12T17:06:18.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.600+0000 7f19e3600640 1 -- 172.21.15.52:0/3848690109 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19dc0b3890 con 0x7f19dc0b4a60 2024-03-12T17:06:18.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.603+0000 7f19d8c00640 1 -- 172.21.15.52:0/3848690109 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10440 (secure 0 0 0) 0x7f19d0048090 con 0x7f19dc0b4a60 2024-03-12T17:06:18.596 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:06:18.596 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":13,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:06:17.807936+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:06:18.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.605+0000 7f19e3600640 1 -- 172.21.15.52:0/3848690109 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f19bc06ab70 msgr2=0x7f19bc06d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:18.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.605+0000 7f19e3600640 1 --2- 172.21.15.52:0/3848690109 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f19bc06ab70 0x7f19bc06d030 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f19c8000f60 tx=0x7f19c8000ed0 comp rx=0 tx=0).stop 2024-03-12T17:06:18.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.605+0000 7f19e3600640 1 -- 172.21.15.52:0/3848690109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19dc0b4a60 msgr2=0x7f19dc1f1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:18.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.605+0000 7f19e3600640 1 --2- 172.21.15.52:0/3848690109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19dc0b4a60 0x7f19dc1f1c40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f19dc0d39d0 tx=0x7f19d0004730 comp rx=0 tx=0).stop 2024-03-12T17:06:18.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.605+0000 7f19e3600640 1 -- 172.21.15.52:0/3848690109 shutdown_connections 2024-03-12T17:06:18.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.605+0000 7f19e3600640 1 --2- 172.21.15.52:0/3848690109 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f19bc06ab70 0x7f19bc06d030 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:18.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.605+0000 7f19e3600640 1 --2- 172.21.15.52:0/3848690109 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f19dc0b5a30 0x7f19dc0c7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:18.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.605+0000 7f19e3600640 1 --2- 172.21.15.52:0/3848690109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19dc15cda0 0x7f19dc0b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:18.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.605+0000 7f19e3600640 1 --2- 172.21.15.52:0/3848690109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19dc0b4a60 0x7f19dc1f1c40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:18.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.605+0000 7f19e3600640 1 -- 172.21.15.52:0/3848690109 >> 172.21.15.52:0/3848690109 conn(0x7f19dc0cf350 msgr2=0x7f19dc0b7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:18.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.605+0000 7f19e3600640 1 -- 172.21.15.52:0/3848690109 shutdown_connections 2024-03-12T17:06:18.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.605+0000 7f19e3600640 1 -- 172.21.15.52:0/3848690109 wait complete. 2024-03-12T17:06:18.616 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-03-12T17:06:18.616 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T17:06:18.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.838+0000 7f937a600640 1 Processor -- start 2024-03-12T17:06:18.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.838+0000 7f937a600640 1 -- start start 2024-03-12T17:06:18.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.840+0000 7f937a600640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f93740cbab0 0x7f93740b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:18.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.840+0000 7f937a600640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f937406d390 0x7f937406d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:18.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.840+0000 7f937a600640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93740b16a0 con 0x7f93740adbc0 2024-03-12T17:06:18.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.840+0000 7f937a600640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f937406dd60 con 0x7f937406d390 2024-03-12T17:06:18.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.840+0000 7f937a600640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93740cbfb0 con 0x7f93740cbab0 2024-03-12T17:06:18.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.840+0000 7f9373400640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f93740adbc0 0x7f93740b41c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:53318/0 (socket says 172.21.15.52:53318) 2024-03-12T17:06:18.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.840+0000 7f9373400640 1 -- 172.21.15.52:0/2800765235 learned_addr learned my addr 172.21.15.52:0/2800765235 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:06:18.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.840+0000 7f9372a00640 1 --2- 172.21.15.52:0/2800765235 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f93740cbab0 0x7f93740b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:18.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.840+0000 7f9373e00640 1 --2- 172.21.15.52:0/2800765235 >> v2:172.21.15.52:3300/0 conn(0x7f937406d390 0x7f937406d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:18.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.841+0000 7f9372000640 1 -- 172.21.15.52:0/2800765235 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 604628491 0 0) 0x7f93740b16a0 con 0x7f93740adbc0 2024-03-12T17:06:18.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.841+0000 7f9372000640 1 -- 172.21.15.52:0/2800765235 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9358003520 con 0x7f93740adbc0 2024-03-12T17:06:18.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.841+0000 7f9372000640 1 -- 172.21.15.52:0/2800765235 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 175785243 0 0) 0x7f9358003520 con 0x7f93740adbc0 2024-03-12T17:06:18.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.841+0000 7f9373e00640 1 -- 172.21.15.52:0/2800765235 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f93740cbab0 msgr2=0x7f93740b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:18.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.841+0000 7f9373e00640 1 --2- 172.21.15.52:0/2800765235 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f93740cbab0 0x7f93740b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:18.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.841+0000 7f9373e00640 1 -- 172.21.15.52:0/2800765235 >> v1:172.21.15.52:6789/0 conn(0x7f93740adbc0 legacy=0x7f93740b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:18.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.841+0000 7f9373e00640 1 -- 172.21.15.52:0/2800765235 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93740cc290 con 0x7f937406d390 2024-03-12T17:06:18.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.842+0000 7f9373e00640 1 --2- 172.21.15.52:0/2800765235 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f937406d390 0x7f937406d820 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f936c009d90 tx=0x7f936c02bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74b9693a9ea0bfd server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:18.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.842+0000 7f9372000640 1 -- 172.21.15.52:0/2800765235 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9364001670 con 0x7f937406d390 2024-03-12T17:06:18.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.843+0000 7f9372000640 1 -- 172.21.15.52:0/2800765235 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936c02c810 con 0x7f937406d390 2024-03-12T17:06:18.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.843+0000 7f9372000640 1 -- 172.21.15.52:0/2800765235 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c02cba0 con 0x7f937406d390 2024-03-12T17:06:18.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.843+0000 7f937a600640 1 -- 172.21.15.52:0/2800765235 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f937406d390 msgr2=0x7f937406d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:18.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.843+0000 7f937a600640 1 --2- 172.21.15.52:0/2800765235 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f937406d390 0x7f937406d820 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f936c009d90 tx=0x7f936c02bce0 comp rx=0 tx=0).stop 2024-03-12T17:06:18.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.843+0000 7f937a600640 1 -- 172.21.15.52:0/2800765235 shutdown_connections 2024-03-12T17:06:18.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.843+0000 7f937a600640 1 --2- 172.21.15.52:0/2800765235 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f937406d390 0x7f937406d820 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:18.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.843+0000 7f937a600640 1 --2- 172.21.15.52:0/2800765235 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f93740cbab0 0x7f93740b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:18.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.843+0000 7f937a600640 1 -- 172.21.15.52:0/2800765235 >> 172.21.15.52:0/2800765235 conn(0x7f937406a3d0 msgr2=0x7f93740b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:18.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.843+0000 7f937a600640 1 -- 172.21.15.52:0/2800765235 shutdown_connections 2024-03-12T17:06:18.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.843+0000 7f937a600640 1 -- 172.21.15.52:0/2800765235 wait complete. 2024-03-12T17:06:18.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.845+0000 7f937a600640 1 Processor -- start 2024-03-12T17:06:18.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.845+0000 7f937a600640 1 -- start start 2024-03-12T17:06:18.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.846+0000 7f937a600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f937406d390 0x7f937415c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:18.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.846+0000 7f937a600640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f93740adbc0 0x7f937415a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:18.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.846+0000 7f937a600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93740cbab0 0x7f9374155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:18.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.846+0000 7f937a600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9374159360 con 0x7f93740cbab0 2024-03-12T17:06:18.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.846+0000 7f937a600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937415ce60 con 0x7f937406d390 2024-03-12T17:06:18.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.846+0000 7f937a600640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93741362d0 con 0x7f93740adbc0 2024-03-12T17:06:18.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.846+0000 7f9373e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93740cbab0 0x7f9374155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:18.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.846+0000 7f9373e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93740cbab0 0x7f9374155e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38072/0 (socket says 172.21.15.52:38072) 2024-03-12T17:06:18.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.846+0000 7f9373e00640 1 -- 172.21.15.52:0/3075741176 learned_addr learned my addr 172.21.15.52:0/3075741176 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:18.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.847+0000 7f9372a00640 1 --2- 172.21.15.52:0/3075741176 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f93740adbc0 0x7f937415a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:18.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.847+0000 7f9373400640 1 --2- 172.21.15.52:0/3075741176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f937406d390 0x7f937415c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:18.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.847+0000 7f9373e00640 1 -- 172.21.15.52:0/3075741176 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f93740adbc0 msgr2=0x7f937415a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:18.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.847+0000 7f9373e00640 1 --2- 172.21.15.52:0/3075741176 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f93740adbc0 0x7f937415a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:18.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.847+0000 7f9373e00640 1 -- 172.21.15.52:0/3075741176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f937406d390 msgr2=0x7f937415c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:18.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.847+0000 7f9373e00640 1 --2- 172.21.15.52:0/3075741176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f937406d390 0x7f937415c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:18.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.847+0000 7f9373e00640 1 -- 172.21.15.52:0/3075741176 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936c009a70 con 0x7f93740cbab0 2024-03-12T17:06:18.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.847+0000 7f9372a00640 1 --2- 172.21.15.52:0/3075741176 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f93740adbc0 0x7f937415a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:18.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.847+0000 7f9373400640 1 --2- 172.21.15.52:0/3075741176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f937406d390 0x7f937415c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:18.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.847+0000 7f9373e00640 1 --2- 172.21.15.52:0/3075741176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93740cbab0 0x7f9374155e50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f936c009d90 tx=0x7f936c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:18.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.847+0000 7f936b400640 1 -- 172.21.15.52:0/3075741176 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c009450 con 0x7f93740cbab0 2024-03-12T17:06:18.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.847+0000 7f936b400640 1 -- 172.21.15.52:0/3075741176 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936c034c50 con 0x7f93740cbab0 2024-03-12T17:06:18.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.847+0000 7f936b400640 1 -- 172.21.15.52:0/3075741176 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c040450 con 0x7f93740cbab0 2024-03-12T17:06:18.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.847+0000 7f937a600640 1 -- 172.21.15.52:0/3075741176 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9374136890 con 0x7f93740cbab0 2024-03-12T17:06:18.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.848+0000 7f937a600640 1 -- 172.21.15.52:0/3075741176 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f937415af60 con 0x7f93740cbab0 2024-03-12T17:06:18.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.849+0000 7f937a600640 1 -- 172.21.15.52:0/3075741176 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f937406ac20 con 0x7f93740cbab0 2024-03-12T17:06:18.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.850+0000 7f936b400640 1 -- 172.21.15.52:0/3075741176 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 4) v1 ==== 89081+0+0 (secure 0 0 0) 0x7f936c0405b0 con 0x7f93740cbab0 2024-03-12T17:06:18.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.850+0000 7f936b400640 1 --2- 172.21.15.52:0/3075741176 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f935406ab70 0x7f935406d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:18.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.851+0000 7f936b400640 1 -- 172.21.15.52:0/3075741176 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f936c0ab620 con 0x7f93740cbab0 2024-03-12T17:06:18.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.851+0000 7f9373400640 1 --2- 172.21.15.52:0/3075741176 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f935406ab70 0x7f935406d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:18.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.852+0000 7f9373400640 1 --2- 172.21.15.52:0/3075741176 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f935406ab70 0x7f935406d030 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f93640011e0 tx=0x7f9364009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:18.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:18.855+0000 7f936b400640 1 -- 172.21.15.52:0/3075741176 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f936c076e40 con 0x7f93740cbab0 2024-03-12T17:06:19.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:19.097+0000 7f937a600640 1 -- 172.21.15.52:0/3075741176 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93740b2210 con 0x7f93740cbab0 2024-03-12T17:06:19.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:19.098+0000 7f936b400640 1 -- 172.21.15.52:0/3075741176 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1253 (secure 0 0 0) 0x7f936c07a460 con 0x7f93740cbab0 2024-03-12T17:06:19.092 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:06:19.092 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/846108993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":846108993},{"type":"v1","addr":"172.21.15.52:6835","nonce":846108993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-03-12T17:06:19.092 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 2 2024-03-12T17:06:19.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:19.101+0000 7f937a600640 1 -- 172.21.15.52:0/3075741176 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f935406ab70 msgr2=0x7f935406d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:19.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:19.101+0000 7f937a600640 1 --2- 172.21.15.52:0/3075741176 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f935406ab70 0x7f935406d030 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f93640011e0 tx=0x7f9364009040 comp rx=0 tx=0).stop 2024-03-12T17:06:19.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:19.101+0000 7f937a600640 1 -- 172.21.15.52:0/3075741176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93740cbab0 msgr2=0x7f9374155e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:19.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:19.101+0000 7f937a600640 1 --2- 172.21.15.52:0/3075741176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93740cbab0 0x7f9374155e50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f936c009d90 tx=0x7f936c004a40 comp rx=0 tx=0).stop 2024-03-12T17:06:19.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:19.101+0000 7f937a600640 1 -- 172.21.15.52:0/3075741176 shutdown_connections 2024-03-12T17:06:19.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:19.101+0000 7f937a600640 1 --2- 172.21.15.52:0/3075741176 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f935406ab70 0x7f935406d030 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:19.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:19.101+0000 7f937a600640 1 --2- 172.21.15.52:0/3075741176 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f93740adbc0 0x7f937415a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:19.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:19.101+0000 7f937a600640 1 --2- 172.21.15.52:0/3075741176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93740cbab0 0x7f9374155e50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:19.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:19.101+0000 7f937a600640 1 --2- 172.21.15.52:0/3075741176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f937406d390 0x7f937415c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:19.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:19.101+0000 7f937a600640 1 -- 172.21.15.52:0/3075741176 >> 172.21.15.52:0/3075741176 conn(0x7f937406a3d0 msgr2=0x7f937406c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:19.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:19.101+0000 7f937a600640 1 -- 172.21.15.52:0/3075741176 shutdown_connections 2024-03-12T17:06:19.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:19.101+0000 7f937a600640 1 -- 172.21.15.52:0/3075741176 wait complete. 2024-03-12T17:06:21.113 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T17:06:21.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.349+0000 7ff0a3200640 1 Processor -- start 2024-03-12T17:06:21.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.349+0000 7ff0a3200640 1 -- start start 2024-03-12T17:06:21.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.350+0000 7ff0a3200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff0a4143930 0x7ff0a40cd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:21.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.350+0000 7ff0a3200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0a40cdc50 0x7ff0a4070020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:21.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.350+0000 7ff0a3200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0a415b2b0 con 0x7ff0a4143460 2024-03-12T17:06:21.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.351+0000 7ff0a3200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0a40cc6b0 con 0x7ff0a4143930 2024-03-12T17:06:21.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.351+0000 7ff0a3200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a40cc950 con 0x7ff0a40cdc50 2024-03-12T17:06:21.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.351+0000 7ff0a1400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff0a4143930 0x7ff0a40cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:21.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.351+0000 7ff0a1400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff0a4143930 0x7ff0a40cd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38082/0 (socket says 172.21.15.52:38082) 2024-03-12T17:06:21.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.351+0000 7ff0a1400640 1 -- 172.21.15.52:0/1431756799 learned_addr learned my addr 172.21.15.52:0/1431756799 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:21.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.351+0000 7ff0a2800640 1 --2- 172.21.15.52:0/1431756799 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0a40cdc50 0x7ff0a4070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:21.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.351+0000 7ff0a1400640 1 -- 172.21.15.52:0/1431756799 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0a40cdc50 msgr2=0x7ff0a4070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:21.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.351+0000 7ff0a1400640 1 --2- 172.21.15.52:0/1431756799 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0a40cdc50 0x7ff0a4070020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:21.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.351+0000 7ff0a1400640 1 -- 172.21.15.52:0/1431756799 >> v1:172.21.15.102:6789/0 conn(0x7ff0a4143460 legacy=0x7ff0a415aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:21.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.351+0000 7ff0a1400640 1 -- 172.21.15.52:0/1431756799 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a40ccc30 con 0x7ff0a4143930 2024-03-12T17:06:21.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.352+0000 7ff0a1400640 1 --2- 172.21.15.52:0/1431756799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0a4143930 0x7ff0a40cd710 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff090009870 tx=0x7ff09002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa6979cfdbd6791f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:21.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.353+0000 7ff0a0a00640 1 -- 172.21.15.52:0/1431756799 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09002f840 con 0x7ff0a4143930 2024-03-12T17:06:21.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.353+0000 7ff0a0a00640 1 -- 172.21.15.52:0/1431756799 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09002f9a0 con 0x7ff0a4143930 2024-03-12T17:06:21.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.353+0000 7ff0a0a00640 1 -- 172.21.15.52:0/1431756799 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09002fd30 con 0x7ff0a4143930 2024-03-12T17:06:21.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.353+0000 7ff0a3200640 1 -- 172.21.15.52:0/1431756799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0a4143930 msgr2=0x7ff0a40cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:21.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.353+0000 7ff0a3200640 1 --2- 172.21.15.52:0/1431756799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0a4143930 0x7ff0a40cd710 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff090009870 tx=0x7ff09002ba20 comp rx=0 tx=0).stop 2024-03-12T17:06:21.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.353+0000 7ff0a3200640 1 -- 172.21.15.52:0/1431756799 shutdown_connections 2024-03-12T17:06:21.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.353+0000 7ff0a3200640 1 --2- 172.21.15.52:0/1431756799 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0a40cdc50 0x7ff0a4070020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:21.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.353+0000 7ff0a3200640 1 --2- 172.21.15.52:0/1431756799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0a4143930 0x7ff0a40cd710 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:21.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.353+0000 7ff0a3200640 1 -- 172.21.15.52:0/1431756799 >> 172.21.15.52:0/1431756799 conn(0x7ff0a4065df0 msgr2=0x7ff0a4069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:21.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.354+0000 7ff0a3200640 1 -- 172.21.15.52:0/1431756799 shutdown_connections 2024-03-12T17:06:21.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.354+0000 7ff0a3200640 1 -- 172.21.15.52:0/1431756799 wait complete. 2024-03-12T17:06:21.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.355+0000 7ff0a3200640 1 Processor -- start 2024-03-12T17:06:21.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.355+0000 7ff0a3200640 1 -- start start 2024-03-12T17:06:21.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.356+0000 7ff0a3200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0a40cdc50 0x7ff0a414ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:21.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.356+0000 7ff0a3200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0a4143460 0x7ff0a41373e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:21.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.356+0000 7ff0a3200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0a4143930 0x7ff0a40acc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:21.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.356+0000 7ff0a3200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a414d3b0 con 0x7ff0a4143930 2024-03-12T17:06:21.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.356+0000 7ff0a3200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a4138a70 con 0x7ff0a4143460 2024-03-12T17:06:21.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.356+0000 7ff0a3200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a4137e60 con 0x7ff0a40cdc50 2024-03-12T17:06:21.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.357+0000 7ff0a2800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0a4143930 0x7ff0a40acc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:21.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.357+0000 7ff0a2800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0a4143930 0x7ff0a40acc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38088/0 (socket says 172.21.15.52:38088) 2024-03-12T17:06:21.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.357+0000 7ff0a2800640 1 -- 172.21.15.52:0/1937274606 learned_addr learned my addr 172.21.15.52:0/1937274606 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:21.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.357+0000 7ff0a1400640 1 --2- 172.21.15.52:0/1937274606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0a4143460 0x7ff0a41373e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:21.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.357+0000 7ff0a1e00640 1 --2- 172.21.15.52:0/1937274606 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0a40cdc50 0x7ff0a414ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:21.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.357+0000 7ff0a2800640 1 -- 172.21.15.52:0/1937274606 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0a40cdc50 msgr2=0x7ff0a414ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:21.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.357+0000 7ff0a2800640 1 --2- 172.21.15.52:0/1937274606 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0a40cdc50 0x7ff0a414ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:21.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.357+0000 7ff0a2800640 1 -- 172.21.15.52:0/1937274606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0a4143460 msgr2=0x7ff0a41373e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:21.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.357+0000 7ff0a2800640 1 --2- 172.21.15.52:0/1937274606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0a4143460 0x7ff0a41373e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:21.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.357+0000 7ff0a2800640 1 -- 172.21.15.52:0/1937274606 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0900094d0 con 0x7ff0a4143930 2024-03-12T17:06:21.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.357+0000 7ff0a1e00640 1 --2- 172.21.15.52:0/1937274606 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0a40cdc50 0x7ff0a414ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:21.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.358+0000 7ff0a2800640 1 --2- 172.21.15.52:0/1937274606 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0a4143930 0x7ff0a40acc60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff09c007e40 tx=0x7ff09c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:21.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.358+0000 7ff09a000640 1 -- 172.21.15.52:0/1937274606 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c014620 con 0x7ff0a4143930 2024-03-12T17:06:21.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.358+0000 7ff09a000640 1 -- 172.21.15.52:0/1937274606 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09c00ee90 con 0x7ff0a4143930 2024-03-12T17:06:21.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.358+0000 7ff09a000640 1 -- 172.21.15.52:0/1937274606 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c0149e0 con 0x7ff0a4143930 2024-03-12T17:06:21.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.358+0000 7ff0a3200640 1 -- 172.21.15.52:0/1937274606 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0a4138140 con 0x7ff0a4143930 2024-03-12T17:06:21.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.358+0000 7ff0a3200640 1 -- 172.21.15.52:0/1937274606 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a4138700 con 0x7ff0a4143930 2024-03-12T17:06:21.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.359+0000 7ff0a3200640 1 -- 172.21.15.52:0/1937274606 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0a4138be0 con 0x7ff0a4143930 2024-03-12T17:06:21.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.360+0000 7ff09a000640 1 -- 172.21.15.52:0/1937274606 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 4) v1 ==== 89081+0+0 (secure 0 0 0) 0x7ff09c014b40 con 0x7ff0a4143930 2024-03-12T17:06:21.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.361+0000 7ff09a000640 1 --2- 172.21.15.52:0/1937274606 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff08406ab70 0x7ff08406d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:21.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.361+0000 7ff09a000640 1 -- 172.21.15.52:0/1937274606 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff09c0898c0 con 0x7ff0a4143930 2024-03-12T17:06:21.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.362+0000 7ff0a1e00640 1 --2- 172.21.15.52:0/1937274606 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff08406ab70 0x7ff08406d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:21.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.363+0000 7ff0a1e00640 1 --2- 172.21.15.52:0/1937274606 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff08406ab70 0x7ff08406d030 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff094000f60 tx=0x7ff094009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:21.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.367+0000 7ff09a000640 1 -- 172.21.15.52:0/1937274606 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7ff09c054e30 con 0x7ff0a4143930 2024-03-12T17:06:21.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.604+0000 7ff0a3200640 1 -- 172.21.15.52:0/1937274606 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0a404e030 con 0x7ff0a4143930 2024-03-12T17:06:21.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.605+0000 7ff09a000640 1 -- 172.21.15.52:0/1937274606 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3527 (secure 0 0 0) 0x7ff09c058450 con 0x7ff0a4143930 2024-03-12T17:06:21.599 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:06:21.599 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2950099092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2950099092},{"type":"v1","addr":"172.21.15.52:6834","nonce":2950099092}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/846108993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":846108993},{"type":"v1","addr":"172.21.15.52:6835","nonce":846108993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6836/3002500679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3002500679},{"type":"v1","addr":"172.21.15.102:6836","nonce":3002500679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/3333451504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6835","nonce":3333451504},{"type":"v1","addr":"172.21.15.102:6837","nonce":3333451504}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-03-12T17:06:21.599 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 3 2024-03-12T17:06:21.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.608+0000 7ff0a3200640 1 -- 172.21.15.52:0/1937274606 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff08406ab70 msgr2=0x7ff08406d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:21.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.608+0000 7ff0a3200640 1 --2- 172.21.15.52:0/1937274606 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff08406ab70 0x7ff08406d030 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff094000f60 tx=0x7ff094009040 comp rx=0 tx=0).stop 2024-03-12T17:06:21.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.608+0000 7ff0a3200640 1 -- 172.21.15.52:0/1937274606 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0a4143930 msgr2=0x7ff0a40acc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:21.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.608+0000 7ff0a3200640 1 --2- 172.21.15.52:0/1937274606 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0a4143930 0x7ff0a40acc60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff09c007e40 tx=0x7ff09c00e4b0 comp rx=0 tx=0).stop 2024-03-12T17:06:21.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.608+0000 7ff0a3200640 1 -- 172.21.15.52:0/1937274606 shutdown_connections 2024-03-12T17:06:21.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.608+0000 7ff0a3200640 1 --2- 172.21.15.52:0/1937274606 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff08406ab70 0x7ff08406d030 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:21.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.608+0000 7ff0a3200640 1 --2- 172.21.15.52:0/1937274606 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0a4143930 0x7ff0a40acc60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:21.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.608+0000 7ff0a3200640 1 --2- 172.21.15.52:0/1937274606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0a4143460 0x7ff0a41373e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:21.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.608+0000 7ff0a3200640 1 --2- 172.21.15.52:0/1937274606 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0a40cdc50 0x7ff0a414ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:21.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.608+0000 7ff0a3200640 1 -- 172.21.15.52:0/1937274606 >> 172.21.15.52:0/1937274606 conn(0x7ff0a4065df0 msgr2=0x7ff0a413d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:21.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.608+0000 7ff0a3200640 1 -- 172.21.15.52:0/1937274606 shutdown_connections 2024-03-12T17:06:21.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.609+0000 7ff0a3200640 1 -- 172.21.15.52:0/1937274606 wait complete. 2024-03-12T17:06:21.621 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:06:21.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.850+0000 7f59ebe00640 1 Processor -- start 2024-03-12T17:06:21.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.850+0000 7f59ebe00640 1 -- start start 2024-03-12T17:06:21.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.852+0000 7f59ebe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f59ec0b8050 0x7f59ec0d1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:21.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.852+0000 7f59ebe00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59ec15ce90 con 0x7f59ec0cf5c0 2024-03-12T17:06:21.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.852+0000 7f59ebe00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59ec0cddf0 con 0x7f59ec15cab0 2024-03-12T17:06:21.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.852+0000 7f59ebe00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59ec0cd570 con 0x7f59ec0b8050 2024-03-12T17:06:21.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.852+0000 7f59eb400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f59ec0b8050 0x7f59ec0d1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:21.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.852+0000 7f59eb400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f59ec0b8050 0x7f59ec0d1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38104/0 (socket says 172.21.15.52:38104) 2024-03-12T17:06:21.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.852+0000 7f59eb400640 1 -- 172.21.15.52:0/2178591379 learned_addr learned my addr 172.21.15.52:0/2178591379 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:21.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.853+0000 7f59eb400640 1 -- 172.21.15.52:0/2178591379 >> v1:172.21.15.102:6789/0 conn(0x7f59ec15cab0 legacy=0x7f59ec0ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:21.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.853+0000 7f59eb400640 1 -- 172.21.15.52:0/2178591379 >> v1:172.21.15.52:6789/0 conn(0x7f59ec0cf5c0 legacy=0x7f59ec0d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:21.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.853+0000 7f59ea000640 1 -- 172.21.15.52:0/2178591379 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 123148718 0 0) 0x7f59ec15ce90 con 0x7f59ec0cf5c0 2024-03-12T17:06:21.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.853+0000 7f59eb400640 1 -- 172.21.15.52:0/2178591379 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ec0b84a0 con 0x7f59ec0b8050 2024-03-12T17:06:21.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.854+0000 7f59eb400640 1 --2- 172.21.15.52:0/2178591379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59ec0b8050 0x7f59ec0d1330 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f59e4009e50 tx=0x7f59e402c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b673d442031be96b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:21.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.854+0000 7f59ea000640 1 -- 172.21.15.52:0/2178591379 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e402cbb0 con 0x7f59ec0b8050 2024-03-12T17:06:21.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.854+0000 7f59ea000640 1 -- 172.21.15.52:0/2178591379 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e402cd10 con 0x7f59ec0b8050 2024-03-12T17:06:21.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.854+0000 7f59ea000640 1 -- 172.21.15.52:0/2178591379 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e4034610 con 0x7f59ec0b8050 2024-03-12T17:06:21.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.855+0000 7f59ebe00640 1 -- 172.21.15.52:0/2178591379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59ec0b8050 msgr2=0x7f59ec0d1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:21.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.855+0000 7f59ebe00640 1 --2- 172.21.15.52:0/2178591379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59ec0b8050 0x7f59ec0d1330 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f59e4009e50 tx=0x7f59e402c010 comp rx=0 tx=0).stop 2024-03-12T17:06:21.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.855+0000 7f59ebe00640 1 -- 172.21.15.52:0/2178591379 shutdown_connections 2024-03-12T17:06:21.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.855+0000 7f59ebe00640 1 --2- 172.21.15.52:0/2178591379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59ec0b8050 0x7f59ec0d1330 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:21.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.855+0000 7f59ebe00640 1 -- 172.21.15.52:0/2178591379 >> 172.21.15.52:0/2178591379 conn(0x7f59ec06ae90 msgr2=0x7f59ec06b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:21.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.855+0000 7f59ebe00640 1 -- 172.21.15.52:0/2178591379 shutdown_connections 2024-03-12T17:06:21.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.855+0000 7f59ebe00640 1 -- 172.21.15.52:0/2178591379 wait complete. 2024-03-12T17:06:21.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.857+0000 7f59ebe00640 1 Processor -- start 2024-03-12T17:06:21.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.857+0000 7f59ebe00640 1 -- start start 2024-03-12T17:06:21.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.858+0000 7f59ebe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59ec0b8050 0x7f59ec154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:21.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.858+0000 7f59ebe00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f59ec0cf5c0 0x7f59ec0b20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:21.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.858+0000 7f59ebe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59ec15cab0 0x7f59ec0b6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:21.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.858+0000 7f59ebe00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec155230 con 0x7f59ec15cab0 2024-03-12T17:06:21.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.858+0000 7f59ebe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec0c6a70 con 0x7f59ec0b8050 2024-03-12T17:06:21.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.858+0000 7f59ebe00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec0b2a20 con 0x7f59ec0cf5c0 2024-03-12T17:06:21.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.859+0000 7f59eb400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59ec15cab0 0x7f59ec0b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:21.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.859+0000 7f59eb400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59ec15cab0 0x7f59ec0b6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38112/0 (socket says 172.21.15.52:38112) 2024-03-12T17:06:21.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.859+0000 7f59eb400640 1 -- 172.21.15.52:0/240788236 learned_addr learned my addr 172.21.15.52:0/240788236 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:21.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.859+0000 7f59e3e00640 1 --2- 172.21.15.52:0/240788236 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f59ec0cf5c0 0x7f59ec0b20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:21.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.859+0000 7f59eaa00640 1 --2- 172.21.15.52:0/240788236 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59ec0b8050 0x7f59ec154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:21.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.859+0000 7f59eb400640 1 -- 172.21.15.52:0/240788236 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f59ec0cf5c0 msgr2=0x7f59ec0b20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:21.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.859+0000 7f59eb400640 1 --2- 172.21.15.52:0/240788236 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f59ec0cf5c0 0x7f59ec0b20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:21.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.859+0000 7f59eb400640 1 -- 172.21.15.52:0/240788236 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59ec0b8050 msgr2=0x7f59ec154cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:21.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.859+0000 7f59eb400640 1 --2- 172.21.15.52:0/240788236 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59ec0b8050 0x7f59ec154cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:21.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.859+0000 7f59eb400640 1 -- 172.21.15.52:0/240788236 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e4009b30 con 0x7f59ec15cab0 2024-03-12T17:06:21.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.860+0000 7f59eb400640 1 --2- 172.21.15.52:0/240788236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59ec15cab0 0x7f59ec0b6820 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f59e4036040 tx=0x7f59e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:21.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.860+0000 7f59e2a00640 1 -- 172.21.15.52:0/240788236 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e403e450 con 0x7f59ec15cab0 2024-03-12T17:06:21.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.860+0000 7f59e2a00640 1 -- 172.21.15.52:0/240788236 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e403e5d0 con 0x7f59ec15cab0 2024-03-12T17:06:21.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.860+0000 7f59ebe00640 1 -- 172.21.15.52:0/240788236 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59ec0b2d00 con 0x7f59ec15cab0 2024-03-12T17:06:21.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.860+0000 7f59e2a00640 1 -- 172.21.15.52:0/240788236 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e403e980 con 0x7f59ec15cab0 2024-03-12T17:06:21.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.860+0000 7f59ebe00640 1 -- 172.21.15.52:0/240788236 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59ec0b32c0 con 0x7f59ec15cab0 2024-03-12T17:06:21.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.861+0000 7f59ebe00640 1 -- 172.21.15.52:0/240788236 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59ec04e030 con 0x7f59ec15cab0 2024-03-12T17:06:21.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.862+0000 7f59e2a00640 1 -- 172.21.15.52:0/240788236 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 4) v1 ==== 89081+0+0 (secure 0 0 0) 0x7f59e403eae0 con 0x7f59ec15cab0 2024-03-12T17:06:21.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.863+0000 7f59e2a00640 1 --2- 172.21.15.52:0/240788236 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f59c806ab70 0x7f59c806d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:21.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.863+0000 7f59e2a00640 1 -- 172.21.15.52:0/240788236 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f59e40abdf0 con 0x7f59ec15cab0 2024-03-12T17:06:21.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.864+0000 7f59eaa00640 1 --2- 172.21.15.52:0/240788236 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f59c806ab70 0x7f59c806d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:21.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.865+0000 7f59eaa00640 1 --2- 172.21.15.52:0/240788236 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f59c806ab70 0x7f59c806d030 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f59dc000f60 tx=0x7f59dc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:21.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:21.868+0000 7f59e2a00640 1 -- 172.21.15.52:0/240788236 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f59e4077410 con 0x7f59ec15cab0 2024-03-12T17:06:22.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.074+0000 7f59ebe00640 1 -- 172.21.15.52:0/240788236 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59ec0c6bb0 con 0x7f59ec15cab0 2024-03-12T17:06:22.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.075+0000 7f59e2a00640 1 -- 172.21.15.52:0/240788236 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12382 (secure 0 0 0) 0x7f59e407aa30 con 0x7f59ec15cab0 2024-03-12T17:06:22.069 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:06:22.069 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":17,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:06:21.941424+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T17:06:19.218543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.733031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.775178+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.591522+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.820838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.829252+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.582481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:06:22.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.077+0000 7f59ebe00640 1 -- 172.21.15.52:0/240788236 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f59c806ab70 msgr2=0x7f59c806d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:22.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.077+0000 7f59ebe00640 1 --2- 172.21.15.52:0/240788236 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f59c806ab70 0x7f59c806d030 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f59dc000f60 tx=0x7f59dc009040 comp rx=0 tx=0).stop 2024-03-12T17:06:22.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.077+0000 7f59ebe00640 1 -- 172.21.15.52:0/240788236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59ec15cab0 msgr2=0x7f59ec0b6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:22.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.077+0000 7f59ebe00640 1 --2- 172.21.15.52:0/240788236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59ec15cab0 0x7f59ec0b6820 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f59e4036040 tx=0x7f59e4004a40 comp rx=0 tx=0).stop 2024-03-12T17:06:22.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.078+0000 7f59ebe00640 1 -- 172.21.15.52:0/240788236 shutdown_connections 2024-03-12T17:06:22.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.078+0000 7f59ebe00640 1 --2- 172.21.15.52:0/240788236 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f59c806ab70 0x7f59c806d030 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:22.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.078+0000 7f59ebe00640 1 --2- 172.21.15.52:0/240788236 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f59ec0cf5c0 0x7f59ec0b20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:22.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.078+0000 7f59ebe00640 1 --2- 172.21.15.52:0/240788236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59ec15cab0 0x7f59ec0b6820 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:22.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.078+0000 7f59ebe00640 1 --2- 172.21.15.52:0/240788236 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59ec0b8050 0x7f59ec154cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:22.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.078+0000 7f59ebe00640 1 -- 172.21.15.52:0/240788236 >> 172.21.15.52:0/240788236 conn(0x7f59ec06ae90 msgr2=0x7f59ec0b1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:22.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.078+0000 7f59ebe00640 1 -- 172.21.15.52:0/240788236 shutdown_connections 2024-03-12T17:06:22.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.078+0000 7f59ebe00640 1 -- 172.21.15.52:0/240788236 wait complete. 2024-03-12T17:06:22.090 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T17:06:19.218543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-12T17:06:22.090 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T17:06:22.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.307+0000 7fa23d200640 1 Processor -- start 2024-03-12T17:06:22.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.307+0000 7fa23d200640 1 -- start start 2024-03-12T17:06:22.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.309+0000 7fa23d200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa2380add30 0x7fa2380ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:22.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.309+0000 7fa23d200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa23806ce90 0x7fa23806d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:22.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.309+0000 7fa23d200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2380cf250 con 0x7fa2380ce930 2024-03-12T17:06:22.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.309+0000 7fa23d200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa23806c760 con 0x7fa23806ce90 2024-03-12T17:06:22.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.309+0000 7fa23d200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2380b42d0 con 0x7fa2380add30 2024-03-12T17:06:22.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.309+0000 7fa236800640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa23806ce90 0x7fa23806d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:22.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.309+0000 7fa236800640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa23806ce90 0x7fa23806d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38126/0 (socket says 172.21.15.52:38126) 2024-03-12T17:06:22.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.309+0000 7fa236800640 1 -- 172.21.15.52:0/1134910217 learned_addr learned my addr 172.21.15.52:0/1134910217 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:22.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.309+0000 7fa235400640 1 --2- 172.21.15.52:0/1134910217 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa2380add30 0x7fa2380ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:22.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.310+0000 7fa236800640 1 -- 172.21.15.52:0/1134910217 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa2380add30 msgr2=0x7fa2380ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:22.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.310+0000 7fa236800640 1 --2- 172.21.15.52:0/1134910217 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa2380add30 0x7fa2380ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:22.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.310+0000 7fa236800640 1 -- 172.21.15.52:0/1134910217 >> v1:172.21.15.102:6789/0 conn(0x7fa2380ce930 legacy=0x7fa2380cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:22.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.310+0000 7fa236800640 1 -- 172.21.15.52:0/1134910217 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2380cdba0 con 0x7fa23806ce90 2024-03-12T17:06:22.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.311+0000 7fa236800640 1 --2- 172.21.15.52:0/1134910217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa23806ce90 0x7fa23806d320 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa230009e50 tx=0x7fa23002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f75553f7a8a5fd87 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:22.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.311+0000 7fa234a00640 1 -- 172.21.15.52:0/1134910217 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23002cb00 con 0x7fa23806ce90 2024-03-12T17:06:22.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.311+0000 7fa234a00640 1 -- 172.21.15.52:0/1134910217 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa23002cc60 con 0x7fa23806ce90 2024-03-12T17:06:22.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.311+0000 7fa234a00640 1 -- 172.21.15.52:0/1134910217 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2300343f0 con 0x7fa23806ce90 2024-03-12T17:06:22.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.312+0000 7fa23d200640 1 -- 172.21.15.52:0/1134910217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa23806ce90 msgr2=0x7fa23806d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:22.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.312+0000 7fa23d200640 1 --2- 172.21.15.52:0/1134910217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa23806ce90 0x7fa23806d320 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa230009e50 tx=0x7fa23002bf60 comp rx=0 tx=0).stop 2024-03-12T17:06:22.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.312+0000 7fa23d200640 1 -- 172.21.15.52:0/1134910217 shutdown_connections 2024-03-12T17:06:22.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.312+0000 7fa23d200640 1 --2- 172.21.15.52:0/1134910217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa23806ce90 0x7fa23806d320 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:22.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.312+0000 7fa23d200640 1 --2- 172.21.15.52:0/1134910217 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa2380add30 0x7fa2380ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:22.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.312+0000 7fa23d200640 1 -- 172.21.15.52:0/1134910217 >> 172.21.15.52:0/1134910217 conn(0x7fa23806a0b0 msgr2=0x7fa23806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:22.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.312+0000 7fa23d200640 1 -- 172.21.15.52:0/1134910217 shutdown_connections 2024-03-12T17:06:22.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.312+0000 7fa23d200640 1 -- 172.21.15.52:0/1134910217 wait complete. 2024-03-12T17:06:22.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.314+0000 7fa23d200640 1 Processor -- start 2024-03-12T17:06:22.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.314+0000 7fa23d200640 1 -- start start 2024-03-12T17:06:22.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.315+0000 7fa23d200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa23806ce90 0x7fa23814bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:22.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.315+0000 7fa23d200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2380add30 0x7fa2381592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:22.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.315+0000 7fa23d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2380ce930 0x7fa2381f2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:22.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.315+0000 7fa23d200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23815afc0 con 0x7fa2380add30 2024-03-12T17:06:22.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.315+0000 7fa23d200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23814c2b0 con 0x7fa2380ce930 2024-03-12T17:06:22.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.315+0000 7fa23d200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa238158000 con 0x7fa23806ce90 2024-03-12T17:06:22.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.315+0000 7fa235400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2380add30 0x7fa2381592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:22.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.315+0000 7fa235400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2380add30 0x7fa2381592f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38138/0 (socket says 172.21.15.52:38138) 2024-03-12T17:06:22.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.315+0000 7fa235400640 1 -- 172.21.15.52:0/4105782813 learned_addr learned my addr 172.21.15.52:0/4105782813 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:22.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.316+0000 7fa235e00640 1 --2- 172.21.15.52:0/4105782813 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa23806ce90 0x7fa23814bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:22.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.316+0000 7fa236800640 1 --2- 172.21.15.52:0/4105782813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2380ce930 0x7fa2381f2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:22.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.316+0000 7fa235400640 1 -- 172.21.15.52:0/4105782813 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa23806ce90 msgr2=0x7fa23814bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:22.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.316+0000 7fa235400640 1 --2- 172.21.15.52:0/4105782813 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa23806ce90 0x7fa23814bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:22.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.316+0000 7fa235400640 1 -- 172.21.15.52:0/4105782813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2380ce930 msgr2=0x7fa2381f2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:22.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.316+0000 7fa235400640 1 --2- 172.21.15.52:0/4105782813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2380ce930 0x7fa2381f2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:22.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.316+0000 7fa235400640 1 -- 172.21.15.52:0/4105782813 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa230009b30 con 0x7fa2380add30 2024-03-12T17:06:22.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.316+0000 7fa235400640 1 --2- 172.21.15.52:0/4105782813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2380add30 0x7fa2381592f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa22400d800 tx=0x7fa22400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:22.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.316+0000 7fa22e000640 1 -- 172.21.15.52:0/4105782813 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22400b6c0 con 0x7fa2380add30 2024-03-12T17:06:22.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.316+0000 7fa22e000640 1 -- 172.21.15.52:0/4105782813 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22400b820 con 0x7fa2380add30 2024-03-12T17:06:22.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.316+0000 7fa22e000640 1 -- 172.21.15.52:0/4105782813 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22400bbb0 con 0x7fa2380add30 2024-03-12T17:06:22.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.316+0000 7fa23d200640 1 -- 172.21.15.52:0/4105782813 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa238158250 con 0x7fa2380add30 2024-03-12T17:06:22.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.317+0000 7fa23d200640 1 -- 172.21.15.52:0/4105782813 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa238158810 con 0x7fa2380add30 2024-03-12T17:06:22.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.318+0000 7fa23d200640 1 -- 172.21.15.52:0/4105782813 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa238066bb0 con 0x7fa2380add30 2024-03-12T17:06:22.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.319+0000 7fa22e000640 1 -- 172.21.15.52:0/4105782813 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 4) v1 ==== 89081+0+0 (secure 0 0 0) 0x7fa22400bd10 con 0x7fa2380add30 2024-03-12T17:06:22.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.319+0000 7fa22e000640 1 --2- 172.21.15.52:0/4105782813 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa21806ab70 0x7fa21806d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:22.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.320+0000 7fa22e000640 1 -- 172.21.15.52:0/4105782813 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa224089490 con 0x7fa2380add30 2024-03-12T17:06:22.319 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-12T17:06:22.324 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-12T17:06:22.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.328+0000 7fa22e000640 1 -- 172.21.15.52:0/4105782813 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fa224054aa0 con 0x7fa2380add30 2024-03-12T17:06:22.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.330+0000 7fa235e00640 1 --2- 172.21.15.52:0/4105782813 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa21806ab70 0x7fa21806d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:22.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.331+0000 7fa235e00640 1 --2- 172.21.15.52:0/4105782813 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa21806ab70 0x7fa21806d030 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa228000f60 tx=0x7fa228009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:22.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.546+0000 7fa23d200640 1 -- 172.21.15.52:0/4105782813 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa238073390 con 0x7fa2380add30 2024-03-12T17:06:22.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.547+0000 7fa22e000640 1 -- 172.21.15.52:0/4105782813 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa2240580c0 con 0x7fa2380add30 2024-03-12T17:06:22.540 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-03-12T17:06:22.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.549+0000 7fa23d200640 1 -- 172.21.15.52:0/4105782813 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa21806ab70 msgr2=0x7fa21806d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:22.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.549+0000 7fa23d200640 1 --2- 172.21.15.52:0/4105782813 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa21806ab70 0x7fa21806d030 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa228000f60 tx=0x7fa228009040 comp rx=0 tx=0).stop 2024-03-12T17:06:22.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.549+0000 7fa23d200640 1 -- 172.21.15.52:0/4105782813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2380add30 msgr2=0x7fa2381592f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:22.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.549+0000 7fa23d200640 1 --2- 172.21.15.52:0/4105782813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2380add30 0x7fa2381592f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa22400d800 tx=0x7fa22400dcc0 comp rx=0 tx=0).stop 2024-03-12T17:06:22.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.549+0000 7fa23d200640 1 -- 172.21.15.52:0/4105782813 shutdown_connections 2024-03-12T17:06:22.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.550+0000 7fa23d200640 1 --2- 172.21.15.52:0/4105782813 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa21806ab70 0x7fa21806d030 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:22.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.550+0000 7fa23d200640 1 --2- 172.21.15.52:0/4105782813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2380ce930 0x7fa2381f2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:22.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.550+0000 7fa23d200640 1 --2- 172.21.15.52:0/4105782813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2380add30 0x7fa2381592f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:22.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.550+0000 7fa23d200640 1 --2- 172.21.15.52:0/4105782813 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa23806ce90 0x7fa23814bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:22.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.550+0000 7fa23d200640 1 -- 172.21.15.52:0/4105782813 >> 172.21.15.52:0/4105782813 conn(0x7fa23806a0b0 msgr2=0x7fa23815cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:22.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.550+0000 7fa23d200640 1 -- 172.21.15.52:0/4105782813 shutdown_connections 2024-03-12T17:06:22.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.550+0000 7fa23d200640 1 -- 172.21.15.52:0/4105782813 wait complete. 2024-03-12T17:06:22.567 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-12T17:06:22.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.775+0000 7f60d0800640 1 Processor -- start 2024-03-12T17:06:22.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.775+0000 7f60d0800640 1 -- start start 2024-03-12T17:06:22.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.776+0000 7f60d0800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f60c815cda0 0x7f60c80cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:22.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.776+0000 7f60d0800640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60c80d1830 con 0x7f60c80b5a30 2024-03-12T17:06:22.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.776+0000 7f60d0800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60c80b1f20 con 0x7f60c80b4a60 2024-03-12T17:06:22.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.776+0000 7f60d0800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60c80d0c20 con 0x7f60c815cda0 2024-03-12T17:06:22.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.777+0000 7f60cd400640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f60c80b5a30 0x7f60c80b4350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:53344/0 (socket says 172.21.15.52:53344) 2024-03-12T17:06:22.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.777+0000 7f60cd400640 1 -- 172.21.15.52:0/2660019813 learned_addr learned my addr 172.21.15.52:0/2660019813 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:06:22.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.777+0000 7f60cde00640 1 --2- 172.21.15.52:0/2660019813 >> v2:172.21.15.102:3300/0 conn(0x7f60c815cda0 0x7f60c80cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:22.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.777+0000 7f60cca00640 1 -- 172.21.15.52:0/2660019813 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 488604492 0 0) 0x7f60c80d1830 con 0x7f60c80b5a30 2024-03-12T17:06:22.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.777+0000 7f60cca00640 1 -- 172.21.15.52:0/2660019813 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60b0003520 con 0x7f60c80b5a30 2024-03-12T17:06:22.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.778+0000 7f60cca00640 1 -- 172.21.15.52:0/2660019813 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1071578369 0 0) 0x7f60b0003520 con 0x7f60c80b5a30 2024-03-12T17:06:22.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.778+0000 7f60cca00640 1 -- 172.21.15.52:0/2660019813 >> v2:172.21.15.102:3300/0 conn(0x7f60c815cda0 msgr2=0x7f60c80cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:22.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.778+0000 7f60cca00640 1 --2- 172.21.15.52:0/2660019813 >> v2:172.21.15.102:3300/0 conn(0x7f60c815cda0 0x7f60c80cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:22.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.778+0000 7f60cca00640 1 -- 172.21.15.52:0/2660019813 >> v1:172.21.15.102:6789/0 conn(0x7f60c80b4a60 legacy=0x7f60c80d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:22.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.778+0000 7f60cca00640 1 -- 172.21.15.52:0/2660019813 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c80d0f00 con 0x7f60c80b5a30 2024-03-12T17:06:22.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.778+0000 7f60cca00640 1 -- 172.21.15.52:0/2660019813 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f60b8001670 con 0x7f60c80b5a30 2024-03-12T17:06:22.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.778+0000 7f60cca00640 1 -- 172.21.15.52:0/2660019813 >> v1:172.21.15.52:6789/0 conn(0x7f60c80b5a30 legacy=0x7f60c80b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:22.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.778+0000 7f60cca00640 1 --2- 172.21.15.52:0/2660019813 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f60b00039c0 0x7f60b0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:22.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.778+0000 7f60cca00640 1 --2- 172.21.15.52:0/2660019813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60b00047c0 0x7f60b0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:22.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.778+0000 7f60cca00640 1 --2- 172.21.15.52:0/2660019813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60b0005180 0x7f60b000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:22.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.778+0000 7f60cca00640 1 -- 172.21.15.52:0/2660019813 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c80d0c20 con 0x7f60b0005180 2024-03-12T17:06:22.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.778+0000 7f60cca00640 1 -- 172.21.15.52:0/2660019813 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b000c080 con 0x7f60b00047c0 2024-03-12T17:06:22.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.778+0000 7f60cca00640 1 -- 172.21.15.52:0/2660019813 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60b000c240 con 0x7f60b00039c0 2024-03-12T17:06:22.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.778+0000 7f60ce800640 1 --2- 172.21.15.52:0/2660019813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60b0005180 0x7f60b000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:22.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.779+0000 7f60cd400640 1 --2- 172.21.15.52:0/2660019813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60b00047c0 0x7f60b0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:22.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.779+0000 7f60cde00640 1 --2- 172.21.15.52:0/2660019813 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f60b00039c0 0x7f60b0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:22.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.779+0000 7f60ce800640 1 -- 172.21.15.52:0/2660019813 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f60b00039c0 msgr2=0x7f60b0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:22.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.779+0000 7f60ce800640 1 --2- 172.21.15.52:0/2660019813 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f60b00039c0 0x7f60b0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:22.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.779+0000 7f60ce800640 1 -- 172.21.15.52:0/2660019813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60b00047c0 msgr2=0x7f60b0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:22.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.779+0000 7f60ce800640 1 --2- 172.21.15.52:0/2660019813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60b00047c0 0x7f60b0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:22.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.779+0000 7f60ce800640 1 -- 172.21.15.52:0/2660019813 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b0003520 con 0x7f60b0005180 2024-03-12T17:06:22.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.780+0000 7f60ce800640 1 -- 172.21.15.52:0/2660019813 reap_dead start 2024-03-12T17:06:22.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.780+0000 7f60ce800640 1 --2- 172.21.15.52:0/2660019813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60b0005180 0x7f60b000b980 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f60c808b4f0 tx=0x7f60b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd5b9ccbddd6594 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:22.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.780+0000 7f60cca00640 1 -- 172.21.15.52:0/2660019813 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c4003b70 con 0x7f60b0005180 2024-03-12T17:06:22.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.780+0000 7f60cca00640 1 -- 172.21.15.52:0/2660019813 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c4003cd0 con 0x7f60b0005180 2024-03-12T17:06:22.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.780+0000 7f60cca00640 1 -- 172.21.15.52:0/2660019813 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c4005310 con 0x7f60b0005180 2024-03-12T17:06:22.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.781+0000 7f60d0800640 1 -- 172.21.15.52:0/2660019813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60b0005180 msgr2=0x7f60b000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:22.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.781+0000 7f60d0800640 1 --2- 172.21.15.52:0/2660019813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60b0005180 0x7f60b000b980 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f60c808b4f0 tx=0x7f60b00057f0 comp rx=0 tx=0).stop 2024-03-12T17:06:22.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.781+0000 7f60d0800640 1 -- 172.21.15.52:0/2660019813 shutdown_connections 2024-03-12T17:06:22.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.781+0000 7f60d0800640 1 --2- 172.21.15.52:0/2660019813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60b0005180 0x7f60b000b980 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:22.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.781+0000 7f60d0800640 1 -- 172.21.15.52:0/2660019813 >> 172.21.15.52:0/2660019813 conn(0x7f60c80cf350 msgr2=0x7f60c80cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:22.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.781+0000 7f60d0800640 1 -- 172.21.15.52:0/2660019813 shutdown_connections 2024-03-12T17:06:22.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.781+0000 7f60d0800640 1 -- 172.21.15.52:0/2660019813 wait complete. 2024-03-12T17:06:22.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.783+0000 7f60d0800640 1 Processor -- start 2024-03-12T17:06:22.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.783+0000 7f60d0800640 1 -- start start 2024-03-12T17:06:22.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.784+0000 7f60d0800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60b0005180 0x7f60c8156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:22.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.784+0000 7f60d0800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f60c8155950 0x7f60c8156e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:22.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.784+0000 7f60d0800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60c81573a0 0x7f60c81f5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:22.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.784+0000 7f60d0800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c8157820 con 0x7f60b0005180 2024-03-12T17:06:22.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.784+0000 7f60d0800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c81f20a0 con 0x7f60c81573a0 2024-03-12T17:06:22.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.784+0000 7f60d0800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c80c63c0 con 0x7f60c8155950 2024-03-12T17:06:22.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.784+0000 7f60cde00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60b0005180 0x7f60c8156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:22.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.784+0000 7f60cde00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60b0005180 0x7f60c8156550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38142/0 (socket says 172.21.15.52:38142) 2024-03-12T17:06:22.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.784+0000 7f60cde00640 1 -- 172.21.15.52:0/1770848843 learned_addr learned my addr 172.21.15.52:0/1770848843 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:22.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.785+0000 7f60cd400640 1 --2- 172.21.15.52:0/1770848843 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f60c8155950 0x7f60c8156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:22.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.785+0000 7f60ce800640 1 --2- 172.21.15.52:0/1770848843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60c81573a0 0x7f60c81f5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:22.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.785+0000 7f60cde00640 1 -- 172.21.15.52:0/1770848843 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f60c8155950 msgr2=0x7f60c8156e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:22.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.785+0000 7f60cde00640 1 --2- 172.21.15.52:0/1770848843 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f60c8155950 0x7f60c8156e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:22.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.785+0000 7f60cde00640 1 -- 172.21.15.52:0/1770848843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60c81573a0 msgr2=0x7f60c81f5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:22.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.785+0000 7f60cde00640 1 --2- 172.21.15.52:0/1770848843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60c81573a0 0x7f60c81f5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:22.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.785+0000 7f60cde00640 1 -- 172.21.15.52:0/1770848843 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c80c66a0 con 0x7f60b0005180 2024-03-12T17:06:22.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.785+0000 7f60cde00640 1 --2- 172.21.15.52:0/1770848843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60b0005180 0x7f60c8156550 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f60bc011570 tx=0x7f60bc011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:22.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.785+0000 7f60c2000640 1 -- 172.21.15.52:0/1770848843 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60bc015930 con 0x7f60b0005180 2024-03-12T17:06:22.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.786+0000 7f60c2000640 1 -- 172.21.15.52:0/1770848843 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60bc015a90 con 0x7f60b0005180 2024-03-12T17:06:22.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.786+0000 7f60d0800640 1 -- 172.21.15.52:0/1770848843 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c8156a90 con 0x7f60b0005180 2024-03-12T17:06:22.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.786+0000 7f60c2000640 1 -- 172.21.15.52:0/1770848843 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60bc015e60 con 0x7f60b0005180 2024-03-12T17:06:22.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.786+0000 7f60d0800640 1 -- 172.21.15.52:0/1770848843 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c8159400 con 0x7f60b0005180 2024-03-12T17:06:22.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.787+0000 7f60d0800640 1 -- 172.21.15.52:0/1770848843 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60c8073390 con 0x7f60b0005180 2024-03-12T17:06:22.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.788+0000 7f60c2000640 1 -- 172.21.15.52:0/1770848843 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 4) v1 ==== 89081+0+0 (secure 0 0 0) 0x7f60bc00f460 con 0x7f60b0005180 2024-03-12T17:06:22.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.789+0000 7f60c2000640 1 --2- 172.21.15.52:0/1770848843 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f60ac06aba0 0x7f60ac06d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:22.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.789+0000 7f60c2000640 1 -- 172.21.15.52:0/1770848843 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f60bc0896b0 con 0x7f60b0005180 2024-03-12T17:06:22.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.790+0000 7f60cd400640 1 --2- 172.21.15.52:0/1770848843 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f60ac06aba0 0x7f60ac06d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:22.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.791+0000 7f60cd400640 1 --2- 172.21.15.52:0/1770848843 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f60ac06aba0 0x7f60ac06d060 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f60b8001eb0 tx=0x7f60b8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:22.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:22.793+0000 7f60c2000640 1 -- 172.21.15.52:0/1770848843 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f60bc054d50 con 0x7f60b0005180 2024-03-12T17:06:23.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.019+0000 7f60d0800640 1 -- 172.21.15.52:0/1770848843 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f60c80c5e30 con 0x7f60b0005180 2024-03-12T17:06:23.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.020+0000 7f60c2000640 1 -- 172.21.15.52:0/1770848843 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7f60bc058370 con 0x7f60b0005180 2024-03-12T17:06:23.014 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:06:23.014 INFO:teuthology.orchestra.run.smithi052.stdout:[ 2024-03-12T17:06:23.014 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-12T17:06:23.014 INFO:teuthology.orchestra.run.smithi052.stdout: "poolnum": 1, 2024-03-12T17:06:23.014 INFO:teuthology.orchestra.run.smithi052.stdout: "poolname": ".mgr" 2024-03-12T17:06:23.014 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-12T17:06:23.014 INFO:teuthology.orchestra.run.smithi052.stdout:] 2024-03-12T17:06:23.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.023+0000 7f60d0800640 1 -- 172.21.15.52:0/1770848843 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f60ac06aba0 msgr2=0x7f60ac06d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:23.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.023+0000 7f60d0800640 1 --2- 172.21.15.52:0/1770848843 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f60ac06aba0 0x7f60ac06d060 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f60b8001eb0 tx=0x7f60b8009040 comp rx=0 tx=0).stop 2024-03-12T17:06:23.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.023+0000 7f60d0800640 1 -- 172.21.15.52:0/1770848843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60b0005180 msgr2=0x7f60c8156550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:23.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.023+0000 7f60d0800640 1 --2- 172.21.15.52:0/1770848843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60b0005180 0x7f60c8156550 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f60bc011570 tx=0x7f60bc011a30 comp rx=0 tx=0).stop 2024-03-12T17:06:23.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.023+0000 7f60d0800640 1 -- 172.21.15.52:0/1770848843 shutdown_connections 2024-03-12T17:06:23.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.023+0000 7f60d0800640 1 --2- 172.21.15.52:0/1770848843 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f60ac06aba0 0x7f60ac06d060 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:23.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.023+0000 7f60d0800640 1 --2- 172.21.15.52:0/1770848843 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f60c8155950 0x7f60c8156e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:23.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.023+0000 7f60d0800640 1 --2- 172.21.15.52:0/1770848843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60c81573a0 0x7f60c81f5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:23.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.023+0000 7f60d0800640 1 --2- 172.21.15.52:0/1770848843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60b0005180 0x7f60c8156550 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:23.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.023+0000 7f60d0800640 1 -- 172.21.15.52:0/1770848843 >> 172.21.15.52:0/1770848843 conn(0x7f60c80cf350 msgr2=0x7f60c80b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:23.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.023+0000 7f60d0800640 1 -- 172.21.15.52:0/1770848843 shutdown_connections 2024-03-12T17:06:23.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.023+0000 7f60d0800640 1 -- 172.21.15.52:0/1770848843 wait complete. 2024-03-12T17:06:23.036 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-12T17:06:23.036 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-12T17:06:23.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.246+0000 7fc878e00640 1 Processor -- start 2024-03-12T17:06:23.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.247+0000 7fc878e00640 1 -- start start 2024-03-12T17:06:23.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.249+0000 7fc878e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc8740ce930 0x7fc8740add30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:23.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.249+0000 7fc878e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc87406ce90 0x7fc87406d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:23.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.249+0000 7fc878e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8740cdba0 con 0x7fc8740cd3e0 2024-03-12T17:06:23.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.249+0000 7fc878e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc87406c760 con 0x7fc8740ce930 2024-03-12T17:06:23.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.249+0000 7fc878e00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8740b42d0 con 0x7fc87406ce90 2024-03-12T17:06:23.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.249+0000 7fc871e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc8740ce930 0x7fc8740add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:23.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.249+0000 7fc871e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc8740ce930 0x7fc8740add30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:52626/0 (socket says 172.21.15.52:52626) 2024-03-12T17:06:23.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.249+0000 7fc871e00640 1 -- 172.21.15.52:0/428858997 learned_addr learned my addr 172.21.15.52:0/428858997 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:23.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.249+0000 7fc872800640 1 --2- 172.21.15.52:0/428858997 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc87406ce90 0x7fc87406d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:23.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.250+0000 7fc871e00640 1 -- 172.21.15.52:0/428858997 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc87406ce90 msgr2=0x7fc87406d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:23.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.250+0000 7fc871e00640 1 --2- 172.21.15.52:0/428858997 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc87406ce90 0x7fc87406d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:23.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.250+0000 7fc871e00640 1 -- 172.21.15.52:0/428858997 >> v1:172.21.15.102:6789/0 conn(0x7fc8740cd3e0 legacy=0x7fc8740ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:23.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.250+0000 7fc871e00640 1 -- 172.21.15.52:0/428858997 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8740cde80 con 0x7fc8740ce930 2024-03-12T17:06:23.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.252+0000 7fc871e00640 1 --2- 172.21.15.52:0/428858997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8740ce930 0x7fc8740add30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc864009870 tx=0x7fc86402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ae5638ba840ce4e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:23.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.252+0000 7fc870a00640 1 -- 172.21.15.52:0/428858997 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc864030840 con 0x7fc8740ce930 2024-03-12T17:06:23.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.252+0000 7fc870a00640 1 -- 172.21.15.52:0/428858997 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8640309a0 con 0x7fc8740ce930 2024-03-12T17:06:23.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.252+0000 7fc870a00640 1 -- 172.21.15.52:0/428858997 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc864030d30 con 0x7fc8740ce930 2024-03-12T17:06:23.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.253+0000 7fc878e00640 1 -- 172.21.15.52:0/428858997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8740ce930 msgr2=0x7fc8740add30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:23.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.253+0000 7fc878e00640 1 --2- 172.21.15.52:0/428858997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8740ce930 0x7fc8740add30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc864009870 tx=0x7fc86402b980 comp rx=0 tx=0).stop 2024-03-12T17:06:23.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.253+0000 7fc878e00640 1 -- 172.21.15.52:0/428858997 shutdown_connections 2024-03-12T17:06:23.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.253+0000 7fc878e00640 1 --2- 172.21.15.52:0/428858997 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc87406ce90 0x7fc87406d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:23.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.253+0000 7fc878e00640 1 --2- 172.21.15.52:0/428858997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8740ce930 0x7fc8740add30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:23.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.253+0000 7fc878e00640 1 -- 172.21.15.52:0/428858997 >> 172.21.15.52:0/428858997 conn(0x7fc87406a0b0 msgr2=0x7fc87406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:23.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.253+0000 7fc878e00640 1 -- 172.21.15.52:0/428858997 shutdown_connections 2024-03-12T17:06:23.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.253+0000 7fc878e00640 1 -- 172.21.15.52:0/428858997 wait complete. 2024-03-12T17:06:23.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.255+0000 7fc878e00640 1 Processor -- start 2024-03-12T17:06:23.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.255+0000 7fc878e00640 1 -- start start 2024-03-12T17:06:23.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.256+0000 7fc878e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc87406ce90 0x7fc8741488a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:23.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.256+0000 7fc878e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc8740cd3e0 0x7fc8741efe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:23.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.257+0000 7fc878e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8740ce930 0x7fc87414ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:23.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.257+0000 7fc878e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc874148de0 con 0x7fc87406ce90 2024-03-12T17:06:23.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.257+0000 7fc878e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87414c7d0 con 0x7fc8740ce930 2024-03-12T17:06:23.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.257+0000 7fc878e00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc87414cae0 con 0x7fc8740cd3e0 2024-03-12T17:06:23.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.257+0000 7fc871e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc87406ce90 0x7fc8741488a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:23.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.257+0000 7fc871e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc87406ce90 0x7fc8741488a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38144/0 (socket says 172.21.15.52:38144) 2024-03-12T17:06:23.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.257+0000 7fc871e00640 1 -- 172.21.15.52:0/2824341586 learned_addr learned my addr 172.21.15.52:0/2824341586 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:23.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.257+0000 7fc872800640 1 --2- 172.21.15.52:0/2824341586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8740ce930 0x7fc87414ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:23.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.257+0000 7fc871400640 1 --2- 172.21.15.52:0/2824341586 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc8740cd3e0 0x7fc8741efe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:23.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.257+0000 7fc871e00640 1 -- 172.21.15.52:0/2824341586 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc8740cd3e0 msgr2=0x7fc8741efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:23.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.257+0000 7fc871e00640 1 --2- 172.21.15.52:0/2824341586 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc8740cd3e0 0x7fc8741efe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:23.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.257+0000 7fc871e00640 1 -- 172.21.15.52:0/2824341586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8740ce930 msgr2=0x7fc87414ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:23.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.257+0000 7fc871e00640 1 --2- 172.21.15.52:0/2824341586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8740ce930 0x7fc87414ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:23.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.257+0000 7fc871e00640 1 -- 172.21.15.52:0/2824341586 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8640094d0 con 0x7fc87406ce90 2024-03-12T17:06:23.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.257+0000 7fc872800640 1 --2- 172.21.15.52:0/2824341586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8740ce930 0x7fc87414ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:06:23.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.258+0000 7fc871e00640 1 --2- 172.21.15.52:0/2824341586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc87406ce90 0x7fc8741488a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc864000c00 tx=0x7fc864004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:23.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.258+0000 7fc86a000640 1 -- 172.21.15.52:0/2824341586 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc864039a90 con 0x7fc87406ce90 2024-03-12T17:06:23.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.258+0000 7fc86a000640 1 -- 172.21.15.52:0/2824341586 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc864039bf0 con 0x7fc87406ce90 2024-03-12T17:06:23.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.258+0000 7fc86a000640 1 -- 172.21.15.52:0/2824341586 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc864043410 con 0x7fc87406ce90 2024-03-12T17:06:23.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.258+0000 7fc878e00640 1 -- 172.21.15.52:0/2824341586 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc87414cdc0 con 0x7fc87406ce90 2024-03-12T17:06:23.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.258+0000 7fc878e00640 1 -- 172.21.15.52:0/2824341586 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc87414d380 con 0x7fc87406ce90 2024-03-12T17:06:23.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.259+0000 7fc878e00640 1 -- 172.21.15.52:0/2824341586 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc874073390 con 0x7fc87406ce90 2024-03-12T17:06:23.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.261+0000 7fc86a000640 1 -- 172.21.15.52:0/2824341586 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 4) v1 ==== 89081+0+0 (secure 0 0 0) 0x7fc86402f390 con 0x7fc87406ce90 2024-03-12T17:06:23.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.262+0000 7fc86a000640 1 --2- 172.21.15.52:0/2824341586 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc85006ab70 0x7fc85006d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:23.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.263+0000 7fc86a000640 1 -- 172.21.15.52:0/2824341586 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc8640ab3a0 con 0x7fc87406ce90 2024-03-12T17:06:23.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.263+0000 7fc871400640 1 --2- 172.21.15.52:0/2824341586 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc85006ab70 0x7fc85006d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:23.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.264+0000 7fc871400640 1 --2- 172.21.15.52:0/2824341586 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc85006ab70 0x7fc85006d030 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc8740ca9f0 tx=0x7fc85c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:23.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.267+0000 7fc86a000640 1 -- 172.21.15.52:0/2824341586 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fc864076ac0 con 0x7fc87406ce90 2024-03-12T17:06:23.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.471+0000 7fc878e00640 1 -- 172.21.15.52:0/2824341586 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc87406c540 con 0x7fc87406ce90 2024-03-12T17:06:23.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:23.972+0000 7fc86a000640 1 -- 172.21.15.52:0/2824341586 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc8640ab0c0 con 0x7fc87406ce90 2024-03-12T17:06:24.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.406+0000 7fc86a000640 1 -- 172.21.15.52:0/2824341586 <== mon.0 v2:172.21.15.52:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc86407e060 con 0x7fc87406ce90 2024-03-12T17:06:24.400 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_metadata' created 2024-03-12T17:06:24.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.410+0000 7fc878e00640 1 -- 172.21.15.52:0/2824341586 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc85006ab70 msgr2=0x7fc85006d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:24.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.410+0000 7fc878e00640 1 --2- 172.21.15.52:0/2824341586 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc85006ab70 0x7fc85006d030 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc8740ca9f0 tx=0x7fc85c009040 comp rx=0 tx=0).stop 2024-03-12T17:06:24.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.410+0000 7fc878e00640 1 -- 172.21.15.52:0/2824341586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc87406ce90 msgr2=0x7fc8741488a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:24.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.410+0000 7fc878e00640 1 --2- 172.21.15.52:0/2824341586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc87406ce90 0x7fc8741488a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc864000c00 tx=0x7fc864004730 comp rx=0 tx=0).stop 2024-03-12T17:06:24.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.411+0000 7fc878e00640 1 -- 172.21.15.52:0/2824341586 shutdown_connections 2024-03-12T17:06:24.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.411+0000 7fc878e00640 1 --2- 172.21.15.52:0/2824341586 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc85006ab70 0x7fc85006d030 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:24.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.411+0000 7fc878e00640 1 --2- 172.21.15.52:0/2824341586 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc8740cd3e0 0x7fc8741efe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:24.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.411+0000 7fc878e00640 1 --2- 172.21.15.52:0/2824341586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8740ce930 0x7fc87414ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:24.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.411+0000 7fc878e00640 1 --2- 172.21.15.52:0/2824341586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc87406ce90 0x7fc8741488a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:24.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.411+0000 7fc878e00640 1 -- 172.21.15.52:0/2824341586 >> 172.21.15.52:0/2824341586 conn(0x7fc87406a0b0 msgr2=0x7fc8740afd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:24.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.411+0000 7fc878e00640 1 -- 172.21.15.52:0/2824341586 shutdown_connections 2024-03-12T17:06:24.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.411+0000 7fc878e00640 1 -- 172.21.15.52:0/2824341586 wait complete. 2024-03-12T17:06:24.428 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-03-12T17:06:24.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.640+0000 7fd686200640 1 Processor -- start 2024-03-12T17:06:24.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.640+0000 7fd686200640 1 -- start start 2024-03-12T17:06:24.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.642+0000 7fd686200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd680098a40 0x7fd680098e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:24.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.642+0000 7fd686200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd6800cf120 0x7fd68015e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:24.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.642+0000 7fd686200640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd680099360 con 0x7fd68015eb50 2024-03-12T17:06:24.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.642+0000 7fd686200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6800cb5d0 con 0x7fd6800cf120 2024-03-12T17:06:24.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.642+0000 7fd686200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6800cb8b0 con 0x7fd680098a40 2024-03-12T17:06:24.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.643+0000 7fd67ea00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd6800cf120 0x7fd68015e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:24.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.643+0000 7fd67ea00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd6800cf120 0x7fd68015e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38158/0 (socket says 172.21.15.52:38158) 2024-03-12T17:06:24.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.643+0000 7fd67ea00640 1 -- 172.21.15.52:0/3980116144 learned_addr learned my addr 172.21.15.52:0/3980116144 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:24.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.643+0000 7fd67f400640 1 --2- 172.21.15.52:0/3980116144 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd680098a40 0x7fd680098e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:24.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.643+0000 7fd67ea00640 1 -- 172.21.15.52:0/3980116144 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd680098a40 msgr2=0x7fd680098e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:24.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.643+0000 7fd67ea00640 1 --2- 172.21.15.52:0/3980116144 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd680098a40 0x7fd680098e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:24.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.643+0000 7fd67ea00640 1 -- 172.21.15.52:0/3980116144 >> v1:172.21.15.52:6789/0 conn(0x7fd68015eb50 legacy=0x7fd6800b34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:24.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.643+0000 7fd67ea00640 1 -- 172.21.15.52:0/3980116144 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6800cbb90 con 0x7fd6800cf120 2024-03-12T17:06:24.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.644+0000 7fd67ea00640 1 --2- 172.21.15.52:0/3980116144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6800cf120 0x7fd68015e610 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd66c009870 tx=0x7fd66c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5b66b62dfe85c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:24.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.644+0000 7fd67e000640 1 -- 172.21.15.52:0/3980116144 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c030840 con 0x7fd6800cf120 2024-03-12T17:06:24.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.645+0000 7fd67e000640 1 -- 172.21.15.52:0/3980116144 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66c0309a0 con 0x7fd6800cf120 2024-03-12T17:06:24.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.645+0000 7fd67e000640 1 -- 172.21.15.52:0/3980116144 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c030d30 con 0x7fd6800cf120 2024-03-12T17:06:24.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.645+0000 7fd686200640 1 -- 172.21.15.52:0/3980116144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6800cf120 msgr2=0x7fd68015e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:24.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.645+0000 7fd686200640 1 --2- 172.21.15.52:0/3980116144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6800cf120 0x7fd68015e610 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd66c009870 tx=0x7fd66c02b980 comp rx=0 tx=0).stop 2024-03-12T17:06:24.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.645+0000 7fd686200640 1 -- 172.21.15.52:0/3980116144 shutdown_connections 2024-03-12T17:06:24.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.645+0000 7fd686200640 1 --2- 172.21.15.52:0/3980116144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6800cf120 0x7fd68015e610 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:24.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.645+0000 7fd686200640 1 --2- 172.21.15.52:0/3980116144 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd680098a40 0x7fd680098e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:24.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.645+0000 7fd686200640 1 -- 172.21.15.52:0/3980116144 >> 172.21.15.52:0/3980116144 conn(0x7fd6800b2180 msgr2=0x7fd6800b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:24.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.645+0000 7fd686200640 1 -- 172.21.15.52:0/3980116144 shutdown_connections 2024-03-12T17:06:24.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.645+0000 7fd686200640 1 -- 172.21.15.52:0/3980116144 wait complete. 2024-03-12T17:06:24.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.647+0000 7fd686200640 1 Processor -- start 2024-03-12T17:06:24.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.647+0000 7fd686200640 1 -- start start 2024-03-12T17:06:24.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.648+0000 7fd686200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd680098a40 0x7fd68015c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:24.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.648+0000 7fd686200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd6800cf120 0x7fd6800c5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:24.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.648+0000 7fd686200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd68015eb50 0x7fd68006f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:24.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.648+0000 7fd686200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68009d0c0 con 0x7fd68015eb50 2024-03-12T17:06:24.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.648+0000 7fd686200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68009b060 con 0x7fd680098a40 2024-03-12T17:06:24.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.648+0000 7fd686200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd68009b300 con 0x7fd6800cf120 2024-03-12T17:06:24.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.649+0000 7fd67fe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd68015eb50 0x7fd68006f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:24.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.649+0000 7fd67fe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd68015eb50 0x7fd68006f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38162/0 (socket says 172.21.15.52:38162) 2024-03-12T17:06:24.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.649+0000 7fd67fe00640 1 -- 172.21.15.52:0/2204807361 learned_addr learned my addr 172.21.15.52:0/2204807361 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:24.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.649+0000 7fd67ea00640 1 --2- 172.21.15.52:0/2204807361 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd6800cf120 0x7fd6800c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:24.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.649+0000 7fd67fe00640 1 -- 172.21.15.52:0/2204807361 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd6800cf120 msgr2=0x7fd6800c5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:24.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.649+0000 7fd67fe00640 1 --2- 172.21.15.52:0/2204807361 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd6800cf120 0x7fd6800c5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:24.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.649+0000 7fd67fe00640 1 -- 172.21.15.52:0/2204807361 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd680098a40 msgr2=0x7fd68015c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:24.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.649+0000 7fd67fe00640 1 --2- 172.21.15.52:0/2204807361 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd680098a40 0x7fd68015c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:24.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.649+0000 7fd67fe00640 1 -- 172.21.15.52:0/2204807361 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd66c0094d0 con 0x7fd68015eb50 2024-03-12T17:06:24.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.649+0000 7fd67fe00640 1 --2- 172.21.15.52:0/2204807361 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd68015eb50 0x7fd68006f4b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd67800c7a0 tx=0x7fd67800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:24.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.650+0000 7fd677400640 1 -- 172.21.15.52:0/2204807361 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd678003b70 con 0x7fd68015eb50 2024-03-12T17:06:24.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.650+0000 7fd677400640 1 -- 172.21.15.52:0/2204807361 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd678003cf0 con 0x7fd68015eb50 2024-03-12T17:06:24.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.650+0000 7fd677400640 1 -- 172.21.15.52:0/2204807361 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd678003ea0 con 0x7fd68015eb50 2024-03-12T17:06:24.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.650+0000 7fd686200640 1 -- 172.21.15.52:0/2204807361 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd68009b5e0 con 0x7fd68015eb50 2024-03-12T17:06:24.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.650+0000 7fd686200640 1 -- 172.21.15.52:0/2204807361 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd68009bba0 con 0x7fd68015eb50 2024-03-12T17:06:24.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.651+0000 7fd686200640 1 -- 172.21.15.52:0/2204807361 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6800b1f80 con 0x7fd68015eb50 2024-03-12T17:06:24.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.653+0000 7fd677400640 1 -- 172.21.15.52:0/2204807361 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd67800aa90 con 0x7fd68015eb50 2024-03-12T17:06:24.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.653+0000 7fd677400640 1 --2- 172.21.15.52:0/2204807361 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd66006ac00 0x7fd66006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:24.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.653+0000 7fd677400640 1 -- 172.21.15.52:0/2204807361 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fd678089a30 con 0x7fd68015eb50 2024-03-12T17:06:24.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.654+0000 7fd67f400640 1 --2- 172.21.15.52:0/2204807361 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd66006ac00 0x7fd66006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:24.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.654+0000 7fd67f400640 1 --2- 172.21.15.52:0/2204807361 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd66006ac00 0x7fd66006d0c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd670006a90 tx=0x7fd670006a20 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:24.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.657+0000 7fd677400640 1 -- 172.21.15.52:0/2204807361 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fd678054de0 con 0x7fd68015eb50 2024-03-12T17:06:24.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:24.856+0000 7fd686200640 1 -- 172.21.15.52:0/2204807361 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fd6800cfc70 con 0x7fd68015eb50 2024-03-12T17:06:25.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.414+0000 7fd677400640 1 -- 172.21.15.52:0/2204807361 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) v1 ==== 171+0+0 (secure 0 0 0) 0x7fd678058400 con 0x7fd68015eb50 2024-03-12T17:06:25.410 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_data' created 2024-03-12T17:06:25.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.419+0000 7fd686200640 1 -- 172.21.15.52:0/2204807361 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd66006ac00 msgr2=0x7fd66006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:25.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.419+0000 7fd686200640 1 --2- 172.21.15.52:0/2204807361 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd66006ac00 0x7fd66006d0c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd670006a90 tx=0x7fd670006a20 comp rx=0 tx=0).stop 2024-03-12T17:06:25.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.419+0000 7fd686200640 1 -- 172.21.15.52:0/2204807361 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd68015eb50 msgr2=0x7fd68006f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:25.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.419+0000 7fd686200640 1 --2- 172.21.15.52:0/2204807361 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd68015eb50 0x7fd68006f4b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd67800c7a0 tx=0x7fd67800cc60 comp rx=0 tx=0).stop 2024-03-12T17:06:25.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.420+0000 7fd686200640 1 -- 172.21.15.52:0/2204807361 shutdown_connections 2024-03-12T17:06:25.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.420+0000 7fd686200640 1 --2- 172.21.15.52:0/2204807361 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd66006ac00 0x7fd66006d0c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:25.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.420+0000 7fd686200640 1 --2- 172.21.15.52:0/2204807361 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd6800cf120 0x7fd6800c5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:25.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.420+0000 7fd686200640 1 --2- 172.21.15.52:0/2204807361 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd68015eb50 0x7fd68006f4b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:25.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.420+0000 7fd686200640 1 --2- 172.21.15.52:0/2204807361 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd680098a40 0x7fd68015c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:25.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.420+0000 7fd686200640 1 -- 172.21.15.52:0/2204807361 >> 172.21.15.52:0/2204807361 conn(0x7fd6800b2180 msgr2=0x7fd6800cdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:25.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.425+0000 7fd686200640 1 -- 172.21.15.52:0/2204807361 shutdown_connections 2024-03-12T17:06:25.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.425+0000 7fd686200640 1 -- 172.21.15.52:0/2204807361 wait complete. 2024-03-12T17:06:25.453 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-12T17:06:25.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.644+0000 7f052f800640 1 Processor -- start 2024-03-12T17:06:25.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.644+0000 7f052f800640 1 -- start start 2024-03-12T17:06:25.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.646+0000 7f052f800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05280ce930 0x7f05280add30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:25.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.646+0000 7f052f800640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f05280cd3e0 0x7f05280ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:25.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.646+0000 7f052f800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f052806ce90 0x7f052806d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:25.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.646+0000 7f052f800640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f052806c760 con 0x7f05280cd3e0 2024-03-12T17:06:25.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.646+0000 7f052f800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05280b42d0 con 0x7f052806ce90 2024-03-12T17:06:25.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.646+0000 7f052f800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05280cf250 con 0x7f05280ce930 2024-03-12T17:06:25.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.647+0000 7f0527e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f05280cd3e0 0x7f05280ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:25.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.647+0000 7f0527e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f05280cd3e0 0x7f05280ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52006/0 (socket says 172.21.15.52:52006) 2024-03-12T17:06:25.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.647+0000 7f0527e00640 1 -- 172.21.15.52:0/618433844 learned_addr learned my addr 172.21.15.52:0/618433844 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:25.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.647+0000 7f052ce00640 1 --2- 172.21.15.52:0/618433844 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05280ce930 0x7f05280add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:25.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.647+0000 7f052d800640 1 --2- 172.21.15.52:0/618433844 >> v2:172.21.15.102:3300/0 conn(0x7f052806ce90 0x7f052806d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:25.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.647+0000 7f0527e00640 1 -- 172.21.15.52:0/618433844 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05280ce930 msgr2=0x7f05280add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:25.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.647+0000 7f0527e00640 1 --2- 172.21.15.52:0/618433844 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05280ce930 0x7f05280add30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:25.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.647+0000 7f0527e00640 1 -- 172.21.15.52:0/618433844 >> v2:172.21.15.102:3300/0 conn(0x7f052806ce90 msgr2=0x7f052806d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:25.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.647+0000 7f0527e00640 1 --2- 172.21.15.52:0/618433844 >> v2:172.21.15.102:3300/0 conn(0x7f052806ce90 0x7f052806d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:25.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.647+0000 7f0527e00640 1 -- 172.21.15.52:0/618433844 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05280cdba0 con 0x7f05280cd3e0 2024-03-12T17:06:25.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.647+0000 7f052d800640 1 --2- 172.21.15.52:0/618433844 >> v2:172.21.15.102:3300/0 conn(0x7f052806ce90 0x7f052806d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:25.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.647+0000 7f052ce00640 1 --2- 172.21.15.52:0/618433844 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05280ce930 0x7f05280add30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:25.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.648+0000 7f0527e00640 1 --2- 172.21.15.52:0/618433844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05280cd3e0 0x7f05280ced10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f05280ccc70 tx=0x7f051402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841cc4b9e8fe1d2d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:25.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.649+0000 7f0527400640 1 -- 172.21.15.52:0/618433844 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051402ca80 con 0x7f05280cd3e0 2024-03-12T17:06:25.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.649+0000 7f0527400640 1 -- 172.21.15.52:0/618433844 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051402cbe0 con 0x7f05280cd3e0 2024-03-12T17:06:25.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.649+0000 7f0527400640 1 -- 172.21.15.52:0/618433844 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05140343f0 con 0x7f05280cd3e0 2024-03-12T17:06:25.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.649+0000 7f052f800640 1 -- 172.21.15.52:0/618433844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05280cd3e0 msgr2=0x7f05280ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:25.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.649+0000 7f052f800640 1 --2- 172.21.15.52:0/618433844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05280cd3e0 0x7f05280ced10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f05280ccc70 tx=0x7f051402bee0 comp rx=0 tx=0).stop 2024-03-12T17:06:25.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.650+0000 7f052f800640 1 -- 172.21.15.52:0/618433844 shutdown_connections 2024-03-12T17:06:25.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.650+0000 7f052f800640 1 --2- 172.21.15.52:0/618433844 >> v2:172.21.15.102:3300/0 conn(0x7f052806ce90 0x7f052806d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:25.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.650+0000 7f052f800640 1 --2- 172.21.15.52:0/618433844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05280cd3e0 0x7f05280ced10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:25.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.650+0000 7f052f800640 1 --2- 172.21.15.52:0/618433844 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05280ce930 0x7f05280add30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:25.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.650+0000 7f052f800640 1 -- 172.21.15.52:0/618433844 >> 172.21.15.52:0/618433844 conn(0x7f052806a0b0 msgr2=0x7f052806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:25.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.650+0000 7f052f800640 1 -- 172.21.15.52:0/618433844 shutdown_connections 2024-03-12T17:06:25.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.650+0000 7f052f800640 1 -- 172.21.15.52:0/618433844 wait complete. 2024-03-12T17:06:25.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.652+0000 7f052f800640 1 Processor -- start 2024-03-12T17:06:25.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.652+0000 7f052f800640 1 -- start start 2024-03-12T17:06:25.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.653+0000 7f052f800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f052806ce90 0x7f0528144690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:25.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.653+0000 7f052f800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05280cd3e0 0x7f0528157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:25.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.653+0000 7f052f800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05280ce930 0x7f0528151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:25.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.653+0000 7f052f800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05281456b0 con 0x7f05280cd3e0 2024-03-12T17:06:25.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.653+0000 7f052f800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0528157a90 con 0x7f052806ce90 2024-03-12T17:06:25.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.653+0000 7f052f800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05281590b0 con 0x7f05280ce930 2024-03-12T17:06:25.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.654+0000 7f0527e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05280cd3e0 0x7f0528157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:25.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.654+0000 7f0527e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05280cd3e0 0x7f0528157220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52022/0 (socket says 172.21.15.52:52022) 2024-03-12T17:06:25.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.654+0000 7f0527e00640 1 -- 172.21.15.52:0/4136552243 learned_addr learned my addr 172.21.15.52:0/4136552243 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:25.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.654+0000 7f052d800640 1 --2- 172.21.15.52:0/4136552243 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05280ce930 0x7f0528151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:25.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.654+0000 7f052ce00640 1 --2- 172.21.15.52:0/4136552243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f052806ce90 0x7f0528144690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:25.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.654+0000 7f0527e00640 1 -- 172.21.15.52:0/4136552243 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05280ce930 msgr2=0x7f0528151b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:25.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.654+0000 7f0527e00640 1 --2- 172.21.15.52:0/4136552243 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05280ce930 0x7f0528151b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:25.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.654+0000 7f0527e00640 1 -- 172.21.15.52:0/4136552243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f052806ce90 msgr2=0x7f0528144690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:25.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.654+0000 7f0527e00640 1 --2- 172.21.15.52:0/4136552243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f052806ce90 0x7f0528144690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:25.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.654+0000 7f0527e00640 1 -- 172.21.15.52:0/4136552243 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0514009b30 con 0x7f05280cd3e0 2024-03-12T17:06:25.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.655+0000 7f0527e00640 1 --2- 172.21.15.52:0/4136552243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05280cd3e0 0x7f0528157220 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0514000f80 tx=0x7f0514004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:25.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.655+0000 7f0524c00640 1 -- 172.21.15.52:0/4136552243 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051403e450 con 0x7f05280cd3e0 2024-03-12T17:06:25.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.655+0000 7f0524c00640 1 -- 172.21.15.52:0/4136552243 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051403e5f0 con 0x7f05280cd3e0 2024-03-12T17:06:25.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.655+0000 7f0524c00640 1 -- 172.21.15.52:0/4136552243 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051403e9a0 con 0x7f05280cd3e0 2024-03-12T17:06:25.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.655+0000 7f052f800640 1 -- 172.21.15.52:0/4136552243 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0528159390 con 0x7f05280cd3e0 2024-03-12T17:06:25.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.655+0000 7f052f800640 1 -- 172.21.15.52:0/4136552243 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0528159950 con 0x7f05280cd3e0 2024-03-12T17:06:25.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.656+0000 7f052f800640 1 -- 172.21.15.52:0/4136552243 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0528073390 con 0x7f05280cd3e0 2024-03-12T17:06:25.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.657+0000 7f0524c00640 1 -- 172.21.15.52:0/4136552243 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f051402cd50 con 0x7f05280cd3e0 2024-03-12T17:06:25.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.658+0000 7f0524c00640 1 --2- 172.21.15.52:0/4136552243 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f050806ac30 0x7f050806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:25.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.658+0000 7f0524c00640 1 -- 172.21.15.52:0/4136552243 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f05140abcd0 con 0x7f05280cd3e0 2024-03-12T17:06:25.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.659+0000 7f052ce00640 1 --2- 172.21.15.52:0/4136552243 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f050806ac30 0x7f050806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:25.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.660+0000 7f052ce00640 1 --2- 172.21.15.52:0/4136552243 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f050806ac30 0x7f050806d0f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f051800a910 tx=0x7f0518008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:25.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.663+0000 7f0524c00640 1 -- 172.21.15.52:0/4136552243 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f0514076e70 con 0x7f05280cd3e0 2024-03-12T17:06:25.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.903+0000 7f052f800640 1 -- 172.21.15.52:0/4136552243 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f052806c540 con 0x7f05280cd3e0 2024-03-12T17:06:25.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.912+0000 7f0524c00640 1 -- 172.21.15.52:0/4136552243 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f0514033e80 con 0x7f05280cd3e0 2024-03-12T17:06:25.908 INFO:teuthology.orchestra.run.smithi052.stderr:new fs with metadata pool 2 and data pool 3 2024-03-12T17:06:25.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.918+0000 7f052f800640 1 -- 172.21.15.52:0/4136552243 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f050806ac30 msgr2=0x7f050806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:25.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.918+0000 7f052f800640 1 --2- 172.21.15.52:0/4136552243 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f050806ac30 0x7f050806d0f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f051800a910 tx=0x7f0518008040 comp rx=0 tx=0).stop 2024-03-12T17:06:25.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.918+0000 7f052f800640 1 -- 172.21.15.52:0/4136552243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05280cd3e0 msgr2=0x7f0528157220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:25.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.918+0000 7f052f800640 1 --2- 172.21.15.52:0/4136552243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05280cd3e0 0x7f0528157220 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0514000f80 tx=0x7f0514004a40 comp rx=0 tx=0).stop 2024-03-12T17:06:25.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.919+0000 7f052f800640 1 -- 172.21.15.52:0/4136552243 shutdown_connections 2024-03-12T17:06:25.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.919+0000 7f052f800640 1 --2- 172.21.15.52:0/4136552243 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f050806ac30 0x7f050806d0f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:25.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.919+0000 7f052f800640 1 --2- 172.21.15.52:0/4136552243 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05280ce930 0x7f0528151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:25.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.919+0000 7f052f800640 1 --2- 172.21.15.52:0/4136552243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05280cd3e0 0x7f0528157220 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0514000f80 tx=0x7f0514004a40 comp rx=0 tx=0).stop 2024-03-12T17:06:25.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.919+0000 7f052f800640 1 --2- 172.21.15.52:0/4136552243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f052806ce90 0x7f0528144690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:25.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.919+0000 7f052f800640 1 -- 172.21.15.52:0/4136552243 >> 172.21.15.52:0/4136552243 conn(0x7f052806a0b0 msgr2=0x7f052815c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:25.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.924+0000 7f052f800640 1 -- 172.21.15.52:0/4136552243 shutdown_connections 2024-03-12T17:06:25.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:25.925+0000 7f052f800640 1 -- 172.21.15.52:0/4136552243 wait complete. 2024-03-12T17:06:25.947 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-03-12T17:06:25.947 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-03-12T17:06:26.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.140+0000 7f3945a00640 1 Processor -- start 2024-03-12T17:06:26.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.141+0000 7f3945a00640 1 -- start start 2024-03-12T17:06:26.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.142+0000 7f3945a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f39400ca7b0 0x7f39400cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:26.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.142+0000 7f3945a00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f39400ce930 0x7f39400cedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:26.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.142+0000 7f3945a00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3940159c10 con 0x7f3940159830 2024-03-12T17:06:26.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.142+0000 7f3945a00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394006de50 con 0x7f39400ce930 2024-03-12T17:06:26.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.142+0000 7f3945a00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39400cf300 con 0x7f39400ca7b0 2024-03-12T17:06:26.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.142+0000 7f393f400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f39400ce930 0x7f39400cedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:26.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.143+0000 7f393f400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f39400ce930 0x7f39400cedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52036/0 (socket says 172.21.15.52:52036) 2024-03-12T17:06:26.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.143+0000 7f393f400640 1 -- 172.21.15.52:0/2422164592 learned_addr learned my addr 172.21.15.52:0/2422164592 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:26.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.143+0000 7f3937e00640 1 --2- 172.21.15.52:0/2422164592 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f39400ca7b0 0x7f39400cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:26.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.143+0000 7f393f400640 1 -- 172.21.15.52:0/2422164592 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f39400ca7b0 msgr2=0x7f39400cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:26.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.143+0000 7f393f400640 1 --2- 172.21.15.52:0/2422164592 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f39400ca7b0 0x7f39400cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:26.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.143+0000 7f393f400640 1 -- 172.21.15.52:0/2422164592 >> v1:172.21.15.102:6789/0 conn(0x7f3940159830 legacy=0x7f39400cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:26.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.143+0000 7f393f400640 1 -- 172.21.15.52:0/2422164592 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39400cd840 con 0x7f39400ce930 2024-03-12T17:06:26.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.144+0000 7f393f400640 1 --2- 172.21.15.52:0/2422164592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39400ce930 0x7f39400cedc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3938009e50 tx=0x7f393802bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb20c9a9deaf1647 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:26.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.144+0000 7f393e000640 1 -- 172.21.15.52:0/2422164592 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393802cb90 con 0x7f39400ce930 2024-03-12T17:06:26.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.144+0000 7f393e000640 1 -- 172.21.15.52:0/2422164592 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393802ccf0 con 0x7f39400ce930 2024-03-12T17:06:26.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.144+0000 7f393e000640 1 -- 172.21.15.52:0/2422164592 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938034610 con 0x7f39400ce930 2024-03-12T17:06:26.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.145+0000 7f3945a00640 1 -- 172.21.15.52:0/2422164592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39400ce930 msgr2=0x7f39400cedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:26.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.145+0000 7f3945a00640 1 --2- 172.21.15.52:0/2422164592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39400ce930 0x7f39400cedc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3938009e50 tx=0x7f393802bff0 comp rx=0 tx=0).stop 2024-03-12T17:06:26.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.145+0000 7f3945a00640 1 -- 172.21.15.52:0/2422164592 shutdown_connections 2024-03-12T17:06:26.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.145+0000 7f3945a00640 1 --2- 172.21.15.52:0/2422164592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39400ce930 0x7f39400cedc0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:26.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.145+0000 7f3945a00640 1 --2- 172.21.15.52:0/2422164592 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f39400ca7b0 0x7f39400cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:26.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.145+0000 7f3945a00640 1 -- 172.21.15.52:0/2422164592 >> 172.21.15.52:0/2422164592 conn(0x7f394006a0b0 msgr2=0x7f394006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:26.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.145+0000 7f3945a00640 1 -- 172.21.15.52:0/2422164592 shutdown_connections 2024-03-12T17:06:26.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.145+0000 7f3945a00640 1 -- 172.21.15.52:0/2422164592 wait complete. 2024-03-12T17:06:26.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.147+0000 7f3945a00640 1 Processor -- start 2024-03-12T17:06:26.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.147+0000 7f3945a00640 1 -- start start 2024-03-12T17:06:26.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.148+0000 7f3945a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f39400ca7b0 0x7f3940159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:26.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.148+0000 7f3945a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39400ce930 0x7f3940142200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:26.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.148+0000 7f3945a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3940159830 0x7f394015c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:26.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.148+0000 7f3945a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3940150710 con 0x7f3940159830 2024-03-12T17:06:26.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.148+0000 7f3945a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394015a490 con 0x7f39400ce930 2024-03-12T17:06:26.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.148+0000 7f3945a00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f394015a730 con 0x7f39400ca7b0 2024-03-12T17:06:26.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.148+0000 7f393f400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3940159830 0x7f394015c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:26.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.148+0000 7f393f400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3940159830 0x7f394015c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52048/0 (socket says 172.21.15.52:52048) 2024-03-12T17:06:26.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.148+0000 7f393f400640 1 -- 172.21.15.52:0/3084539346 learned_addr learned my addr 172.21.15.52:0/3084539346 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:26.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.148+0000 7f393ea00640 1 --2- 172.21.15.52:0/3084539346 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f39400ca7b0 0x7f3940159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:26.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.148+0000 7f3937e00640 1 --2- 172.21.15.52:0/3084539346 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39400ce930 0x7f3940142200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:26.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.148+0000 7f393f400640 1 -- 172.21.15.52:0/3084539346 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f39400ca7b0 msgr2=0x7f3940159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:26.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.148+0000 7f393f400640 1 --2- 172.21.15.52:0/3084539346 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f39400ca7b0 0x7f3940159f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:26.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.148+0000 7f393f400640 1 -- 172.21.15.52:0/3084539346 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39400ce930 msgr2=0x7f3940142200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:26.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.148+0000 7f393f400640 1 --2- 172.21.15.52:0/3084539346 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39400ce930 0x7f3940142200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:26.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.148+0000 7f393f400640 1 -- 172.21.15.52:0/3084539346 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3938009b30 con 0x7f3940159830 2024-03-12T17:06:26.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.149+0000 7f3937e00640 1 --2- 172.21.15.52:0/3084539346 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39400ce930 0x7f3940142200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:26.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.149+0000 7f393f400640 1 --2- 172.21.15.52:0/3084539346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3940159830 0x7f394015c050 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3938009e50 tx=0x7f3938004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:26.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.149+0000 7f3936a00640 1 -- 172.21.15.52:0/3084539346 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938009510 con 0x7f3940159830 2024-03-12T17:06:26.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.149+0000 7f3936a00640 1 -- 172.21.15.52:0/3084539346 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393803e450 con 0x7f3940159830 2024-03-12T17:06:26.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.149+0000 7f3936a00640 1 -- 172.21.15.52:0/3084539346 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393803e800 con 0x7f3940159830 2024-03-12T17:06:26.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.149+0000 7f3945a00640 1 -- 172.21.15.52:0/3084539346 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f394015aa10 con 0x7f3940159830 2024-03-12T17:06:26.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.149+0000 7f3945a00640 1 -- 172.21.15.52:0/3084539346 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3940150c90 con 0x7f3940159830 2024-03-12T17:06:26.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.151+0000 7f3945a00640 1 -- 172.21.15.52:0/3084539346 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3940073390 con 0x7f3940159830 2024-03-12T17:06:26.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.152+0000 7f3936a00640 1 -- 172.21.15.52:0/3084539346 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f393803e960 con 0x7f3940159830 2024-03-12T17:06:26.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.153+0000 7f3936a00640 1 --2- 172.21.15.52:0/3084539346 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f391c06ac00 0x7f391c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:26.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.153+0000 7f3936a00640 1 -- 172.21.15.52:0/3084539346 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f39380ac340 con 0x7f3940159830 2024-03-12T17:06:26.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.153+0000 7f393ea00640 1 --2- 172.21.15.52:0/3084539346 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f391c06ac00 0x7f391c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:26.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.154+0000 7f393ea00640 1 --2- 172.21.15.52:0/3084539346 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f391c06ac00 0x7f391c06d0c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3930000f60 tx=0x7f3930009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:26.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.157+0000 7f3936a00640 1 -- 172.21.15.52:0/3084539346 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f39380776b0 con 0x7f3940159830 2024-03-12T17:06:26.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.362+0000 7f3945a00640 1 -- 172.21.15.52:0/3084539346 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f39400cee50 con 0x7f3940159830 2024-03-12T17:06:26.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.915+0000 7f3936a00640 1 -- 172.21.15.52:0/3084539346 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) v1 ==== 157+0+0 (secure 0 0 0) 0x7f393802ce60 con 0x7f3940159830 2024-03-12T17:06:26.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.919+0000 7f3945a00640 1 -- 172.21.15.52:0/3084539346 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f391c06ac00 msgr2=0x7f391c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:26.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.919+0000 7f3945a00640 1 --2- 172.21.15.52:0/3084539346 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f391c06ac00 0x7f391c06d0c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3930000f60 tx=0x7f3930009040 comp rx=0 tx=0).stop 2024-03-12T17:06:26.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.919+0000 7f3945a00640 1 -- 172.21.15.52:0/3084539346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3940159830 msgr2=0x7f394015c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:26.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.919+0000 7f3945a00640 1 --2- 172.21.15.52:0/3084539346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3940159830 0x7f394015c050 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3938009e50 tx=0x7f3938004a40 comp rx=0 tx=0).stop 2024-03-12T17:06:26.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.919+0000 7f3945a00640 1 -- 172.21.15.52:0/3084539346 shutdown_connections 2024-03-12T17:06:26.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.919+0000 7f3945a00640 1 --2- 172.21.15.52:0/3084539346 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f391c06ac00 0x7f391c06d0c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:26.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.919+0000 7f3945a00640 1 --2- 172.21.15.52:0/3084539346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3940159830 0x7f394015c050 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:26.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.919+0000 7f3945a00640 1 --2- 172.21.15.52:0/3084539346 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39400ce930 0x7f3940142200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:26.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.919+0000 7f3945a00640 1 --2- 172.21.15.52:0/3084539346 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f39400ca7b0 0x7f3940159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:26.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.919+0000 7f3945a00640 1 -- 172.21.15.52:0/3084539346 >> 172.21.15.52:0/3084539346 conn(0x7f394006a0b0 msgr2=0x7f3940152b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:26.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.920+0000 7f3945a00640 1 -- 172.21.15.52:0/3084539346 shutdown_connections 2024-03-12T17:06:26.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:26.920+0000 7f3945a00640 1 -- 172.21.15.52:0/3084539346 wait complete. 2024-03-12T17:06:26.934 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-03-12T17:06:27.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.171+0000 7f097a800640 1 Processor -- start 2024-03-12T17:06:27.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.171+0000 7f097a800640 1 -- start start 2024-03-12T17:06:27.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.173+0000 7f097a800640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f09740b4a10 0x7f09740d0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:27.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.173+0000 7f097a800640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09740d1830 con 0x7f09740d10f0 2024-03-12T17:06:27.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.173+0000 7f097a800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09740d1b10 con 0x7f097415cd50 2024-03-12T17:06:27.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.173+0000 7f097a800640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09740d1df0 con 0x7f09740b4a10 2024-03-12T17:06:27.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.173+0000 7f0972a00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f09740b4a10 0x7f09740d0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:27.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.173+0000 7f0972a00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f09740b4a10 0x7f09740d0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52056/0 (socket says 172.21.15.52:52056) 2024-03-12T17:06:27.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.173+0000 7f0972a00640 1 -- 172.21.15.52:0/1467461540 learned_addr learned my addr 172.21.15.52:0/1467461540 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:27.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.173+0000 7f0972a00640 1 -- 172.21.15.52:0/1467461540 >> v1:172.21.15.102:6789/0 conn(0x7f097415cd50 legacy=0x7f09740b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:27.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.173+0000 7f0972a00640 1 -- 172.21.15.52:0/1467461540 >> v1:172.21.15.52:6789/0 conn(0x7f09740d10f0 legacy=0x7f09740b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:27.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.173+0000 7f0972a00640 1 -- 172.21.15.52:0/1467461540 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09740d20d0 con 0x7f09740b4a10 2024-03-12T17:06:27.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.174+0000 7f0972a00640 1 --2- 172.21.15.52:0/1467461540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09740b4a10 0x7f09740d0bb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0960009870 tx=0x7f096002b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418938662987c94e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:27.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.175+0000 7f0972000640 1 -- 172.21.15.52:0/1467461540 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0960030850 con 0x7f09740b4a10 2024-03-12T17:06:27.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.175+0000 7f0972000640 1 -- 172.21.15.52:0/1467461540 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09600309b0 con 0x7f09740b4a10 2024-03-12T17:06:27.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.175+0000 7f0972000640 1 -- 172.21.15.52:0/1467461540 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0960030d40 con 0x7f09740b4a10 2024-03-12T17:06:27.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.175+0000 7f097a800640 1 -- 172.21.15.52:0/1467461540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09740b4a10 msgr2=0x7f09740d0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:27.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.175+0000 7f097a800640 1 --2- 172.21.15.52:0/1467461540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09740b4a10 0x7f09740d0bb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0960009870 tx=0x7f096002b990 comp rx=0 tx=0).stop 2024-03-12T17:06:27.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.176+0000 7f097a800640 1 -- 172.21.15.52:0/1467461540 shutdown_connections 2024-03-12T17:06:27.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.176+0000 7f097a800640 1 --2- 172.21.15.52:0/1467461540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09740b4a10 0x7f09740d0bb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:27.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.176+0000 7f097a800640 1 -- 172.21.15.52:0/1467461540 >> 172.21.15.52:0/1467461540 conn(0x7f09740cf300 msgr2=0x7f09740cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:27.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.176+0000 7f097a800640 1 -- 172.21.15.52:0/1467461540 shutdown_connections 2024-03-12T17:06:27.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.176+0000 7f097a800640 1 -- 172.21.15.52:0/1467461540 wait complete. 2024-03-12T17:06:27.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.177+0000 7f097a800640 1 Processor -- start 2024-03-12T17:06:27.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.178+0000 7f097a800640 1 -- start start 2024-03-12T17:06:27.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.178+0000 7f097a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09740b4a10 0x7f09741f2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:27.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.178+0000 7f097a800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09740d10f0 0x7f0974158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:27.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.178+0000 7f097a800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f097415cd50 0x7f09740cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:27.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.179+0000 7f097a800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09741f2ea0 con 0x7f09740d10f0 2024-03-12T17:06:27.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.179+0000 7f097a800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09740cbef0 con 0x7f09740b4a10 2024-03-12T17:06:27.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.179+0000 7f097a800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09740ca040 con 0x7f097415cd50 2024-03-12T17:06:27.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.179+0000 7f0972a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09740d10f0 0x7f0974158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:27.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.179+0000 7f0972a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09740d10f0 0x7f0974158a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52058/0 (socket says 172.21.15.52:52058) 2024-03-12T17:06:27.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.179+0000 7f0972a00640 1 -- 172.21.15.52:0/4262571046 learned_addr learned my addr 172.21.15.52:0/4262571046 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:27.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.179+0000 7f0973400640 1 --2- 172.21.15.52:0/4262571046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09740b4a10 0x7f09741f2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:27.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.179+0000 7f0973e00640 1 --2- 172.21.15.52:0/4262571046 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f097415cd50 0x7f09740cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:27.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.179+0000 7f0972a00640 1 -- 172.21.15.52:0/4262571046 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f097415cd50 msgr2=0x7f09740cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:27.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.179+0000 7f0972a00640 1 --2- 172.21.15.52:0/4262571046 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f097415cd50 0x7f09740cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:27.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.179+0000 7f0972a00640 1 -- 172.21.15.52:0/4262571046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09740b4a10 msgr2=0x7f09741f2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:27.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.179+0000 7f0972a00640 1 --2- 172.21.15.52:0/4262571046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09740b4a10 0x7f09741f2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:27.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.179+0000 7f0972a00640 1 -- 172.21.15.52:0/4262571046 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09600094d0 con 0x7f09740d10f0 2024-03-12T17:06:27.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.179+0000 7f0973e00640 1 --2- 172.21.15.52:0/4262571046 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f097415cd50 0x7f09740cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:27.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.180+0000 7f0972a00640 1 --2- 172.21.15.52:0/4262571046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09740d10f0 0x7f0974158a50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0960009870 tx=0x7f0960004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:27.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.180+0000 7f096b400640 1 -- 172.21.15.52:0/4262571046 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0960030800 con 0x7f09740d10f0 2024-03-12T17:06:27.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.180+0000 7f096b400640 1 -- 172.21.15.52:0/4262571046 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0960043400 con 0x7f09740d10f0 2024-03-12T17:06:27.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.180+0000 7f096b400640 1 -- 172.21.15.52:0/4262571046 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09600437b0 con 0x7f09740d10f0 2024-03-12T17:06:27.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.180+0000 7f097a800640 1 -- 172.21.15.52:0/4262571046 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09740ca320 con 0x7f09740d10f0 2024-03-12T17:06:27.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.180+0000 7f097a800640 1 -- 172.21.15.52:0/4262571046 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09740ca8e0 con 0x7f09740d10f0 2024-03-12T17:06:27.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.181+0000 7f097a800640 1 -- 172.21.15.52:0/4262571046 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0974073390 con 0x7f09740d10f0 2024-03-12T17:06:27.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.182+0000 7f096b400640 1 -- 172.21.15.52:0/4262571046 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f096002e070 con 0x7f09740d10f0 2024-03-12T17:06:27.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.183+0000 7f096b400640 1 --2- 172.21.15.52:0/4262571046 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f095406ac30 0x7f095406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:27.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.183+0000 7f096b400640 1 -- 172.21.15.52:0/4262571046 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5935+0+0 (secure 0 0 0) 0x7f096003e070 con 0x7f09740d10f0 2024-03-12T17:06:27.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.184+0000 7f0973400640 1 --2- 172.21.15.52:0/4262571046 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f095406ac30 0x7f095406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:27.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.185+0000 7f0973400640 1 --2- 172.21.15.52:0/4262571046 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f095406ac30 0x7f095406d0f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0964006fd0 tx=0x7f0964008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:27.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.188+0000 7f096b400640 1 -- 172.21.15.52:0/4262571046 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f09600774e0 con 0x7f09740d10f0 2024-03-12T17:06:27.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:27.387+0000 7f097a800640 1 -- 172.21.15.52:0/4262571046 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0974067040 con 0x7f09740d10f0 2024-03-12T17:06:28.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:28.968+0000 7f096b400640 1 -- 172.21.15.52:0/4262571046 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) v1 ==== 228+0+0 (secure 0 0 0) 0x7f096007ab00 con 0x7f09740d10f0 2024-03-12T17:06:28.961 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_data_ec' created 2024-03-12T17:06:28.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:28.973+0000 7f097a800640 1 -- 172.21.15.52:0/4262571046 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f095406ac30 msgr2=0x7f095406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:28.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:28.973+0000 7f097a800640 1 --2- 172.21.15.52:0/4262571046 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f095406ac30 0x7f095406d0f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0964006fd0 tx=0x7f0964008040 comp rx=0 tx=0).stop 2024-03-12T17:06:28.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:28.973+0000 7f097a800640 1 -- 172.21.15.52:0/4262571046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09740d10f0 msgr2=0x7f0974158a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:28.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:28.973+0000 7f097a800640 1 --2- 172.21.15.52:0/4262571046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09740d10f0 0x7f0974158a50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0960009870 tx=0x7f0960004730 comp rx=0 tx=0).stop 2024-03-12T17:06:28.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:28.975+0000 7f097a800640 1 -- 172.21.15.52:0/4262571046 shutdown_connections 2024-03-12T17:06:28.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:28.975+0000 7f097a800640 1 --2- 172.21.15.52:0/4262571046 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f095406ac30 0x7f095406d0f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:28.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:28.975+0000 7f097a800640 1 --2- 172.21.15.52:0/4262571046 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f097415cd50 0x7f09740cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:28.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:28.975+0000 7f097a800640 1 --2- 172.21.15.52:0/4262571046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09740d10f0 0x7f0974158a50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:28.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:28.975+0000 7f097a800640 1 --2- 172.21.15.52:0/4262571046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09740b4a10 0x7f09741f2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:28.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:28.975+0000 7f097a800640 1 -- 172.21.15.52:0/4262571046 >> 172.21.15.52:0/4262571046 conn(0x7f09740cf300 msgr2=0x7f09740b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:28.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:28.975+0000 7f097a800640 1 -- 172.21.15.52:0/4262571046 shutdown_connections 2024-03-12T17:06:28.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:28.975+0000 7f097a800640 1 -- 172.21.15.52:0/4262571046 wait complete. 2024-03-12T17:06:28.995 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-03-12T17:06:29.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.198+0000 7fb606400640 1 Processor -- start 2024-03-12T17:06:29.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.199+0000 7fb606400640 1 -- start start 2024-03-12T17:06:29.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.200+0000 7fb606400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb600158db0 0x7fb60015d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:29.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.200+0000 7fb606400640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb60015d870 con 0x7fb6000cd820 2024-03-12T17:06:29.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.200+0000 7fb606400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6000caec0 con 0x7fb600070150 2024-03-12T17:06:29.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.200+0000 7fb606400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb60006d0b0 con 0x7fb600158db0 2024-03-12T17:06:29.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.200+0000 7fb5ffe00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fb6000cd820 0x7fb60006c9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:42842/0 (socket says 172.21.15.52:42842) 2024-03-12T17:06:29.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.200+0000 7fb5ffe00640 1 -- 172.21.15.52:0/1742639054 learned_addr learned my addr 172.21.15.52:0/1742639054 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:06:29.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.200+0000 7fb5ff400640 1 --2- 172.21.15.52:0/1742639054 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb600158db0 0x7fb60015d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:29.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.201+0000 7fb5fe000640 1 -- 172.21.15.52:0/1742639054 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2741850455 0 0) 0x7fb60015d870 con 0x7fb6000cd820 2024-03-12T17:06:29.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.201+0000 7fb5fe000640 1 -- 172.21.15.52:0/1742639054 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5e0003520 con 0x7fb6000cd820 2024-03-12T17:06:29.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.201+0000 7fb5fe000640 1 -- 172.21.15.52:0/1742639054 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1812477360 0 0) 0x7fb5e0003520 con 0x7fb6000cd820 2024-03-12T17:06:29.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.201+0000 7fb5fe000640 1 -- 172.21.15.52:0/1742639054 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb600158db0 msgr2=0x7fb60015d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:29.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.201+0000 7fb5fe000640 1 --2- 172.21.15.52:0/1742639054 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb600158db0 0x7fb60015d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:29.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.201+0000 7fb5fe000640 1 -- 172.21.15.52:0/1742639054 >> v1:172.21.15.102:6789/0 conn(0x7fb600070150 legacy=0x7fb60006c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:29.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.201+0000 7fb5fe000640 1 -- 172.21.15.52:0/1742639054 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6000cba90 con 0x7fb6000cd820 2024-03-12T17:06:29.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.201+0000 7fb5fe000640 1 -- 172.21.15.52:0/1742639054 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7fb5f8001670 con 0x7fb6000cd820 2024-03-12T17:06:29.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.201+0000 7fb5fe000640 1 -- 172.21.15.52:0/1742639054 >> v1:172.21.15.52:6789/0 conn(0x7fb6000cd820 legacy=0x7fb60006c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:29.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.201+0000 7fb5fe000640 1 --2- 172.21.15.52:0/1742639054 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb600158db0 0x7fb5e00039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:29.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.201+0000 7fb5fe000640 1 --2- 172.21.15.52:0/1742639054 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5e0003f20 0x7fb5e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:29.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.201+0000 7fb5fe000640 1 --2- 172.21.15.52:0/1742639054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5e0004d00 0x7fb5e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:29.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.201+0000 7fb5fe000640 1 -- 172.21.15.52:0/1742639054 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60006d0b0 con 0x7fb5e0003f20 2024-03-12T17:06:29.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.201+0000 7fb5fe000640 1 -- 172.21.15.52:0/1742639054 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e000ba60 con 0x7fb5e0004d00 2024-03-12T17:06:29.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.201+0000 7fb5fe000640 1 -- 172.21.15.52:0/1742639054 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e000de30 con 0x7fb600158db0 2024-03-12T17:06:29.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.202+0000 7fb5fea00640 1 --2- 172.21.15.52:0/1742639054 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5e0003f20 0x7fb5e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:29.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.202+0000 7fb5ff400640 1 --2- 172.21.15.52:0/1742639054 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb600158db0 0x7fb5e00039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:29.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.202+0000 7fb5ffe00640 1 --2- 172.21.15.52:0/1742639054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5e0004d00 0x7fb5e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:29.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.202+0000 7fb5fea00640 1 -- 172.21.15.52:0/1742639054 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb600158db0 msgr2=0x7fb5e00039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:29.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.202+0000 7fb5fea00640 1 --2- 172.21.15.52:0/1742639054 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb600158db0 0x7fb5e00039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:29.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.202+0000 7fb5fea00640 1 -- 172.21.15.52:0/1742639054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5e0004d00 msgr2=0x7fb5e0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:29.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.202+0000 7fb5fea00640 1 --2- 172.21.15.52:0/1742639054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5e0004d00 0x7fb5e0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:29.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.202+0000 7fb5fea00640 1 -- 172.21.15.52:0/1742639054 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e0003520 con 0x7fb5e0003f20 2024-03-12T17:06:29.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.203+0000 7fb5fea00640 1 --2- 172.21.15.52:0/1742639054 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5e0003f20 0x7fb5e00047c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8001e50 tx=0x7fb5e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36666e1523f14c2b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:29.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.203+0000 7fb5fe000640 1 -- 172.21.15.52:0/1742639054 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8003b70 con 0x7fb5e0003f20 2024-03-12T17:06:29.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.203+0000 7fb5fe000640 1 -- 172.21.15.52:0/1742639054 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e8003cd0 con 0x7fb5e0003f20 2024-03-12T17:06:29.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.203+0000 7fb5fe000640 1 -- 172.21.15.52:0/1742639054 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8005310 con 0x7fb5e0003f20 2024-03-12T17:06:29.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.204+0000 7fb606400640 1 -- 172.21.15.52:0/1742639054 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5e0003f20 msgr2=0x7fb5e00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:29.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.204+0000 7fb606400640 1 --2- 172.21.15.52:0/1742639054 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5e0003f20 0x7fb5e00047c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8001e50 tx=0x7fb5e00057f0 comp rx=0 tx=0).stop 2024-03-12T17:06:29.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.204+0000 7fb5ffe00640 1 -- 172.21.15.52:0/1742639054 reap_dead start 2024-03-12T17:06:29.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.204+0000 7fb606400640 1 -- 172.21.15.52:0/1742639054 shutdown_connections 2024-03-12T17:06:29.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.204+0000 7fb606400640 1 -- 172.21.15.52:0/1742639054 >> 172.21.15.52:0/1742639054 conn(0x7fb60006a0b0 msgr2=0x7fb60006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:29.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.204+0000 7fb606400640 1 -- 172.21.15.52:0/1742639054 shutdown_connections 2024-03-12T17:06:29.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.204+0000 7fb606400640 1 -- 172.21.15.52:0/1742639054 wait complete. 2024-03-12T17:06:29.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.206+0000 7fb606400640 1 Processor -- start 2024-03-12T17:06:29.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.206+0000 7fb606400640 1 -- start start 2024-03-12T17:06:29.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.207+0000 7fb606400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5e0003f20 0x7fb600148d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:29.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.207+0000 7fb606400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb600155350 0x7fb6001e3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:29.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.207+0000 7fb606400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6001571b0 0x7fb6001e4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:29.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.207+0000 7fb606400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60014ab90 con 0x7fb6001571b0 2024-03-12T17:06:29.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.207+0000 7fb606400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb600156460 con 0x7fb5e0003f20 2024-03-12T17:06:29.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.207+0000 7fb606400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb600149280 con 0x7fb600155350 2024-03-12T17:06:29.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.207+0000 7fb5ffe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6001571b0 0x7fb6001e4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:29.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.207+0000 7fb5ffe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6001571b0 0x7fb6001e4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52078/0 (socket says 172.21.15.52:52078) 2024-03-12T17:06:29.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.207+0000 7fb5ffe00640 1 -- 172.21.15.52:0/2767770130 learned_addr learned my addr 172.21.15.52:0/2767770130 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:29.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.208+0000 7fb5fea00640 1 --2- 172.21.15.52:0/2767770130 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb600155350 0x7fb6001e3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:29.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.208+0000 7fb5ff400640 1 --2- 172.21.15.52:0/2767770130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5e0003f20 0x7fb600148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:29.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.208+0000 7fb5ffe00640 1 -- 172.21.15.52:0/2767770130 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb600155350 msgr2=0x7fb6001e3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:29.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.208+0000 7fb5ffe00640 1 --2- 172.21.15.52:0/2767770130 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb600155350 0x7fb6001e3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:29.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.208+0000 7fb5ffe00640 1 -- 172.21.15.52:0/2767770130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5e0003f20 msgr2=0x7fb600148d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:29.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.208+0000 7fb5ffe00640 1 --2- 172.21.15.52:0/2767770130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5e0003f20 0x7fb600148d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:29.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.208+0000 7fb5ffe00640 1 -- 172.21.15.52:0/2767770130 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb600155850 con 0x7fb6001571b0 2024-03-12T17:06:29.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.208+0000 7fb5fea00640 1 --2- 172.21.15.52:0/2767770130 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb600155350 0x7fb6001e3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:29.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.208+0000 7fb5ffe00640 1 --2- 172.21.15.52:0/2767770130 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6001571b0 0x7fb6001e4000 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb6000cb520 tx=0x7fb5f800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:29.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.208+0000 7fb5f7400640 1 -- 172.21.15.52:0/2767770130 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f800eed0 con 0x7fb6001571b0 2024-03-12T17:06:29.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.208+0000 7fb5f7400640 1 -- 172.21.15.52:0/2767770130 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f800c630 con 0x7fb6001571b0 2024-03-12T17:06:29.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.208+0000 7fb5f7400640 1 -- 172.21.15.52:0/2767770130 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f800c9c0 con 0x7fb6001571b0 2024-03-12T17:06:29.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.208+0000 7fb606400640 1 -- 172.21.15.52:0/2767770130 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb600155b30 con 0x7fb6001571b0 2024-03-12T17:06:29.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.209+0000 7fb606400640 1 -- 172.21.15.52:0/2767770130 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6001560f0 con 0x7fb6001571b0 2024-03-12T17:06:29.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.210+0000 7fb606400640 1 -- 172.21.15.52:0/2767770130 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb60006df70 con 0x7fb6001571b0 2024-03-12T17:06:29.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.211+0000 7fb5f7400640 1 -- 172.21.15.52:0/2767770130 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb5f800cb20 con 0x7fb6001571b0 2024-03-12T17:06:29.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.211+0000 7fb5f7400640 1 --2- 172.21.15.52:0/2767770130 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb5dc06ac30 0x7fb5dc06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:29.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.212+0000 7fb5f7400640 1 -- 172.21.15.52:0/2767770130 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6428+0+0 (secure 0 0 0) 0x7fb5f80887a0 con 0x7fb6001571b0 2024-03-12T17:06:29.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.212+0000 7fb5ff400640 1 --2- 172.21.15.52:0/2767770130 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb5dc06ac30 0x7fb5dc06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:29.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.213+0000 7fb5ff400640 1 --2- 172.21.15.52:0/2767770130 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb5dc06ac30 0x7fb5dc06d0f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0009950 tx=0x7fb5f0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:29.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.217+0000 7fb5f7400640 1 -- 172.21.15.52:0/2767770130 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fb5f80536b0 con 0x7fb6001571b0 2024-03-12T17:06:29.361 INFO:tasks.ceph.mds.b.smithi102.stderr:2024-03-12T17:06:29.367+0000 7f5214c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-12T17:06:29.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.411+0000 7fb606400640 1 -- 172.21.15.52:0/2767770130 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fb600073390 con 0x7fb6001571b0 2024-03-12T17:06:29.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.982+0000 7fb5f7400640 1 -- 172.21.15.52:0/2767770130 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb5f80190c0 con 0x7fb6001571b0 2024-03-12T17:06:29.976 INFO:teuthology.orchestra.run.smithi052.stderr:set pool 4 allow_ec_overwrites to true 2024-03-12T17:06:29.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.989+0000 7fb606400640 1 -- 172.21.15.52:0/2767770130 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb5dc06ac30 msgr2=0x7fb5dc06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:29.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.989+0000 7fb606400640 1 --2- 172.21.15.52:0/2767770130 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb5dc06ac30 0x7fb5dc06d0f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0009950 tx=0x7fb5f0008040 comp rx=0 tx=0).stop 2024-03-12T17:06:29.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.989+0000 7fb606400640 1 -- 172.21.15.52:0/2767770130 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6001571b0 msgr2=0x7fb6001e4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:29.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.989+0000 7fb606400640 1 --2- 172.21.15.52:0/2767770130 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6001571b0 0x7fb6001e4000 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb6000cb520 tx=0x7fb5f800eb90 comp rx=0 tx=0).stop 2024-03-12T17:06:29.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.989+0000 7fb606400640 1 -- 172.21.15.52:0/2767770130 shutdown_connections 2024-03-12T17:06:29.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.989+0000 7fb606400640 1 --2- 172.21.15.52:0/2767770130 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb5dc06ac30 0x7fb5dc06d0f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:29.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.989+0000 7fb606400640 1 --2- 172.21.15.52:0/2767770130 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb600155350 0x7fb6001e3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:29.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.989+0000 7fb606400640 1 --2- 172.21.15.52:0/2767770130 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6001571b0 0x7fb6001e4000 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:29.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.989+0000 7fb606400640 1 --2- 172.21.15.52:0/2767770130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5e0003f20 0x7fb600148d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:29.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.989+0000 7fb606400640 1 -- 172.21.15.52:0/2767770130 >> 172.21.15.52:0/2767770130 conn(0x7fb60006a0b0 msgr2=0x7fb600070c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:29.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.990+0000 7fb606400640 1 -- 172.21.15.52:0/2767770130 shutdown_connections 2024-03-12T17:06:29.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:29.991+0000 7fb606400640 1 -- 172.21.15.52:0/2767770130 wait complete. 2024-03-12T17:06:30.021 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-03-12T17:06:30.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.212+0000 7f4a0a000640 1 Processor -- start 2024-03-12T17:06:30.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.213+0000 7f4a0a000640 1 -- start start 2024-03-12T17:06:30.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.214+0000 7f4a0a000640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4a040cd740 0x7f4a040b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:30.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.214+0000 7f4a0a000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4a040d0d40 0x7f4a040cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:30.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.214+0000 7f4a0a000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a0406b0a0 0x7f4a0406b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:30.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.214+0000 7f4a0a000640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a040d20c0 con 0x7f4a040cd740 2024-03-12T17:06:30.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.214+0000 7f4a0a000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a040ce060 con 0x7f4a040d0d40 2024-03-12T17:06:30.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.214+0000 7f4a0a000640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a040d14b0 con 0x7f4a0406b0a0 2024-03-12T17:06:30.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.214+0000 7f4a03400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4a040cd740 0x7f4a040b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:30.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.214+0000 7f4a03400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4a040cd740 0x7f4a040b84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52094/0 (socket says 172.21.15.52:52094) 2024-03-12T17:06:30.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.214+0000 7f4a03400640 1 -- 172.21.15.52:0/3812626881 learned_addr learned my addr 172.21.15.52:0/3812626881 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:30.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.215+0000 7f4a03400640 1 -- 172.21.15.52:0/3812626881 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a0406b0a0 msgr2=0x7f4a0406b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:30.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.215+0000 7f4a03400640 1 --2- 172.21.15.52:0/3812626881 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a0406b0a0 0x7f4a0406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:30.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.215+0000 7f4a03400640 1 -- 172.21.15.52:0/3812626881 >> v2:172.21.15.102:3300/0 conn(0x7f4a040d0d40 msgr2=0x7f4a040cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:30.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.215+0000 7f4a03400640 1 --2- 172.21.15.52:0/3812626881 >> v2:172.21.15.102:3300/0 conn(0x7f4a040d0d40 0x7f4a040cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:30.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.215+0000 7f4a03400640 1 -- 172.21.15.52:0/3812626881 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a040d1790 con 0x7f4a040cd740 2024-03-12T17:06:30.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.216+0000 7f4a03400640 1 --2- 172.21.15.52:0/3812626881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a040cd740 0x7f4a040b84b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4a040b8050 tx=0x7f49f402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16928cfc04820da6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:30.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.216+0000 7f4a02000640 1 -- 172.21.15.52:0/3812626881 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f402ca80 con 0x7f4a040cd740 2024-03-12T17:06:30.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.216+0000 7f4a02000640 1 -- 172.21.15.52:0/3812626881 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f402cbe0 con 0x7f4a040cd740 2024-03-12T17:06:30.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.216+0000 7f4a02000640 1 -- 172.21.15.52:0/3812626881 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f40343f0 con 0x7f4a040cd740 2024-03-12T17:06:30.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.217+0000 7f4a0a000640 1 -- 172.21.15.52:0/3812626881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a040cd740 msgr2=0x7f4a040b84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:30.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.217+0000 7f4a0a000640 1 --2- 172.21.15.52:0/3812626881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a040cd740 0x7f4a040b84b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4a040b8050 tx=0x7f49f402bee0 comp rx=0 tx=0).stop 2024-03-12T17:06:30.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.217+0000 7f4a0a000640 1 -- 172.21.15.52:0/3812626881 shutdown_connections 2024-03-12T17:06:30.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.217+0000 7f4a0a000640 1 --2- 172.21.15.52:0/3812626881 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a0406b0a0 0x7f4a0406b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:30.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.217+0000 7f4a0a000640 1 --2- 172.21.15.52:0/3812626881 >> v2:172.21.15.102:3300/0 conn(0x7f4a040d0d40 0x7f4a040cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:30.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.217+0000 7f4a0a000640 1 --2- 172.21.15.52:0/3812626881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a040cd740 0x7f4a040b84b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:30.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.217+0000 7f4a0a000640 1 -- 172.21.15.52:0/3812626881 >> 172.21.15.52:0/3812626881 conn(0x7f4a040cf5d0 msgr2=0x7f4a040cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:30.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.217+0000 7f4a0a000640 1 -- 172.21.15.52:0/3812626881 shutdown_connections 2024-03-12T17:06:30.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.217+0000 7f4a0a000640 1 -- 172.21.15.52:0/3812626881 wait complete. 2024-03-12T17:06:30.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.219+0000 7f4a0a000640 1 Processor -- start 2024-03-12T17:06:30.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.219+0000 7f4a0a000640 1 -- start start 2024-03-12T17:06:30.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.220+0000 7f4a0a000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a0406b0a0 0x7f4a041f29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:30.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.220+0000 7f4a0a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a040cd740 0x7f4a0414ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:30.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.220+0000 7f4a0a000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a040d0d40 0x7f4a040b6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:30.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.220+0000 7f4a0a000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a040aa100 con 0x7f4a0406b0a0 2024-03-12T17:06:30.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.220+0000 7f4a0a000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a040a9270 con 0x7f4a040cd740 2024-03-12T17:06:30.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.220+0000 7f4a0a000640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a040a9480 con 0x7f4a040d0d40 2024-03-12T17:06:30.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.220+0000 7f4a03400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a0406b0a0 0x7f4a041f29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:30.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.220+0000 7f4a03400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a0406b0a0 0x7f4a041f29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52110/0 (socket says 172.21.15.52:52110) 2024-03-12T17:06:30.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.220+0000 7f4a03400640 1 -- 172.21.15.52:0/3303285882 learned_addr learned my addr 172.21.15.52:0/3303285882 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:30.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.221+0000 7f4a02a00640 1 --2- 172.21.15.52:0/3303285882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a040cd740 0x7f4a0414ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:30.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.221+0000 7f4a03e00640 1 --2- 172.21.15.52:0/3303285882 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a040d0d40 0x7f4a040b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:30.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.221+0000 7f4a03400640 1 -- 172.21.15.52:0/3303285882 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a040d0d40 msgr2=0x7f4a040b6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:30.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.221+0000 7f4a03400640 1 --2- 172.21.15.52:0/3303285882 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a040d0d40 0x7f4a040b6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:30.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.221+0000 7f4a03400640 1 -- 172.21.15.52:0/3303285882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a040cd740 msgr2=0x7f4a0414ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:30.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.221+0000 7f4a03400640 1 --2- 172.21.15.52:0/3303285882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a040cd740 0x7f4a0414ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:30.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.221+0000 7f4a03400640 1 -- 172.21.15.52:0/3303285882 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f4009b30 con 0x7f4a0406b0a0 2024-03-12T17:06:30.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.221+0000 7f4a02a00640 1 --2- 172.21.15.52:0/3303285882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a040cd740 0x7f4a0414ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:30.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.221+0000 7f4a03400640 1 --2- 172.21.15.52:0/3303285882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a0406b0a0 0x7f4a041f29f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f49f4036040 tx=0x7f49f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:30.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.221+0000 7f49fb400640 1 -- 172.21.15.52:0/3303285882 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f4009510 con 0x7f4a0406b0a0 2024-03-12T17:06:30.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.221+0000 7f49fb400640 1 -- 172.21.15.52:0/3303285882 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f403e450 con 0x7f4a0406b0a0 2024-03-12T17:06:30.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.221+0000 7f49fb400640 1 -- 172.21.15.52:0/3303285882 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f403e800 con 0x7f4a0406b0a0 2024-03-12T17:06:30.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.221+0000 7f4a0a000640 1 -- 172.21.15.52:0/3303285882 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a040a9760 con 0x7f4a0406b0a0 2024-03-12T17:06:30.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.222+0000 7f4a0a000640 1 -- 172.21.15.52:0/3303285882 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a040a9d20 con 0x7f4a0406b0a0 2024-03-12T17:06:30.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.223+0000 7f4a0a000640 1 -- 172.21.15.52:0/3303285882 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a040aa420 con 0x7f4a0406b0a0 2024-03-12T17:06:30.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.224+0000 7f49fb400640 1 -- 172.21.15.52:0/3303285882 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f49f402cd50 con 0x7f4a0406b0a0 2024-03-12T17:06:30.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.225+0000 7f49fb400640 1 --2- 172.21.15.52:0/3303285882 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f49e006ac00 0x7f49e006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:30.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.225+0000 7f49fb400640 1 -- 172.21.15.52:0/3303285882 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f49f40ac310 con 0x7f4a0406b0a0 2024-03-12T17:06:30.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.225+0000 7f4a02a00640 1 --2- 172.21.15.52:0/3303285882 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f49e006ac00 0x7f49e006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:30.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.226+0000 7f4a02a00640 1 --2- 172.21.15.52:0/3303285882 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f49e006ac00 0x7f49e006d0c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f49ec006fd0 tx=0x7f49ec008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:30.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.229+0000 7f49fb400640 1 -- 172.21.15.52:0/3303285882 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f4a040aa420 con 0x7f4a0406b0a0 2024-03-12T17:06:30.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.467+0000 7f4a0a000640 1 -- 172.21.15.52:0/3303285882 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f4a040aa420 con 0x7f4a0406b0a0 2024-03-12T17:06:30.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.477+0000 7f49fb400640 1 -- 172.21.15.52:0/3303285882 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7f4a040aa420 con 0x7f4a0406b0a0 2024-03-12T17:06:30.470 INFO:teuthology.orchestra.run.smithi052.stderr:added data pool 4 to fsmap 2024-03-12T17:06:30.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.480+0000 7f4a0a000640 1 -- 172.21.15.52:0/3303285882 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f49e006ac00 msgr2=0x7f49e006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:30.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.480+0000 7f4a0a000640 1 --2- 172.21.15.52:0/3303285882 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f49e006ac00 0x7f49e006d0c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f49ec006fd0 tx=0x7f49ec008040 comp rx=0 tx=0).stop 2024-03-12T17:06:30.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.480+0000 7f4a0a000640 1 -- 172.21.15.52:0/3303285882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a0406b0a0 msgr2=0x7f4a041f29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:30.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.480+0000 7f4a0a000640 1 --2- 172.21.15.52:0/3303285882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a0406b0a0 0x7f4a041f29f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f49f4036040 tx=0x7f49f4004a40 comp rx=0 tx=0).stop 2024-03-12T17:06:30.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.480+0000 7f4a0a000640 1 -- 172.21.15.52:0/3303285882 shutdown_connections 2024-03-12T17:06:30.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.480+0000 7f4a0a000640 1 --2- 172.21.15.52:0/3303285882 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f49e006ac00 0x7f49e006d0c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:30.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.480+0000 7f4a0a000640 1 --2- 172.21.15.52:0/3303285882 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a040d0d40 0x7f4a040b6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:30.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.480+0000 7f4a0a000640 1 --2- 172.21.15.52:0/3303285882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a040cd740 0x7f4a0414ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:30.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.480+0000 7f4a0a000640 1 --2- 172.21.15.52:0/3303285882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a0406b0a0 0x7f4a041f29f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:30.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.480+0000 7f4a0a000640 1 -- 172.21.15.52:0/3303285882 >> 172.21.15.52:0/3303285882 conn(0x7f4a040cf5d0 msgr2=0x7f4a040b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:30.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.481+0000 7f4a0a000640 1 -- 172.21.15.52:0/3303285882 shutdown_connections 2024-03-12T17:06:30.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.481+0000 7f4a0a000640 1 -- 172.21.15.52:0/3303285882 wait complete. 2024-03-12T17:06:30.492 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T17:06:30.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.696+0000 7f39cf200640 1 Processor -- start 2024-03-12T17:06:30.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.696+0000 7f39cf200640 1 -- start start 2024-03-12T17:06:30.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.698+0000 7f39cf200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f39d0143930 0x7f39d00cd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:30.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.698+0000 7f39cf200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f39d00cdc50 0x7f39d0070020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:30.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.698+0000 7f39cf200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39d015b2b0 con 0x7f39d0143460 2024-03-12T17:06:30.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.698+0000 7f39cf200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39d00cc6b0 con 0x7f39d00cdc50 2024-03-12T17:06:30.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.698+0000 7f39cf200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d00cc950 con 0x7f39d0143930 2024-03-12T17:06:30.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.698+0000 7f39ce800640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f39d00cdc50 0x7f39d0070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:30.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.698+0000 7f39ce800640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f39d00cdc50 0x7f39d0070020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52118/0 (socket says 172.21.15.52:52118) 2024-03-12T17:06:30.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.698+0000 7f39ce800640 1 -- 172.21.15.52:0/4069515135 learned_addr learned my addr 172.21.15.52:0/4069515135 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:30.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.698+0000 7f39c5400640 1 --2- 172.21.15.52:0/4069515135 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f39d0143930 0x7f39d00cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:30.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.699+0000 7f39ce800640 1 -- 172.21.15.52:0/4069515135 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f39d0143930 msgr2=0x7f39d00cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:30.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.699+0000 7f39ce800640 1 --2- 172.21.15.52:0/4069515135 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f39d0143930 0x7f39d00cd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:30.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.699+0000 7f39ce800640 1 -- 172.21.15.52:0/4069515135 >> v1:172.21.15.102:6789/0 conn(0x7f39d0143460 legacy=0x7f39d015aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:30.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.699+0000 7f39ce800640 1 -- 172.21.15.52:0/4069515135 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d00ccc30 con 0x7f39d00cdc50 2024-03-12T17:06:30.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.700+0000 7f39ce800640 1 --2- 172.21.15.52:0/4069515135 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39d00cdc50 0x7f39d0070020 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f39c8009870 tx=0x7f39c802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9875152dfbecd15 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:30.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.700+0000 7f39cd400640 1 -- 172.21.15.52:0/4069515135 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c8030840 con 0x7f39d00cdc50 2024-03-12T17:06:30.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.700+0000 7f39cd400640 1 -- 172.21.15.52:0/4069515135 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c80309a0 con 0x7f39d00cdc50 2024-03-12T17:06:30.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.700+0000 7f39cd400640 1 -- 172.21.15.52:0/4069515135 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c8030d30 con 0x7f39d00cdc50 2024-03-12T17:06:30.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.701+0000 7f39cf200640 1 -- 172.21.15.52:0/4069515135 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39d00cdc50 msgr2=0x7f39d0070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:30.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.701+0000 7f39cf200640 1 --2- 172.21.15.52:0/4069515135 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39d00cdc50 0x7f39d0070020 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f39c8009870 tx=0x7f39c802b980 comp rx=0 tx=0).stop 2024-03-12T17:06:30.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.701+0000 7f39cf200640 1 -- 172.21.15.52:0/4069515135 shutdown_connections 2024-03-12T17:06:30.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.701+0000 7f39cf200640 1 --2- 172.21.15.52:0/4069515135 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39d00cdc50 0x7f39d0070020 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:30.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.701+0000 7f39cf200640 1 --2- 172.21.15.52:0/4069515135 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f39d0143930 0x7f39d00cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:30.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.701+0000 7f39cf200640 1 -- 172.21.15.52:0/4069515135 >> 172.21.15.52:0/4069515135 conn(0x7f39d0065df0 msgr2=0x7f39d0069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:30.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.701+0000 7f39cf200640 1 -- 172.21.15.52:0/4069515135 shutdown_connections 2024-03-12T17:06:30.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.701+0000 7f39cf200640 1 -- 172.21.15.52:0/4069515135 wait complete. 2024-03-12T17:06:30.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.703+0000 7f39cf200640 1 Processor -- start 2024-03-12T17:06:30.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.703+0000 7f39cf200640 1 -- start start 2024-03-12T17:06:30.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.704+0000 7f39cf200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39d00cdc50 0x7f39d006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:30.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.704+0000 7f39cf200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39d0143460 0x7f39d015b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:30.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.704+0000 7f39cf200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f39d0143930 0x7f39d0141db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:30.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.704+0000 7f39cf200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d013d5e0 con 0x7f39d00cdc50 2024-03-12T17:06:30.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.704+0000 7f39cf200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d013d8c0 con 0x7f39d0143460 2024-03-12T17:06:30.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.704+0000 7f39cf200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d013dba0 con 0x7f39d0143930 2024-03-12T17:06:30.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.705+0000 7f39cde00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39d00cdc50 0x7f39d006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:30.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.705+0000 7f39cde00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39d00cdc50 0x7f39d006d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52120/0 (socket says 172.21.15.52:52120) 2024-03-12T17:06:30.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.705+0000 7f39cde00640 1 -- 172.21.15.52:0/1534154948 learned_addr learned my addr 172.21.15.52:0/1534154948 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:30.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.705+0000 7f39ce800640 1 --2- 172.21.15.52:0/1534154948 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f39d0143930 0x7f39d0141db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:30.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.705+0000 7f39c5400640 1 --2- 172.21.15.52:0/1534154948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39d0143460 0x7f39d015b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:30.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.705+0000 7f39cde00640 1 -- 172.21.15.52:0/1534154948 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f39d0143930 msgr2=0x7f39d0141db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:30.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.705+0000 7f39cde00640 1 --2- 172.21.15.52:0/1534154948 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f39d0143930 0x7f39d0141db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:30.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.705+0000 7f39cde00640 1 -- 172.21.15.52:0/1534154948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39d0143460 msgr2=0x7f39d015b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:30.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.705+0000 7f39cde00640 1 --2- 172.21.15.52:0/1534154948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39d0143460 0x7f39d015b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:30.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.705+0000 7f39cde00640 1 -- 172.21.15.52:0/1534154948 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c80094d0 con 0x7f39d00cdc50 2024-03-12T17:06:30.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.705+0000 7f39c5400640 1 --2- 172.21.15.52:0/1534154948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39d0143460 0x7f39d015b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:30.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.706+0000 7f39cde00640 1 --2- 172.21.15.52:0/1534154948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39d00cdc50 0x7f39d006d9e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f39c000c7c0 tx=0x7f39c000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:30.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.706+0000 7f39c6a00640 1 -- 172.21.15.52:0/1534154948 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c000aa50 con 0x7f39d00cdc50 2024-03-12T17:06:30.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.706+0000 7f39c6a00640 1 -- 172.21.15.52:0/1534154948 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c000abb0 con 0x7f39d00cdc50 2024-03-12T17:06:30.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.706+0000 7f39cf200640 1 -- 172.21.15.52:0/1534154948 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39d013de80 con 0x7f39d00cdc50 2024-03-12T17:06:30.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.706+0000 7f39c6a00640 1 -- 172.21.15.52:0/1534154948 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c0012420 con 0x7f39d00cdc50 2024-03-12T17:06:30.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.706+0000 7f39cf200640 1 -- 172.21.15.52:0/1534154948 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d006df20 con 0x7f39d00cdc50 2024-03-12T17:06:30.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.707+0000 7f39cf200640 1 -- 172.21.15.52:0/1534154948 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39d006e110 con 0x7f39d00cdc50 2024-03-12T17:06:30.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.708+0000 7f39c6a00640 1 -- 172.21.15.52:0/1534154948 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f39c0012580 con 0x7f39d00cdc50 2024-03-12T17:06:30.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.709+0000 7f39c6a00640 1 --2- 172.21.15.52:0/1534154948 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f39ac06ac00 0x7f39ac06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:30.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.709+0000 7f39c6a00640 1 -- 172.21.15.52:0/1534154948 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f39c0089660 con 0x7f39d00cdc50 2024-03-12T17:06:30.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.710+0000 7f39c5400640 1 --2- 172.21.15.52:0/1534154948 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f39ac06ac00 0x7f39ac06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:30.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.711+0000 7f39c5400640 1 --2- 172.21.15.52:0/1534154948 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f39ac06ac00 0x7f39ac06d0c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f39b8006fd0 tx=0x7f39b8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:30.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.713+0000 7f39c6a00640 1 -- 172.21.15.52:0/1534154948 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f39c00544b0 con 0x7f39d00cdc50 2024-03-12T17:06:30.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.949+0000 7f39cf200640 1 -- 172.21.15.52:0/1534154948 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39d004e030 con 0x7f39d00cdc50 2024-03-12T17:06:30.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.951+0000 7f39c6a00640 1 -- 172.21.15.52:0/1534154948 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4733 (secure 0 0 0) 0x7f39c0057ad0 con 0x7f39d00cdc50 2024-03-12T17:06:30.943 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:06:30.944 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2950099092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2950099092},{"type":"v1","addr":"172.21.15.52:6834","nonce":2950099092}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/846108993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":846108993},{"type":"v1","addr":"172.21.15.52:6835","nonce":846108993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6836/3002500679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3002500679},{"type":"v1","addr":"172.21.15.102:6836","nonce":3002500679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-12T17:06:25.905186+0000","modified":"2024-03-12T17:06:30.468690+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6837/3333451504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6835","nonce":3333451504},{"type":"v1","addr":"172.21.15.102:6837","nonce":3333451504}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-12T17:06:30.944 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 7 2024-03-12T17:06:30.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.954+0000 7f39cf200640 1 -- 172.21.15.52:0/1534154948 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f39ac06ac00 msgr2=0x7f39ac06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:30.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.954+0000 7f39cf200640 1 --2- 172.21.15.52:0/1534154948 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f39ac06ac00 0x7f39ac06d0c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f39b8006fd0 tx=0x7f39b8008040 comp rx=0 tx=0).stop 2024-03-12T17:06:30.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.954+0000 7f39cf200640 1 -- 172.21.15.52:0/1534154948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39d00cdc50 msgr2=0x7f39d006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:30.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.954+0000 7f39cf200640 1 --2- 172.21.15.52:0/1534154948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39d00cdc50 0x7f39d006d9e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f39c000c7c0 tx=0x7f39c000cc80 comp rx=0 tx=0).stop 2024-03-12T17:06:30.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.954+0000 7f39cf200640 1 -- 172.21.15.52:0/1534154948 shutdown_connections 2024-03-12T17:06:30.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.954+0000 7f39cf200640 1 --2- 172.21.15.52:0/1534154948 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f39ac06ac00 0x7f39ac06d0c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:30.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.954+0000 7f39cf200640 1 --2- 172.21.15.52:0/1534154948 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f39d0143930 0x7f39d0141db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:30.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.954+0000 7f39cf200640 1 --2- 172.21.15.52:0/1534154948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39d0143460 0x7f39d015b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:30.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.954+0000 7f39cf200640 1 --2- 172.21.15.52:0/1534154948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39d00cdc50 0x7f39d006d9e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:30.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.954+0000 7f39cf200640 1 -- 172.21.15.52:0/1534154948 >> 172.21.15.52:0/1534154948 conn(0x7f39d0065df0 msgr2=0x7f39d015aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:30.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.954+0000 7f39cf200640 1 -- 172.21.15.52:0/1534154948 shutdown_connections 2024-03-12T17:06:30.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:30.954+0000 7f39cf200640 1 -- 172.21.15.52:0/1534154948 wait complete. 2024-03-12T17:06:30.964 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:06:31.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.176+0000 7fbc46000640 1 Processor -- start 2024-03-12T17:06:31.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.176+0000 7fbc46000640 1 -- start start 2024-03-12T17:06:31.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.178+0000 7fbc46000640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fbc4015c7c0 0x7fbc400d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:31.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.178+0000 7fbc46000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fbc400cdd40 0x7fbc400cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:31.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.178+0000 7fbc46000640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc400d02a0 con 0x7fbc400ce1f0 2024-03-12T17:06:31.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.178+0000 7fbc46000640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc400d17f0 con 0x7fbc4015c7c0 2024-03-12T17:06:31.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.178+0000 7fbc46000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc400d04f0 con 0x7fbc400cdd40 2024-03-12T17:06:31.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.178+0000 7fbc44c00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fbc4015c7c0 0x7fbc400d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:31.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.178+0000 7fbc45600640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fbc400ce1f0 0x7fbc400b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:42854/0 (socket says 172.21.15.52:42854) 2024-03-12T17:06:31.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.178+0000 7fbc44c00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fbc4015c7c0 0x7fbc400d12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52124/0 (socket says 172.21.15.52:52124) 2024-03-12T17:06:31.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.178+0000 7fbc45600640 1 -- 172.21.15.52:0/2097609551 learned_addr learned my addr 172.21.15.52:0/2097609551 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:06:31.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.179+0000 7fbc3fe00640 1 --2- 172.21.15.52:0/2097609551 >> v2:172.21.15.102:3300/0 conn(0x7fbc400cdd40 0x7fbc400cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:31.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.179+0000 7fbc44c00640 1 -- 172.21.15.52:0/2097609551 >> v2:172.21.15.102:3300/0 conn(0x7fbc400cdd40 msgr2=0x7fbc400cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:31.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.179+0000 7fbc44c00640 1 --2- 172.21.15.52:0/2097609551 >> v2:172.21.15.102:3300/0 conn(0x7fbc400cdd40 0x7fbc400cf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:31.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.179+0000 7fbc44c00640 1 -- 172.21.15.52:0/2097609551 >> v1:172.21.15.52:6789/0 conn(0x7fbc400ce1f0 legacy=0x7fbc400b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:31.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.179+0000 7fbc3f400640 1 -- 172.21.15.52:0/2097609551 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3632300201 0 0) 0x7fbc400d02a0 con 0x7fbc400ce1f0 2024-03-12T17:06:31.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.179+0000 7fbc44c00640 1 -- 172.21.15.52:0/2097609551 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc400d07d0 con 0x7fbc4015c7c0 2024-03-12T17:06:31.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.181+0000 7fbc44c00640 1 --2- 172.21.15.52:0/2097609551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc4015c7c0 0x7fbc400d12b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbc40160380 tx=0x7fbc3402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922db00422494ea4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:31.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.181+0000 7fbc3f400640 1 -- 172.21.15.52:0/2097609551 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc34031830 con 0x7fbc4015c7c0 2024-03-12T17:06:31.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.181+0000 7fbc3f400640 1 -- 172.21.15.52:0/2097609551 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc34031990 con 0x7fbc4015c7c0 2024-03-12T17:06:31.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.181+0000 7fbc3f400640 1 -- 172.21.15.52:0/2097609551 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc34031d20 con 0x7fbc4015c7c0 2024-03-12T17:06:31.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.182+0000 7fbc46000640 1 -- 172.21.15.52:0/2097609551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc4015c7c0 msgr2=0x7fbc400d12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:31.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.182+0000 7fbc46000640 1 --2- 172.21.15.52:0/2097609551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc4015c7c0 0x7fbc400d12b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbc40160380 tx=0x7fbc3402b8d0 comp rx=0 tx=0).stop 2024-03-12T17:06:31.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.182+0000 7fbc46000640 1 -- 172.21.15.52:0/2097609551 shutdown_connections 2024-03-12T17:06:31.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.182+0000 7fbc46000640 1 --2- 172.21.15.52:0/2097609551 >> v2:172.21.15.102:3300/0 conn(0x7fbc400cdd40 0x7fbc400cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:31.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.182+0000 7fbc46000640 1 --2- 172.21.15.52:0/2097609551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc4015c7c0 0x7fbc400d12b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:31.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.182+0000 7fbc46000640 1 -- 172.21.15.52:0/2097609551 >> 172.21.15.52:0/2097609551 conn(0x7fbc40089fd0 msgr2=0x7fbc400d26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:31.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.182+0000 7fbc46000640 1 -- 172.21.15.52:0/2097609551 shutdown_connections 2024-03-12T17:06:31.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.182+0000 7fbc46000640 1 -- 172.21.15.52:0/2097609551 wait complete. 2024-03-12T17:06:31.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.184+0000 7fbc46000640 1 Processor -- start 2024-03-12T17:06:31.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.184+0000 7fbc46000640 1 -- start start 2024-03-12T17:06:31.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.185+0000 7fbc46000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbc400cdd40 0x7fbc40150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:31.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.185+0000 7fbc46000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc400ce1f0 0x7fbc400cb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:31.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.185+0000 7fbc46000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc4015c7c0 0x7fbc40152e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:31.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.185+0000 7fbc46000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc400ca890 con 0x7fbc400ce1f0 2024-03-12T17:06:31.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.185+0000 7fbc46000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc40153be0 con 0x7fbc4015c7c0 2024-03-12T17:06:31.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.185+0000 7fbc46000640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc40153ec0 con 0x7fbc400cdd40 2024-03-12T17:06:31.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.185+0000 7fbc3fe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc400ce1f0 0x7fbc400cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:31.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.185+0000 7fbc3fe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc400ce1f0 0x7fbc400cb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52136/0 (socket says 172.21.15.52:52136) 2024-03-12T17:06:31.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.185+0000 7fbc3fe00640 1 -- 172.21.15.52:0/2911811776 learned_addr learned my addr 172.21.15.52:0/2911811776 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:31.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.186+0000 7fbc45600640 1 --2- 172.21.15.52:0/2911811776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc4015c7c0 0x7fbc40152e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:31.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.186+0000 7fbc44c00640 1 --2- 172.21.15.52:0/2911811776 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbc400cdd40 0x7fbc40150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:31.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.186+0000 7fbc3fe00640 1 -- 172.21.15.52:0/2911811776 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbc400cdd40 msgr2=0x7fbc40150260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:31.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.186+0000 7fbc3fe00640 1 --2- 172.21.15.52:0/2911811776 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbc400cdd40 0x7fbc40150260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:31.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.186+0000 7fbc3fe00640 1 -- 172.21.15.52:0/2911811776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc4015c7c0 msgr2=0x7fbc40152e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:31.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.186+0000 7fbc3fe00640 1 --2- 172.21.15.52:0/2911811776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc4015c7c0 0x7fbc40152e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:31.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.186+0000 7fbc3fe00640 1 -- 172.21.15.52:0/2911811776 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc34009400 con 0x7fbc400ce1f0 2024-03-12T17:06:31.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.186+0000 7fbc45600640 1 --2- 172.21.15.52:0/2911811776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc4015c7c0 0x7fbc40152e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:31.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.186+0000 7fbc3fe00640 1 --2- 172.21.15.52:0/2911811776 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc400ce1f0 0x7fbc400cb050 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c00d800 tx=0x7fbc2c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:31.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.186+0000 7fbc3cc00640 1 -- 172.21.15.52:0/2911811776 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c00b6c0 con 0x7fbc400ce1f0 2024-03-12T17:06:31.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.186+0000 7fbc3cc00640 1 -- 172.21.15.52:0/2911811776 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2c00b820 con 0x7fbc400ce1f0 2024-03-12T17:06:31.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.186+0000 7fbc46000640 1 -- 172.21.15.52:0/2911811776 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc400cab30 con 0x7fbc400ce1f0 2024-03-12T17:06:31.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.187+0000 7fbc3cc00640 1 -- 172.21.15.52:0/2911811776 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c00bbb0 con 0x7fbc400ce1f0 2024-03-12T17:06:31.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.187+0000 7fbc46000640 1 -- 172.21.15.52:0/2911811776 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc401541a0 con 0x7fbc400ce1f0 2024-03-12T17:06:31.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.188+0000 7fbc46000640 1 -- 172.21.15.52:0/2911811776 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc400670d0 con 0x7fbc400ce1f0 2024-03-12T17:06:31.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.189+0000 7fbc3cc00640 1 -- 172.21.15.52:0/2911811776 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fbc2c00bd10 con 0x7fbc400ce1f0 2024-03-12T17:06:31.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.190+0000 7fbc3cc00640 1 --2- 172.21.15.52:0/2911811776 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fbc2006ac00 0x7fbc2006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:31.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.190+0000 7fbc3cc00640 1 -- 172.21.15.52:0/2911811776 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbc2c089d70 con 0x7fbc400ce1f0 2024-03-12T17:06:31.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.190+0000 7fbc44c00640 1 --2- 172.21.15.52:0/2911811776 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fbc2006ac00 0x7fbc2006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:31.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.191+0000 7fbc44c00640 1 --2- 172.21.15.52:0/2911811776 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fbc2006ac00 0x7fbc2006d0c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbc34036000 tx=0x7fbc3403c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:31.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.194+0000 7fbc3cc00640 1 -- 172.21.15.52:0/2911811776 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fbc2c054b30 con 0x7fbc400ce1f0 2024-03-12T17:06:31.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.392+0000 7fbc46000640 1 -- 172.21.15.52:0/2911811776 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc4004e030 con 0x7fbc400ce1f0 2024-03-12T17:06:31.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.395+0000 7fbc3cc00640 1 -- 172.21.15.52:0/2911811776 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7fbc2c058150 con 0x7fbc400ce1f0 2024-03-12T17:06:31.388 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:06:31.388 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":25,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:06:30.468476+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T17:06:19.218543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T17:06:23.525474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T17:06:24.911574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-12T17:06:28.017172+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.733031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.775178+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.591522+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.820838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.829252+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.582481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:06:31.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.397+0000 7fbc46000640 1 -- 172.21.15.52:0/2911811776 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fbc2006ac00 msgr2=0x7fbc2006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:31.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.397+0000 7fbc46000640 1 --2- 172.21.15.52:0/2911811776 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fbc2006ac00 0x7fbc2006d0c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbc34036000 tx=0x7fbc3403c040 comp rx=0 tx=0).stop 2024-03-12T17:06:31.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.397+0000 7fbc46000640 1 -- 172.21.15.52:0/2911811776 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc400ce1f0 msgr2=0x7fbc400cb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:31.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.397+0000 7fbc46000640 1 --2- 172.21.15.52:0/2911811776 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc400ce1f0 0x7fbc400cb050 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c00d800 tx=0x7fbc2c00dcc0 comp rx=0 tx=0).stop 2024-03-12T17:06:31.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.397+0000 7fbc46000640 1 -- 172.21.15.52:0/2911811776 shutdown_connections 2024-03-12T17:06:31.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.397+0000 7fbc46000640 1 --2- 172.21.15.52:0/2911811776 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fbc2006ac00 0x7fbc2006d0c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:31.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.397+0000 7fbc46000640 1 --2- 172.21.15.52:0/2911811776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc4015c7c0 0x7fbc40152e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:31.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.397+0000 7fbc46000640 1 --2- 172.21.15.52:0/2911811776 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc400ce1f0 0x7fbc400cb050 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:31.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.397+0000 7fbc46000640 1 --2- 172.21.15.52:0/2911811776 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbc400cdd40 0x7fbc40150260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:31.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.397+0000 7fbc46000640 1 -- 172.21.15.52:0/2911811776 >> 172.21.15.52:0/2911811776 conn(0x7fbc40089fd0 msgr2=0x7fbc400b5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:31.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.398+0000 7fbc46000640 1 -- 172.21.15.52:0/2911811776 shutdown_connections 2024-03-12T17:06:31.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.398+0000 7fbc46000640 1 -- 172.21.15.52:0/2911811776 wait complete. 2024-03-12T17:06:31.410 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:06:31.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.633+0000 7f968f200640 1 Processor -- start 2024-03-12T17:06:31.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.633+0000 7f968f200640 1 -- start start 2024-03-12T17:06:31.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.635+0000 7f968f200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96900cdc00 0x7f96900b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:31.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.635+0000 7f968f200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f96900cae50 0x7f96900cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:31.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.635+0000 7f968f200640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96901575b0 con 0x7f96901571d0 2024-03-12T17:06:31.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.635+0000 7f968f200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96900cb820 con 0x7f96900cae50 2024-03-12T17:06:31.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.635+0000 7f968f200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9690159f20 con 0x7f96900cdc00 2024-03-12T17:06:31.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.635+0000 7f968e800640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f96900cae50 0x7f96900cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:31.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.636+0000 7f968e800640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f96900cae50 0x7f96900cb2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52138/0 (socket says 172.21.15.52:52138) 2024-03-12T17:06:31.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.636+0000 7f968e800640 1 -- 172.21.15.52:0/793418824 learned_addr learned my addr 172.21.15.52:0/793418824 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:31.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.636+0000 7f968de00640 1 --1- 172.21.15.52:0/793418824 >> v1:172.21.15.52:6789/0 conn(0x7f96901571d0 0x7f969015aaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:42868/0 (socket says 172.21.15.52:42868) 2024-03-12T17:06:31.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.636+0000 7f968d400640 1 --2- 172.21.15.52:0/793418824 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96900cdc00 0x7f96900b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:31.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.636+0000 7f968e800640 1 -- 172.21.15.52:0/793418824 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96900cdc00 msgr2=0x7f96900b35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:31.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.636+0000 7f968e800640 1 --2- 172.21.15.52:0/793418824 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96900cdc00 0x7f96900b35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:31.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.636+0000 7f968e800640 1 -- 172.21.15.52:0/793418824 >> v1:172.21.15.52:6789/0 conn(0x7f96901571d0 legacy=0x7f969015aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:31.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.636+0000 7f968e800640 1 -- 172.21.15.52:0/793418824 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969015a200 con 0x7f96900cae50 2024-03-12T17:06:31.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.638+0000 7f968e800640 1 --2- 172.21.15.52:0/793418824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96900cae50 0x7f96900cb2e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9688009e50 tx=0x7f968802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f4a22593b1be1d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:31.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.638+0000 7f968ca00640 1 -- 172.21.15.52:0/793418824 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968802cb00 con 0x7f96900cae50 2024-03-12T17:06:31.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.638+0000 7f968ca00640 1 -- 172.21.15.52:0/793418824 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f968802cc60 con 0x7f96900cae50 2024-03-12T17:06:31.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.638+0000 7f968ca00640 1 -- 172.21.15.52:0/793418824 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96880343f0 con 0x7f96900cae50 2024-03-12T17:06:31.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.639+0000 7f968f200640 1 -- 172.21.15.52:0/793418824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96900cae50 msgr2=0x7f96900cb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:31.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.639+0000 7f968f200640 1 --2- 172.21.15.52:0/793418824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96900cae50 0x7f96900cb2e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9688009e50 tx=0x7f968802bf60 comp rx=0 tx=0).stop 2024-03-12T17:06:31.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.639+0000 7f968f200640 1 -- 172.21.15.52:0/793418824 shutdown_connections 2024-03-12T17:06:31.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.639+0000 7f968f200640 1 --2- 172.21.15.52:0/793418824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96900cae50 0x7f96900cb2e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:31.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.639+0000 7f968f200640 1 --2- 172.21.15.52:0/793418824 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96900cdc00 0x7f96900b35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:31.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.639+0000 7f968f200640 1 -- 172.21.15.52:0/793418824 >> 172.21.15.52:0/793418824 conn(0x7f9690065df0 msgr2=0x7f9690069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:31.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.639+0000 7f968f200640 1 -- 172.21.15.52:0/793418824 shutdown_connections 2024-03-12T17:06:31.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.639+0000 7f968f200640 1 -- 172.21.15.52:0/793418824 wait complete. 2024-03-12T17:06:31.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.641+0000 7f968f200640 1 Processor -- start 2024-03-12T17:06:31.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.641+0000 7f968f200640 1 -- start start 2024-03-12T17:06:31.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.642+0000 7f968f200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96900cae50 0x7f9690155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:31.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.642+0000 7f968f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96900cdc00 0x7f9690070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:31.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.642+0000 7f968f200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96901571d0 0x7f9690070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:31.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.642+0000 7f968f200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96900b09f0 con 0x7f96900cae50 2024-03-12T17:06:31.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.642+0000 7f968f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96900b0e80 con 0x7f96900cdc00 2024-03-12T17:06:31.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.642+0000 7f968f200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96900afde0 con 0x7f96901571d0 2024-03-12T17:06:31.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.642+0000 7f968de00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96900cae50 0x7f9690155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:31.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.642+0000 7f968de00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96900cae50 0x7f9690155fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52148/0 (socket says 172.21.15.52:52148) 2024-03-12T17:06:31.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.642+0000 7f968de00640 1 -- 172.21.15.52:0/3533941479 learned_addr learned my addr 172.21.15.52:0/3533941479 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:31.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.643+0000 7f968e800640 1 --2- 172.21.15.52:0/3533941479 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96901571d0 0x7f9690070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:31.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.643+0000 7f968d400640 1 --2- 172.21.15.52:0/3533941479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96900cdc00 0x7f9690070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:31.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.643+0000 7f968de00640 1 -- 172.21.15.52:0/3533941479 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96901571d0 msgr2=0x7f9690070560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:31.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.643+0000 7f968de00640 1 --2- 172.21.15.52:0/3533941479 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96901571d0 0x7f9690070560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:31.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.643+0000 7f968de00640 1 -- 172.21.15.52:0/3533941479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96900cdc00 msgr2=0x7f9690070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:31.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.643+0000 7f968de00640 1 --2- 172.21.15.52:0/3533941479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96900cdc00 0x7f9690070020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:31.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.643+0000 7f968de00640 1 -- 172.21.15.52:0/3533941479 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9688009b30 con 0x7f96900cae50 2024-03-12T17:06:31.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.643+0000 7f968d400640 1 --2- 172.21.15.52:0/3533941479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96900cdc00 0x7f9690070020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:31.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.643+0000 7f968de00640 1 --2- 172.21.15.52:0/3533941479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96900cae50 0x7f9690155fa0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f968000c4d0 tx=0x7f968000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:31.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.643+0000 7f9686000640 1 -- 172.21.15.52:0/3533941479 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9680007480 con 0x7f96900cae50 2024-03-12T17:06:31.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.643+0000 7f9686000640 1 -- 172.21.15.52:0/3533941479 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96800075e0 con 0x7f96900cae50 2024-03-12T17:06:31.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.643+0000 7f9686000640 1 -- 172.21.15.52:0/3533941479 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96800049e0 con 0x7f96900cae50 2024-03-12T17:06:31.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.643+0000 7f968f200640 1 -- 172.21.15.52:0/3533941479 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96900b00c0 con 0x7f96900cae50 2024-03-12T17:06:31.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.644+0000 7f968f200640 1 -- 172.21.15.52:0/3533941479 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96900b0680 con 0x7f96900cae50 2024-03-12T17:06:31.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.645+0000 7f968f200640 1 -- 172.21.15.52:0/3533941479 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f969004e030 con 0x7f96900cae50 2024-03-12T17:06:31.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.646+0000 7f9686000640 1 -- 172.21.15.52:0/3533941479 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9680004b40 con 0x7f96900cae50 2024-03-12T17:06:31.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.647+0000 7f9686000640 1 --2- 172.21.15.52:0/3533941479 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f967006ac00 0x7f967006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:31.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.647+0000 7f9686000640 1 -- 172.21.15.52:0/3533941479 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9680088680 con 0x7f96900cae50 2024-03-12T17:06:31.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.647+0000 7f968d400640 1 --2- 172.21.15.52:0/3533941479 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f967006ac00 0x7f967006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:31.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.648+0000 7f968d400640 1 --2- 172.21.15.52:0/3533941479 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f967006ac00 0x7f967006d0c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9690070d00 tx=0x7f967c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:31.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.651+0000 7f9686000640 1 -- 172.21.15.52:0/3533941479 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f9680053620 con 0x7f96900cae50 2024-03-12T17:06:31.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.852+0000 7f968f200640 1 -- 172.21.15.52:0/3533941479 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9690157970 con 0x7f96900cae50 2024-03-12T17:06:31.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.855+0000 7f9686000640 1 -- 172.21.15.52:0/3533941479 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7f9680056c40 con 0x7f96900cae50 2024-03-12T17:06:31.848 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:06:31.848 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":25,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:06:30.468476+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T17:06:19.218543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T17:06:23.525474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T17:06:24.911574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-12T17:06:28.017172+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.733031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.775178+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.591522+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.820838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.829252+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.582481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:06:31.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.858+0000 7f968f200640 1 -- 172.21.15.52:0/3533941479 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f967006ac00 msgr2=0x7f967006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:31.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.858+0000 7f968f200640 1 --2- 172.21.15.52:0/3533941479 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f967006ac00 0x7f967006d0c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9690070d00 tx=0x7f967c008040 comp rx=0 tx=0).stop 2024-03-12T17:06:31.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.858+0000 7f968f200640 1 -- 172.21.15.52:0/3533941479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96900cae50 msgr2=0x7f9690155fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:31.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.858+0000 7f968f200640 1 --2- 172.21.15.52:0/3533941479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96900cae50 0x7f9690155fa0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f968000c4d0 tx=0x7f968000c990 comp rx=0 tx=0).stop 2024-03-12T17:06:31.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.858+0000 7f968f200640 1 -- 172.21.15.52:0/3533941479 shutdown_connections 2024-03-12T17:06:31.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.858+0000 7f968f200640 1 --2- 172.21.15.52:0/3533941479 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f967006ac00 0x7f967006d0c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:31.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.858+0000 7f968f200640 1 --2- 172.21.15.52:0/3533941479 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96901571d0 0x7f9690070560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:31.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.858+0000 7f968f200640 1 --2- 172.21.15.52:0/3533941479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96900cdc00 0x7f9690070020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:31.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.858+0000 7f968f200640 1 --2- 172.21.15.52:0/3533941479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96900cae50 0x7f9690155fa0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:31.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.858+0000 7f968f200640 1 -- 172.21.15.52:0/3533941479 >> 172.21.15.52:0/3533941479 conn(0x7f9690065df0 msgr2=0x7f9690069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:31.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.858+0000 7f968f200640 1 -- 172.21.15.52:0/3533941479 shutdown_connections 2024-03-12T17:06:31.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:31.858+0000 7f968f200640 1 -- 172.21.15.52:0/3533941479 wait complete. 2024-03-12T17:06:31.885 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T17:06:32.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.118+0000 7ff8a7200640 1 Processor -- start 2024-03-12T17:06:32.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.118+0000 7ff8a7200640 1 -- start start 2024-03-12T17:06:32.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.120+0000 7ff8a7200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff8a80b3310 0x7ff8a80cae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:32.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.120+0000 7ff8a7200640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8a80cc8c0 con 0x7ff8a8156e30 2024-03-12T17:06:32.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.120+0000 7ff8a7200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8a80cb7c0 con 0x7ff8a80cb370 2024-03-12T17:06:32.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.120+0000 7ff8a7200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8a815ce60 con 0x7ff8a80b3310 2024-03-12T17:06:32.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.120+0000 7ff8a5e00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7ff8a8156e30 0x7ff8a806d0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:42882/0 (socket says 172.21.15.52:42882) 2024-03-12T17:06:32.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.120+0000 7ff8a5e00640 1 -- 172.21.15.52:0/2012679683 learned_addr learned my addr 172.21.15.52:0/2012679683 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:06:32.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.120+0000 7ff89fe00640 1 --2- 172.21.15.52:0/2012679683 >> v2:172.21.15.52:3300/0 conn(0x7ff8a80b3310 0x7ff8a80cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:32.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.121+0000 7ff8a5400640 1 -- 172.21.15.52:0/2012679683 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1513856710 0 0) 0x7ff8a80cc8c0 con 0x7ff8a8156e30 2024-03-12T17:06:32.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.121+0000 7ff89fe00640 1 -- 172.21.15.52:0/2012679683 >> v1:172.21.15.102:6789/0 conn(0x7ff8a80cb370 legacy=0x7ff8a80b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:32.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.121+0000 7ff89fe00640 1 -- 172.21.15.52:0/2012679683 >> v1:172.21.15.52:6789/0 conn(0x7ff8a8156e30 legacy=0x7ff8a806d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:32.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.121+0000 7ff89fe00640 1 -- 172.21.15.52:0/2012679683 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a815d120 con 0x7ff8a80b3310 2024-03-12T17:06:32.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.122+0000 7ff89fe00640 1 --2- 172.21.15.52:0/2012679683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8a80b3310 0x7ff8a80cae30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff8900097a0 tx=0x7ff89002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efaf4a03fb8e744 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:32.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.122+0000 7ff8a5400640 1 -- 172.21.15.52:0/2012679683 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff890030610 con 0x7ff8a80b3310 2024-03-12T17:06:32.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.122+0000 7ff8a5400640 1 -- 172.21.15.52:0/2012679683 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff890030770 con 0x7ff8a80b3310 2024-03-12T17:06:32.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.122+0000 7ff8a5400640 1 -- 172.21.15.52:0/2012679683 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff890030b00 con 0x7ff8a80b3310 2024-03-12T17:06:32.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.123+0000 7ff8a7200640 1 -- 172.21.15.52:0/2012679683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8a80b3310 msgr2=0x7ff8a80cae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:32.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.123+0000 7ff8a7200640 1 --2- 172.21.15.52:0/2012679683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8a80b3310 0x7ff8a80cae30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff8900097a0 tx=0x7ff89002b960 comp rx=0 tx=0).stop 2024-03-12T17:06:32.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.123+0000 7ff8a7200640 1 -- 172.21.15.52:0/2012679683 shutdown_connections 2024-03-12T17:06:32.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.123+0000 7ff8a7200640 1 --2- 172.21.15.52:0/2012679683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8a80b3310 0x7ff8a80cae30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:32.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.123+0000 7ff8a7200640 1 -- 172.21.15.52:0/2012679683 >> 172.21.15.52:0/2012679683 conn(0x7ff8a8065df0 msgr2=0x7ff8a8069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:32.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.123+0000 7ff8a7200640 1 -- 172.21.15.52:0/2012679683 shutdown_connections 2024-03-12T17:06:32.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.123+0000 7ff8a7200640 1 -- 172.21.15.52:0/2012679683 wait complete. 2024-03-12T17:06:32.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.125+0000 7ff8a7200640 1 Processor -- start 2024-03-12T17:06:32.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.125+0000 7ff8a7200640 1 -- start start 2024-03-12T17:06:32.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.126+0000 7ff8a7200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8a80b3310 0x7ff8a814ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:32.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.126+0000 7ff8a7200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff8a80cb370 0x7ff8a80afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:32.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.126+0000 7ff8a7200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8a8156e30 0x7ff8a814bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:32.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.126+0000 7ff8a7200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a80b0120 con 0x7ff8a8156e30 2024-03-12T17:06:32.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.126+0000 7ff8a7200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a814c000 con 0x7ff8a80b3310 2024-03-12T17:06:32.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.126+0000 7ff8a7200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a814f2c0 con 0x7ff8a80cb370 2024-03-12T17:06:32.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.126+0000 7ff8a6800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8a8156e30 0x7ff8a814bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:32.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.126+0000 7ff8a6800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8a8156e30 0x7ff8a814bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52156/0 (socket says 172.21.15.52:52156) 2024-03-12T17:06:32.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.126+0000 7ff8a6800640 1 -- 172.21.15.52:0/1798306458 learned_addr learned my addr 172.21.15.52:0/1798306458 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:32.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.126+0000 7ff8a5e00640 1 --2- 172.21.15.52:0/1798306458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8a80b3310 0x7ff8a814ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:32.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.126+0000 7ff89fe00640 1 --2- 172.21.15.52:0/1798306458 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff8a80cb370 0x7ff8a80afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:32.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.127+0000 7ff8a6800640 1 -- 172.21.15.52:0/1798306458 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff8a80cb370 msgr2=0x7ff8a80afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:32.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.127+0000 7ff8a6800640 1 --2- 172.21.15.52:0/1798306458 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff8a80cb370 0x7ff8a80afbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:32.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.127+0000 7ff8a6800640 1 -- 172.21.15.52:0/1798306458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8a80b3310 msgr2=0x7ff8a814ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:32.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.127+0000 7ff8a6800640 1 --2- 172.21.15.52:0/1798306458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8a80b3310 0x7ff8a814ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:32.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.127+0000 7ff8a6800640 1 -- 172.21.15.52:0/1798306458 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff890009400 con 0x7ff8a8156e30 2024-03-12T17:06:32.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.127+0000 7ff8a5e00640 1 --2- 172.21.15.52:0/1798306458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8a80b3310 0x7ff8a814ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:32.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.127+0000 7ff8a6800640 1 --2- 172.21.15.52:0/1798306458 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8a8156e30 0x7ff8a814bac0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff8a000c7a0 tx=0x7ff8a000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:32.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.127+0000 7ff89e000640 1 -- 172.21.15.52:0/1798306458 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0003b70 con 0x7ff8a8156e30 2024-03-12T17:06:32.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.127+0000 7ff89e000640 1 -- 172.21.15.52:0/1798306458 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8a0003cf0 con 0x7ff8a8156e30 2024-03-12T17:06:32.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.127+0000 7ff8a7200640 1 -- 172.21.15.52:0/1798306458 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8a814f5a0 con 0x7ff8a8156e30 2024-03-12T17:06:32.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.127+0000 7ff89e000640 1 -- 172.21.15.52:0/1798306458 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a000aa90 con 0x7ff8a8156e30 2024-03-12T17:06:32.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.127+0000 7ff8a7200640 1 -- 172.21.15.52:0/1798306458 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8a80af3e0 con 0x7ff8a8156e30 2024-03-12T17:06:32.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.128+0000 7ff8a7200640 1 -- 172.21.15.52:0/1798306458 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8a804e030 con 0x7ff8a8156e30 2024-03-12T17:06:32.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.129+0000 7ff89e000640 1 -- 172.21.15.52:0/1798306458 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff8a000abf0 con 0x7ff8a8156e30 2024-03-12T17:06:32.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.130+0000 7ff89e000640 1 --2- 172.21.15.52:0/1798306458 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff88406ac00 0x7ff88406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:32.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.130+0000 7ff89e000640 1 -- 172.21.15.52:0/1798306458 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff8a0089380 con 0x7ff8a8156e30 2024-03-12T17:06:32.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.131+0000 7ff8a5e00640 1 --2- 172.21.15.52:0/1798306458 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff88406ac00 0x7ff88406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:32.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.132+0000 7ff8a5e00640 1 --2- 172.21.15.52:0/1798306458 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff88406ac00 0x7ff88406d0c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff898000f60 tx=0x7ff898009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:32.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.135+0000 7ff89e000640 1 -- 172.21.15.52:0/1798306458 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7ff8a00541d0 con 0x7ff8a8156e30 2024-03-12T17:06:32.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.364+0000 7ff8a7200640 1 -- 172.21.15.52:0/1798306458 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8a80cb7c0 con 0x7ff8a8156e30 2024-03-12T17:06:32.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.365+0000 7ff89e000640 1 -- 172.21.15.52:0/1798306458 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4733 (secure 0 0 0) 0x7ff8a80cb7c0 con 0x7ff8a8156e30 2024-03-12T17:06:32.358 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:06:32.358 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2950099092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2950099092},{"type":"v1","addr":"172.21.15.52:6834","nonce":2950099092}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/846108993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":846108993},{"type":"v1","addr":"172.21.15.52:6835","nonce":846108993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6836/3002500679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3002500679},{"type":"v1","addr":"172.21.15.102:6836","nonce":3002500679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-12T17:06:25.905186+0000","modified":"2024-03-12T17:06:30.468690+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6837/3333451504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6835","nonce":3333451504},{"type":"v1","addr":"172.21.15.102:6837","nonce":3333451504}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-12T17:06:32.358 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 7 2024-03-12T17:06:32.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.367+0000 7ff8a7200640 1 -- 172.21.15.52:0/1798306458 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff88406ac00 msgr2=0x7ff88406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:32.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.367+0000 7ff8a7200640 1 --2- 172.21.15.52:0/1798306458 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff88406ac00 0x7ff88406d0c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff898000f60 tx=0x7ff898009040 comp rx=0 tx=0).stop 2024-03-12T17:06:32.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.367+0000 7ff8a7200640 1 -- 172.21.15.52:0/1798306458 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8a8156e30 msgr2=0x7ff8a814bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:32.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.367+0000 7ff8a7200640 1 --2- 172.21.15.52:0/1798306458 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8a8156e30 0x7ff8a814bac0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff8a000c7a0 tx=0x7ff8a000cc60 comp rx=0 tx=0).stop 2024-03-12T17:06:32.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.367+0000 7ff8a7200640 1 -- 172.21.15.52:0/1798306458 shutdown_connections 2024-03-12T17:06:32.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.367+0000 7ff8a7200640 1 --2- 172.21.15.52:0/1798306458 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff88406ac00 0x7ff88406d0c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:32.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.367+0000 7ff8a7200640 1 --2- 172.21.15.52:0/1798306458 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff8a80cb370 0x7ff8a80afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:32.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.367+0000 7ff8a7200640 1 --2- 172.21.15.52:0/1798306458 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8a8156e30 0x7ff8a814bac0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:32.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.367+0000 7ff8a7200640 1 --2- 172.21.15.52:0/1798306458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8a80b3310 0x7ff8a814ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:32.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.367+0000 7ff8a7200640 1 -- 172.21.15.52:0/1798306458 >> 172.21.15.52:0/1798306458 conn(0x7ff8a8065df0 msgr2=0x7ff8a80ac0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:32.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.368+0000 7ff8a7200640 1 -- 172.21.15.52:0/1798306458 shutdown_connections 2024-03-12T17:06:32.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.368+0000 7ff8a7200640 1 -- 172.21.15.52:0/1798306458 wait complete. 2024-03-12T17:06:32.379 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:06:32.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.598+0000 7f6688c00640 1 Processor -- start 2024-03-12T17:06:32.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.599+0000 7f6688c00640 1 -- start start 2024-03-12T17:06:32.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.600+0000 7f6688c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6684156e30 0x7f668406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:32.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.600+0000 7f6688c00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f66840b3310 0x7f66840cae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:32.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.600+0000 7f6688c00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66840cc8c0 con 0x7f66840cb370 2024-03-12T17:06:32.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.600+0000 7f6688c00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66840cb7e0 con 0x7f6684156e30 2024-03-12T17:06:32.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.600+0000 7f6688c00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f668406d630 con 0x7f66840b3310 2024-03-12T17:06:32.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.600+0000 7f667aa00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f66840b3310 0x7f66840cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:32.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.601+0000 7f6683400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6684156e30 0x7f668406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:32.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.601+0000 7f667aa00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f66840b3310 0x7f66840cae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.52:49210/0 (socket says 172.21.15.52:49210) 2024-03-12T17:06:32.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.601+0000 7f667aa00640 1 -- 172.21.15.52:0/1081604540 learned_addr learned my addr 172.21.15.52:0/1081604540 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:32.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.601+0000 7f6683400640 1 -- 172.21.15.52:0/1081604540 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f66840b3310 msgr2=0x7f66840cae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:32.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.601+0000 7f6683400640 1 --2- 172.21.15.52:0/1081604540 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f66840b3310 0x7f66840cae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:32.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.601+0000 7f6683400640 1 -- 172.21.15.52:0/1081604540 >> v1:172.21.15.102:6789/0 conn(0x7f66840cb370 legacy=0x7f6684070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:32.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.601+0000 7f6683400640 1 -- 172.21.15.52:0/1081604540 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668415ce60 con 0x7f6684156e30 2024-03-12T17:06:32.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.601+0000 7f667aa00640 1 --2- 172.21.15.52:0/1081604540 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f66840b3310 0x7f66840cae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T17:06:32.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.603+0000 7f6683400640 1 --2- 172.21.15.52:0/1081604540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6684156e30 0x7f668406d0f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f66740097a0 tx=0x7f667402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9428950397fe4107 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:32.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.603+0000 7f6682a00640 1 -- 172.21.15.52:0/1081604540 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667402d040 con 0x7f6684156e30 2024-03-12T17:06:32.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.603+0000 7f6682a00640 1 -- 172.21.15.52:0/1081604540 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6674031810 con 0x7f6684156e30 2024-03-12T17:06:32.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.603+0000 7f6682a00640 1 -- 172.21.15.52:0/1081604540 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6674031ba0 con 0x7f6684156e30 2024-03-12T17:06:32.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.604+0000 7f6688c00640 1 -- 172.21.15.52:0/1081604540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6684156e30 msgr2=0x7f668406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:32.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.604+0000 7f6688c00640 1 --2- 172.21.15.52:0/1081604540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6684156e30 0x7f668406d0f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f66740097a0 tx=0x7f667402b8b0 comp rx=0 tx=0).stop 2024-03-12T17:06:32.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.604+0000 7f6688c00640 1 -- 172.21.15.52:0/1081604540 shutdown_connections 2024-03-12T17:06:32.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.604+0000 7f6688c00640 1 --2- 172.21.15.52:0/1081604540 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f66840b3310 0x7f66840cae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:32.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.604+0000 7f6688c00640 1 --2- 172.21.15.52:0/1081604540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6684156e30 0x7f668406d0f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:32.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.604+0000 7f6688c00640 1 -- 172.21.15.52:0/1081604540 >> 172.21.15.52:0/1081604540 conn(0x7f6684065df0 msgr2=0x7f6684069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:32.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.604+0000 7f6688c00640 1 -- 172.21.15.52:0/1081604540 shutdown_connections 2024-03-12T17:06:32.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.604+0000 7f6688c00640 1 -- 172.21.15.52:0/1081604540 wait complete. 2024-03-12T17:06:32.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.606+0000 7f6688c00640 1 Processor -- start 2024-03-12T17:06:32.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.606+0000 7f6688c00640 1 -- start start 2024-03-12T17:06:32.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.607+0000 7f6688c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66840b3310 0x7f6684157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:32.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.607+0000 7f6688c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f66840cb370 0x7f6684154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:32.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.607+0000 7f6688c00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6684156e30 0x7f66841f2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:32.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.607+0000 7f6688c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6684159bf0 con 0x7f66840b3310 2024-03-12T17:06:32.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.607+0000 7f6688c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6684158fe0 con 0x7f66840cb370 2024-03-12T17:06:32.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.607+0000 7f6688c00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66841592c0 con 0x7f6684156e30 2024-03-12T17:06:32.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.607+0000 7f6683400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66840b3310 0x7f6684157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:32.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.607+0000 7f6683400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66840b3310 0x7f6684157bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52166/0 (socket says 172.21.15.52:52166) 2024-03-12T17:06:32.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.607+0000 7f6683400640 1 -- 172.21.15.52:0/2188048367 learned_addr learned my addr 172.21.15.52:0/2188048367 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:32.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.608+0000 7f667aa00640 1 --2- 172.21.15.52:0/2188048367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f66840cb370 0x7f6684154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:32.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.608+0000 7f6683e00640 1 --2- 172.21.15.52:0/2188048367 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6684156e30 0x7f66841f2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:32.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.608+0000 7f6683400640 1 -- 172.21.15.52:0/2188048367 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6684156e30 msgr2=0x7f66841f2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:32.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.608+0000 7f6683400640 1 --2- 172.21.15.52:0/2188048367 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6684156e30 0x7f66841f2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:32.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.608+0000 7f6683400640 1 -- 172.21.15.52:0/2188048367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f66840cb370 msgr2=0x7f6684154ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:32.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.608+0000 7f6683400640 1 --2- 172.21.15.52:0/2188048367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f66840cb370 0x7f6684154ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:32.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.608+0000 7f6683400640 1 -- 172.21.15.52:0/2188048367 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f666c0094d0 con 0x7f66840b3310 2024-03-12T17:06:32.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.608+0000 7f6683e00640 1 --2- 172.21.15.52:0/2188048367 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6684156e30 0x7f66841f2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:32.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.608+0000 7f667aa00640 1 --2- 172.21.15.52:0/2188048367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f66840cb370 0x7f6684154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:32.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.608+0000 7f6683400640 1 --2- 172.21.15.52:0/2188048367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66840b3310 0x7f6684157bf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6674000c00 tx=0x7f66740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:32.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.609+0000 7f667be00640 1 -- 172.21.15.52:0/2188048367 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6674006e10 con 0x7f66840b3310 2024-03-12T17:06:32.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.609+0000 7f667be00640 1 -- 172.21.15.52:0/2188048367 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667403ad60 con 0x7f66840b3310 2024-03-12T17:06:32.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.609+0000 7f667be00640 1 -- 172.21.15.52:0/2188048367 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6674044630 con 0x7f66840b3310 2024-03-12T17:06:32.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.609+0000 7f6688c00640 1 -- 172.21.15.52:0/2188048367 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6674009400 con 0x7f66840b3310 2024-03-12T17:06:32.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.609+0000 7f6688c00640 1 -- 172.21.15.52:0/2188048367 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6684159880 con 0x7f66840b3310 2024-03-12T17:06:32.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.610+0000 7f6688c00640 1 -- 172.21.15.52:0/2188048367 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f668404e030 con 0x7f66840b3310 2024-03-12T17:06:32.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.611+0000 7f667be00640 1 -- 172.21.15.52:0/2188048367 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f667404d450 con 0x7f66840b3310 2024-03-12T17:06:32.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.612+0000 7f667be00640 1 --2- 172.21.15.52:0/2188048367 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f666006ac00 0x7f666006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:32.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.612+0000 7f667be00640 1 -- 172.21.15.52:0/2188048367 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f66740ab7f0 con 0x7f66840b3310 2024-03-12T17:06:32.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.612+0000 7f667aa00640 1 --2- 172.21.15.52:0/2188048367 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f666006ac00 0x7f666006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:32.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.613+0000 7f667aa00640 1 --2- 172.21.15.52:0/2188048367 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f666006ac00 0x7f666006d0c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6684158de0 tx=0x7f666c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:32.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.616+0000 7f667be00640 1 -- 172.21.15.52:0/2188048367 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f66740767f0 con 0x7f66840b3310 2024-03-12T17:06:32.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.815+0000 7f6688c00640 1 -- 172.21.15.52:0/2188048367 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f668406c110 con 0x7f66840b3310 2024-03-12T17:06:32.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.818+0000 7f667be00640 1 -- 172.21.15.52:0/2188048367 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7f6674079e10 con 0x7f66840b3310 2024-03-12T17:06:32.811 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:06:32.811 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":25,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:06:30.468476+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T17:06:19.218543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T17:06:23.525474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T17:06:24.911574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-12T17:06:28.017172+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.733031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.775178+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.591522+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.820838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.829252+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.582481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:06:32.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.821+0000 7f6688c00640 1 -- 172.21.15.52:0/2188048367 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f666006ac00 msgr2=0x7f666006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:32.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.821+0000 7f6688c00640 1 --2- 172.21.15.52:0/2188048367 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f666006ac00 0x7f666006d0c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6684158de0 tx=0x7f666c008040 comp rx=0 tx=0).stop 2024-03-12T17:06:32.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.821+0000 7f6688c00640 1 -- 172.21.15.52:0/2188048367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66840b3310 msgr2=0x7f6684157bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:32.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.821+0000 7f6688c00640 1 --2- 172.21.15.52:0/2188048367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66840b3310 0x7f6684157bf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6674000c00 tx=0x7f66740046c0 comp rx=0 tx=0).stop 2024-03-12T17:06:32.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.822+0000 7f6688c00640 1 -- 172.21.15.52:0/2188048367 shutdown_connections 2024-03-12T17:06:32.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.822+0000 7f6688c00640 1 --2- 172.21.15.52:0/2188048367 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f666006ac00 0x7f666006d0c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:32.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.822+0000 7f6688c00640 1 --2- 172.21.15.52:0/2188048367 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6684156e30 0x7f66841f2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:32.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.822+0000 7f6688c00640 1 --2- 172.21.15.52:0/2188048367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f66840cb370 0x7f6684154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:32.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.822+0000 7f6688c00640 1 --2- 172.21.15.52:0/2188048367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66840b3310 0x7f6684157bf0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:32.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.822+0000 7f6688c00640 1 -- 172.21.15.52:0/2188048367 >> 172.21.15.52:0/2188048367 conn(0x7f6684065df0 msgr2=0x7f6684069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:32.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.822+0000 7f6688c00640 1 -- 172.21.15.52:0/2188048367 shutdown_connections 2024-03-12T17:06:32.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:32.822+0000 7f6688c00640 1 -- 172.21.15.52:0/2188048367 wait complete. 2024-03-12T17:06:33.236 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-12T17:06:25.905186+0000', 'modified': '2024-03-12T17:06:30.468690+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4264}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4264': {'gid': 4264, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.102:6837/3333451504', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6835', 'nonce': 3333451504}, {'type': 'v1', 'addr': '172.21.15.102:6837', 'nonce': 3333451504}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-03-12T17:06:33.236 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-12T17:06:33.236 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-12T17:06:33.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.480+0000 7f59da400640 1 Processor -- start 2024-03-12T17:06:33.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.480+0000 7f59da400640 1 -- start start 2024-03-12T17:06:33.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.481+0000 7f59da400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f59d413faa0 0x7f59d4160f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:33.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.481+0000 7f59da400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f59d4160800 0x7f59d40b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:33.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.481+0000 7f59da400640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59d40b2030 con 0x7f59d40b1be0 2024-03-12T17:06:33.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.481+0000 7f59da400640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d40b38a0 con 0x7f59d4160800 2024-03-12T17:06:33.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.481+0000 7f59da400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d40b3b80 con 0x7f59d413faa0 2024-03-12T17:06:33.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.482+0000 7f59cbe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f59d4160800 0x7f59d40b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:33.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.482+0000 7f59d3e00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f59d40b1be0 0x7f59d41422b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:42898/0 (socket says 172.21.15.52:42898) 2024-03-12T17:06:33.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.482+0000 7f59d3e00640 1 -- 172.21.15.52:0/1941797970 learned_addr learned my addr 172.21.15.52:0/1941797970 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:06:33.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.482+0000 7f59d3400640 1 --2- 172.21.15.52:0/1941797970 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f59d413faa0 0x7f59d4160f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:33.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.482+0000 7f59cbe00640 1 -- 172.21.15.52:0/1941797970 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f59d413faa0 msgr2=0x7f59d4160f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:33.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.482+0000 7f59cbe00640 1 --2- 172.21.15.52:0/1941797970 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f59d413faa0 0x7f59d4160f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.482+0000 7f59cbe00640 1 -- 172.21.15.52:0/1941797970 >> v1:172.21.15.52:6789/0 conn(0x7f59d40b1be0 legacy=0x7f59d41422b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:33.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.482+0000 7f59cbe00640 1 -- 172.21.15.52:0/1941797970 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d40b3e60 con 0x7f59d4160800 2024-03-12T17:06:33.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.483+0000 7f59cbe00640 1 --2- 172.21.15.52:0/1941797970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59d4160800 0x7f59d40b16a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f59bc0097a0 tx=0x7f59bc02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41440c92b01be364 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:33.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.484+0000 7f59d2a00640 1 -- 172.21.15.52:0/1941797970 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59bc02d040 con 0x7f59d4160800 2024-03-12T17:06:33.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.484+0000 7f59d2a00640 1 -- 172.21.15.52:0/1941797970 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59bc031810 con 0x7f59d4160800 2024-03-12T17:06:33.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.484+0000 7f59d2a00640 1 -- 172.21.15.52:0/1941797970 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59bc031ba0 con 0x7f59d4160800 2024-03-12T17:06:33.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.484+0000 7f59da400640 1 -- 172.21.15.52:0/1941797970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59d4160800 msgr2=0x7f59d40b16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:33.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.484+0000 7f59da400640 1 --2- 172.21.15.52:0/1941797970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59d4160800 0x7f59d40b16a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f59bc0097a0 tx=0x7f59bc02b8b0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.485+0000 7f59da400640 1 -- 172.21.15.52:0/1941797970 shutdown_connections 2024-03-12T17:06:33.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.485+0000 7f59da400640 1 --2- 172.21.15.52:0/1941797970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59d4160800 0x7f59d40b16a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.485+0000 7f59da400640 1 --2- 172.21.15.52:0/1941797970 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f59d413faa0 0x7f59d4160f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.485+0000 7f59da400640 1 -- 172.21.15.52:0/1941797970 >> 172.21.15.52:0/1941797970 conn(0x7f59d406a3d0 msgr2=0x7f59d40b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:33.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.485+0000 7f59da400640 1 -- 172.21.15.52:0/1941797970 shutdown_connections 2024-03-12T17:06:33.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.485+0000 7f59da400640 1 -- 172.21.15.52:0/1941797970 wait complete. 2024-03-12T17:06:33.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.486+0000 7f59da400640 1 Processor -- start 2024-03-12T17:06:33.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.487+0000 7f59da400640 1 -- start start 2024-03-12T17:06:33.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.488+0000 7f59da400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59d40b1be0 0x7f59d413c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:33.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.488+0000 7f59da400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f59d413faa0 0x7f59d4154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:33.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.488+0000 7f59da400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59d4160800 0x7f59d4141470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:33.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.488+0000 7f59da400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d4156630 con 0x7f59d40b1be0 2024-03-12T17:06:33.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.488+0000 7f59da400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d4155590 con 0x7f59d4160800 2024-03-12T17:06:33.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.488+0000 7f59da400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d4155870 con 0x7f59d413faa0 2024-03-12T17:06:33.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.488+0000 7f59d3400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59d40b1be0 0x7f59d413c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:33.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.488+0000 7f59d3400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59d40b1be0 0x7f59d413c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52176/0 (socket says 172.21.15.52:52176) 2024-03-12T17:06:33.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.488+0000 7f59d3400640 1 -- 172.21.15.52:0/2584406900 learned_addr learned my addr 172.21.15.52:0/2584406900 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:33.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.488+0000 7f59cbe00640 1 --2- 172.21.15.52:0/2584406900 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f59d413faa0 0x7f59d4154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:33.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.488+0000 7f59d3e00640 1 --2- 172.21.15.52:0/2584406900 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59d4160800 0x7f59d4141470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:33.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.488+0000 7f59d3400640 1 -- 172.21.15.52:0/2584406900 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f59d413faa0 msgr2=0x7f59d4154c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:33.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.488+0000 7f59d3400640 1 --2- 172.21.15.52:0/2584406900 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f59d413faa0 0x7f59d4154c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.488+0000 7f59d3400640 1 -- 172.21.15.52:0/2584406900 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59d4160800 msgr2=0x7f59d4141470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:33.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.488+0000 7f59d3400640 1 --2- 172.21.15.52:0/2584406900 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59d4160800 0x7f59d4141470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.488+0000 7f59d3400640 1 -- 172.21.15.52:0/2584406900 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59bc009400 con 0x7f59d40b1be0 2024-03-12T17:06:33.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.489+0000 7f59d3400640 1 --2- 172.21.15.52:0/2584406900 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59d40b1be0 0x7f59d413c500 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f59c400d800 tx=0x7f59c400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:33.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.489+0000 7f59cb400640 1 -- 172.21.15.52:0/2584406900 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c400b6c0 con 0x7f59d40b1be0 2024-03-12T17:06:33.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.489+0000 7f59cb400640 1 -- 172.21.15.52:0/2584406900 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c400b820 con 0x7f59d40b1be0 2024-03-12T17:06:33.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.489+0000 7f59cb400640 1 -- 172.21.15.52:0/2584406900 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c400bbb0 con 0x7f59d40b1be0 2024-03-12T17:06:33.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.489+0000 7f59da400640 1 -- 172.21.15.52:0/2584406900 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59d4155b50 con 0x7f59d40b1be0 2024-03-12T17:06:33.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.489+0000 7f59da400640 1 -- 172.21.15.52:0/2584406900 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d4156110 con 0x7f59d40b1be0 2024-03-12T17:06:33.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.492+0000 7f59cb400640 1 -- 172.21.15.52:0/2584406900 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f59c400bd10 con 0x7f59d40b1be0 2024-03-12T17:06:33.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.492+0000 7f59cb400640 1 --2- 172.21.15.52:0/2584406900 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f59b006ac00 0x7f59b006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:33.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.493+0000 7f59cb400640 1 -- 172.21.15.52:0/2584406900 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f59c4089d20 con 0x7f59d40b1be0 2024-03-12T17:06:33.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.493+0000 7f59cbe00640 1 --2- 172.21.15.52:0/2584406900 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f59b006ac00 0x7f59b006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:33.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.493+0000 7f59dd1da740 1 Processor -- start 2024-03-12T17:06:33.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.493+0000 7f59dd1da740 1 -- start start 2024-03-12T17:06:33.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.494+0000 7f59cbe00640 1 --2- 172.21.15.52:0/2584406900 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f59b006ac00 0x7f59b006d0c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f59bc036000 tx=0x7f59bc03c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:33.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.494+0000 7f59dd1da740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a0c733ff0 0x563a0c6eb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:33.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.494+0000 7f59dd1da740 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x563a0c6ea070 0x563a0c6ebb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:33.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.494+0000 7f59dd1da740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a0c6e6510 0x563a0c716300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:33.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.494+0000 7f59dd1da740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0c6c1e40 con 0x563a0c6e6510 2024-03-12T17:06:33.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.494+0000 7f59dd1da740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0c657090 con 0x563a0c733ff0 2024-03-12T17:06:33.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.494+0000 7f59dd1da740 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a0c654820 con 0x563a0c6ea070 2024-03-12T17:06:33.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.495+0000 7f59d3e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a0c6e6510 0x563a0c716300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:33.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.495+0000 7f59d3e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a0c6e6510 0x563a0c716300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52178/0 (socket says 172.21.15.52:52178) 2024-03-12T17:06:33.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.495+0000 7f59d3e00640 1 -- 172.21.15.52:0/909804564 learned_addr learned my addr 172.21.15.52:0/909804564 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:33.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.495+0000 7f59d3400640 1 --2- 172.21.15.52:0/909804564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a0c733ff0 0x563a0c6eb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:33.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.495+0000 7f59cbe00640 1 --2- 172.21.15.52:0/909804564 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x563a0c6ea070 0x563a0c6ebb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:33.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.495+0000 7f59d3e00640 1 -- 172.21.15.52:0/909804564 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x563a0c6ea070 msgr2=0x563a0c6ebb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:33.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.495+0000 7f59d3e00640 1 --2- 172.21.15.52:0/909804564 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x563a0c6ea070 0x563a0c6ebb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.495+0000 7f59d3e00640 1 -- 172.21.15.52:0/909804564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a0c733ff0 msgr2=0x563a0c6eb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:33.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.495+0000 7f59d3e00640 1 --2- 172.21.15.52:0/909804564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a0c733ff0 0x563a0c6eb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.495+0000 7f59d3e00640 1 -- 172.21.15.52:0/909804564 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a0c617780 con 0x563a0c6e6510 2024-03-12T17:06:33.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.496+0000 7f59d3e00640 1 --2- 172.21.15.52:0/909804564 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a0c6e6510 0x563a0c716300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f59cc001f80 tx=0x7f59cc00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=134cea5b7028e666 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:33.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.496+0000 7f59c3e00640 1 -- 172.21.15.52:0/909804564 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc00ee50 con 0x563a0c6e6510 2024-03-12T17:06:33.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.496+0000 7f59c3e00640 1 -- 172.21.15.52:0/909804564 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59cc014480 con 0x563a0c6e6510 2024-03-12T17:06:33.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.496+0000 7f59c3e00640 1 -- 172.21.15.52:0/909804564 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc0147f0 con 0x563a0c6e6510 2024-03-12T17:06:33.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.496+0000 7f59dd1da740 1 -- 172.21.15.52:0/909804564 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a0c6e6510 msgr2=0x563a0c716300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:33.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.496+0000 7f59dd1da740 1 --2- 172.21.15.52:0/909804564 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a0c6e6510 0x563a0c716300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f59cc001f80 tx=0x7f59cc00e530 comp rx=0 tx=0).stop 2024-03-12T17:06:33.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.497+0000 7f59dd1da740 1 -- 172.21.15.52:0/909804564 shutdown_connections 2024-03-12T17:06:33.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.497+0000 7f59dd1da740 1 --2- 172.21.15.52:0/909804564 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x563a0c6ea070 0x563a0c6ebb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.497+0000 7f59dd1da740 1 --2- 172.21.15.52:0/909804564 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a0c6e6510 0x563a0c716300 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.497+0000 7f59dd1da740 1 --2- 172.21.15.52:0/909804564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a0c733ff0 0x563a0c6eb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.497+0000 7f59dd1da740 1 -- 172.21.15.52:0/909804564 >> 172.21.15.52:0/909804564 conn(0x563a0c6e8780 msgr2=0x563a0c731300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:33.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.497+0000 7f59dd1da740 1 -- 172.21.15.52:0/909804564 shutdown_connections 2024-03-12T17:06:33.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.497+0000 7f59dd1da740 1 -- 172.21.15.52:0/909804564 wait complete. 2024-03-12T17:06:33.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.497+0000 7f59dd1da740 1 -- start start 2024-03-12T17:06:33.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.498+0000 7f59dd1da740 1 Processor -- start 2024-03-12T17:06:33.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.499+0000 7f59dd1da740 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x563a0c6ea070 0x563a0c7141e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:33.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.499+0000 7f59dd1da740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a0c733ff0 0x563a0c6ff7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:33.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.499+0000 7f59dd1da740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a0c711600 0x563a0c711a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:33.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.499+0000 7f59dd1da740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0c650110 con 0x563a0c733ff0 2024-03-12T17:06:33.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.499+0000 7f59dd1da740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0c5f18c0 con 0x563a0c711600 2024-03-12T17:06:33.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.499+0000 7f59dd1da740 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a0c5f1560 con 0x563a0c6ea070 2024-03-12T17:06:33.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.499+0000 7f59dd1da740 10 client.0 _ll_register_callbacks cb 0x563a0c6498d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T17:06:33.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.499+0000 7f59cbe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a0c733ff0 0x563a0c6ff7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:33.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.499+0000 7f59cbe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a0c733ff0 0x563a0c6ff7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52184/0 (socket says 172.21.15.52:52184) 2024-03-12T17:06:33.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.499+0000 7f59cbe00640 1 -- 172.21.15.52:0/294757636 learned_addr learned my addr 172.21.15.52:0/294757636 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:33.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.499+0000 7f59d3400640 1 --2- 172.21.15.52:0/294757636 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x563a0c6ea070 0x563a0c7141e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:33.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.499+0000 7f59d3e00640 1 --2- 172.21.15.52:0/294757636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a0c711600 0x563a0c711a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:33.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.499+0000 7f59cbe00640 1 -- 172.21.15.52:0/294757636 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x563a0c6ea070 msgr2=0x563a0c7141e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:33.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.499+0000 7f59cbe00640 1 --2- 172.21.15.52:0/294757636 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x563a0c6ea070 0x563a0c7141e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.499+0000 7f59cbe00640 1 -- 172.21.15.52:0/294757636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a0c711600 msgr2=0x563a0c711a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:33.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.499+0000 7f59cbe00640 1 --2- 172.21.15.52:0/294757636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a0c711600 0x563a0c711a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.499+0000 7f59cbe00640 1 -- 172.21.15.52:0/294757636 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59cc00f040 con 0x563a0c733ff0 2024-03-12T17:06:33.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.500+0000 7f59cbe00640 1 --2- 172.21.15.52:0/294757636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a0c733ff0 0x563a0c6ff7d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f59bc02bec0 tx=0x7f59bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5aa827c245ab8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:33.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.500+0000 7f59c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-03-12T17:06:33.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.500+0000 7f59c0c00640 1 -- 172.21.15.52:0/294757636 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59bc044a90 con 0x563a0c733ff0 2024-03-12T17:06:33.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.500+0000 7f59c0c00640 1 -- 172.21.15.52:0/294757636 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f59bc03f070 con 0x563a0c733ff0 2024-03-12T17:06:33.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.500+0000 7f59dd1da740 1 -- 172.21.15.52:0/294757636 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563a0c67a660 con 0x563a0c733ff0 2024-03-12T17:06:33.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.500+0000 7f59c0c00640 1 -- 172.21.15.52:0/294757636 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59bc044bf0 con 0x563a0c733ff0 2024-03-12T17:06:33.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.500+0000 7f59dd1da740 10 client.4410 fetch_fsmap learned FSMap version 7 2024-03-12T17:06:33.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.500+0000 7f59dd1da740 1 -- 172.21.15.52:0/294757636 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x563a0c64e790 con 0x563a0c733ff0 2024-03-12T17:06:33.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.500+0000 7f59c0c00640 1 -- 172.21.15.52:0/294757636 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563a0c67a660 con 0x563a0c733ff0 2024-03-12T17:06:33.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.501+0000 7f59c0c00640 1 -- 172.21.15.52:0/294757636 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 7) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f59bc04e3f0 con 0x563a0c733ff0 2024-03-12T17:06:33.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.501+0000 7f59dd1da740 10 client.4410 fetch_fsmap finished waiting for FSMap version 7 2024-03-12T17:06:33.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.501+0000 7f59dd1da740 10 client.4410 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-12T17:06:33.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.501+0000 7f59dd1da740 20 client.4410 populate_metadata read hostname 'smithi052' 2024-03-12T17:06:33.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.501+0000 7f59dd1da740 1 --2- 172.21.15.52:0/294757636 >> [v2:172.21.15.102:6835/3333451504,v1:172.21.15.102:6837/3333451504] conn(0x563a0c737710 0x563a0c737af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:33.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.501+0000 7f59dd1da740 4 client.4410 mds_command: new command op to 4264 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T17:06:33.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.501+0000 7f59dd1da740 1 -- 172.21.15.52:0/294757636 --> [v2:172.21.15.102:6835/3333451504,v1:172.21.15.102:6837/3333451504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a0c642e50 con 0x563a0c737710 2024-03-12T17:06:33.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.501+0000 7f59d3400640 1 --2- 172.21.15.52:0/294757636 >> [v2:172.21.15.102:6835/3333451504,v1:172.21.15.102:6837/3333451504] conn(0x563a0c737710 0x563a0c737af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:33.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.502+0000 7f59d3400640 1 --2- 172.21.15.52:0/294757636 >> [v2:172.21.15.102:6835/3333451504,v1:172.21.15.102:6837/3333451504] conn(0x563a0c737710 0x563a0c737af0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f9390d123c479b2 server_cookie=ae1ab6e7ff5456d7 in_seq=0 out_seq=0 2024-03-12T17:06:33.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.502+0000 7f59c0c00640 10 client.4410 ms_handle_connect on v2:172.21.15.102:6835/3333451504 2024-03-12T17:06:33.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.504+0000 7f59c0c00640 1 -- 172.21.15.52:0/294757636 <== mds.0 v2:172.21.15.102:6835/3333451504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563a0c642e50 con 0x563a0c737710 2024-03-12T17:06:33.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.504+0000 7f59c0c00640 10 client.4410 handle_command_reply: tid=0 2024-03-12T17:06:33.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.505+0000 7f59dd1da740 1 client.4410 shutdown 2024-03-12T17:06:33.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.505+0000 7f59dd1da740 1 -- 172.21.15.52:0/294757636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a0c733ff0 msgr2=0x563a0c6ff7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:33.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.505+0000 7f59dd1da740 1 --2- 172.21.15.52:0/294757636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a0c733ff0 0x563a0c6ff7d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f59bc02bec0 tx=0x7f59bc004730 comp rx=0 tx=0).stop 2024-03-12T17:06:33.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.505+0000 7f59dd1da740 1 -- 172.21.15.52:0/294757636 shutdown_connections 2024-03-12T17:06:33.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.505+0000 7f59dd1da740 1 --2- 172.21.15.52:0/294757636 >> [v2:172.21.15.102:6835/3333451504,v1:172.21.15.102:6837/3333451504] conn(0x563a0c737710 0x563a0c737af0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.505+0000 7f59dd1da740 1 --2- 172.21.15.52:0/294757636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a0c711600 0x563a0c711a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.505+0000 7f59dd1da740 1 --2- 172.21.15.52:0/294757636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a0c733ff0 0x563a0c6ff7d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.505+0000 7f59dd1da740 1 --2- 172.21.15.52:0/294757636 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x563a0c6ea070 0x563a0c7141e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.505+0000 7f59c0c00640 0 client.4410 ms_handle_reset on v2:172.21.15.102:6835/3333451504 2024-03-12T17:06:33.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.505+0000 7f59dd1da740 1 -- 172.21.15.52:0/294757636 >> 172.21.15.52:0/294757636 conn(0x563a0c6e6510 msgr2=0x563a0c6ea790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:33.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.505+0000 7f59dd1da740 1 -- 172.21.15.52:0/294757636 shutdown_connections 2024-03-12T17:06:33.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.505+0000 7f59dd1da740 1 -- 172.21.15.52:0/294757636 wait complete. 2024-03-12T17:06:33.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.505+0000 7f59dd1da740 20 client.4410 trim_cache size 0 max 16384 2024-03-12T17:06:33.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.522+0000 7f59dd1da740 1 Processor -- start 2024-03-12T17:06:33.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.522+0000 7f59dd1da740 1 -- start start 2024-03-12T17:06:33.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.523+0000 7f59dd1da740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a0c6ea070 0x563a0c6eb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:33.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.523+0000 7f59dd1da740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a0c711600 0x563a0c7318e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:33.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.523+0000 7f59dd1da740 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x563a0c733ff0 0x563a0c6e55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:33.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.523+0000 7f59dd1da740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0c3b7f20 con 0x563a0c6ea070 2024-03-12T17:06:33.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.523+0000 7f59dd1da740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0c715200 con 0x563a0c711600 2024-03-12T17:06:33.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.523+0000 7f59dd1da740 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a0c6e7260 con 0x563a0c733ff0 2024-03-12T17:06:33.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.523+0000 7f59d3400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a0c6ea070 0x563a0c6eb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:33.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.523+0000 7f59d3400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a0c6ea070 0x563a0c6eb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52188/0 (socket says 172.21.15.52:52188) 2024-03-12T17:06:33.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.523+0000 7f59d3400640 1 -- 172.21.15.52:0/4264031493 learned_addr learned my addr 172.21.15.52:0/4264031493 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:33.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.524+0000 7f59d3400640 1 -- 172.21.15.52:0/4264031493 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x563a0c733ff0 msgr2=0x563a0c6e55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:33.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.524+0000 7f59d3400640 1 --2- 172.21.15.52:0/4264031493 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x563a0c733ff0 0x563a0c6e55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.524+0000 7f59d3400640 1 -- 172.21.15.52:0/4264031493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a0c711600 msgr2=0x563a0c7318e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:33.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.524+0000 7f59d3400640 1 --2- 172.21.15.52:0/4264031493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a0c711600 0x563a0c7318e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.524+0000 7f59d3400640 1 -- 172.21.15.52:0/4264031493 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a0c5f0ad0 con 0x563a0c6ea070 2024-03-12T17:06:33.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.524+0000 7f59d3400640 1 --2- 172.21.15.52:0/4264031493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a0c6ea070 0x563a0c6eb3c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f59c4006f00 tx=0x7f59c4087d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f94cc1dc576bcb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:33.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.524+0000 7f59c8c00640 1 -- 172.21.15.52:0/4264031493 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c4087fd0 con 0x563a0c6ea070 2024-03-12T17:06:33.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.524+0000 7f59c8c00640 1 -- 172.21.15.52:0/4264031493 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c4056b40 con 0x563a0c6ea070 2024-03-12T17:06:33.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.524+0000 7f59c8c00640 1 -- 172.21.15.52:0/4264031493 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c4054eb0 con 0x563a0c6ea070 2024-03-12T17:06:33.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.525+0000 7f59dd1da740 1 -- 172.21.15.52:0/4264031493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a0c6ea070 msgr2=0x563a0c6eb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:33.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.525+0000 7f59dd1da740 1 --2- 172.21.15.52:0/4264031493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a0c6ea070 0x563a0c6eb3c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f59c4006f00 tx=0x7f59c4087d90 comp rx=0 tx=0).stop 2024-03-12T17:06:33.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.525+0000 7f59dd1da740 1 -- 172.21.15.52:0/4264031493 shutdown_connections 2024-03-12T17:06:33.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.525+0000 7f59dd1da740 1 --2- 172.21.15.52:0/4264031493 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x563a0c733ff0 0x563a0c6e55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.525+0000 7f59dd1da740 1 --2- 172.21.15.52:0/4264031493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a0c711600 0x563a0c7318e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.525+0000 7f59dd1da740 1 --2- 172.21.15.52:0/4264031493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a0c6ea070 0x563a0c6eb3c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.525+0000 7f59dd1da740 1 -- 172.21.15.52:0/4264031493 >> 172.21.15.52:0/4264031493 conn(0x563a0c6e6510 msgr2=0x563a0c738ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:33.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.525+0000 7f59dd1da740 1 -- 172.21.15.52:0/4264031493 shutdown_connections 2024-03-12T17:06:33.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.525+0000 7f59dd1da740 1 -- 172.21.15.52:0/4264031493 wait complete. 2024-03-12T17:06:33.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.525+0000 7f59dd1da740 1 -- start start 2024-03-12T17:06:33.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.526+0000 7f59dd1da740 1 Processor -- start 2024-03-12T17:06:33.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.527+0000 7f59dd1da740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a0c6ea070 0x563a0c6e5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:33.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.527+0000 7f59dd1da740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a0c711600 0x563a0c6fe990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:33.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.527+0000 7f59dd1da740 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x563a0c733ff0 0x563a0c79c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:33.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.527+0000 7f59dd1da740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0c6feed0 con 0x563a0c6ea070 2024-03-12T17:06:33.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.527+0000 7f59dd1da740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0c7127d0 con 0x563a0c711600 2024-03-12T17:06:33.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.527+0000 7f59dd1da740 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a0c730e70 con 0x563a0c733ff0 2024-03-12T17:06:33.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.527+0000 7f59dd1da740 10 client.0 _ll_register_callbacks cb 0x563a0c731f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T17:06:33.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.527+0000 7f59d3400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a0c6ea070 0x563a0c6e5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:33.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.527+0000 7f59d3400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a0c6ea070 0x563a0c6e5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52194/0 (socket says 172.21.15.52:52194) 2024-03-12T17:06:33.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.527+0000 7f59d3400640 1 -- 172.21.15.52:0/3927310192 learned_addr learned my addr 172.21.15.52:0/3927310192 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:33.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.527+0000 7f59cbe00640 1 --2- 172.21.15.52:0/3927310192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a0c711600 0x563a0c6fe990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:33.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.527+0000 7f59d3e00640 1 --2- 172.21.15.52:0/3927310192 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x563a0c733ff0 0x563a0c79c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:33.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.527+0000 7f59d3400640 1 -- 172.21.15.52:0/3927310192 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x563a0c733ff0 msgr2=0x563a0c79c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:33.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.527+0000 7f59d3400640 1 --2- 172.21.15.52:0/3927310192 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x563a0c733ff0 0x563a0c79c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.527+0000 7f59d3400640 1 -- 172.21.15.52:0/3927310192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a0c711600 msgr2=0x563a0c6fe990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:33.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.527+0000 7f59d3400640 1 --2- 172.21.15.52:0/3927310192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a0c711600 0x563a0c6fe990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.527+0000 7f59d3400640 1 -- 172.21.15.52:0/3927310192 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d4156110 con 0x563a0c6ea070 2024-03-12T17:06:33.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.528+0000 7f59d3e00640 1 --2- 172.21.15.52:0/3927310192 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x563a0c733ff0 0x563a0c79c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:33.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.528+0000 7f59d3400640 1 --2- 172.21.15.52:0/3927310192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a0c6ea070 0x563a0c6e5ae0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f59c400d7d0 tx=0x7f59bc030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc88948e6382ab45 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:33.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.528+0000 7f59c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-03-12T17:06:33.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.528+0000 7f59c0c00640 1 -- 172.21.15.52:0/3927310192 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c4054eb0 con 0x563a0c6ea070 2024-03-12T17:06:33.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.528+0000 7f59c0c00640 1 -- 172.21.15.52:0/3927310192 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f59c4055010 con 0x563a0c6ea070 2024-03-12T17:06:33.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.528+0000 7f59dd1da740 1 -- 172.21.15.52:0/3927310192 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f59cc014480 con 0x563a0c6ea070 2024-03-12T17:06:33.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.528+0000 7f59c0c00640 1 -- 172.21.15.52:0/3927310192 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59c4057c90 con 0x563a0c6ea070 2024-03-12T17:06:33.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.528+0000 7f59dd1da740 10 client.4416 fetch_fsmap learned FSMap version 7 2024-03-12T17:06:33.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.528+0000 7f59dd1da740 1 -- 172.21.15.52:0/3927310192 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x563a0c615fe0 con 0x563a0c6ea070 2024-03-12T17:06:33.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.529+0000 7f59c0c00640 1 -- 172.21.15.52:0/3927310192 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59cc014480 con 0x563a0c6ea070 2024-03-12T17:06:33.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.529+0000 7f59c0c00640 1 -- 172.21.15.52:0/3927310192 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 7) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f59c4060790 con 0x563a0c6ea070 2024-03-12T17:06:33.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.529+0000 7f59dd1da740 10 client.4416 fetch_fsmap finished waiting for FSMap version 7 2024-03-12T17:06:33.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.529+0000 7f59dd1da740 10 client.4416 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-12T17:06:33.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.529+0000 7f59dd1da740 20 client.4416 populate_metadata read hostname 'smithi052' 2024-03-12T17:06:33.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.529+0000 7f59dd1da740 1 --2- 172.21.15.52:0/3927310192 >> [v2:172.21.15.102:6835/3333451504,v1:172.21.15.102:6837/3333451504] conn(0x563a0c7120d0 0x563a0c737af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:33.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.529+0000 7f59dd1da740 4 client.4416 mds_command: new command op to 4264 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-12T17:06:33.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.529+0000 7f59dd1da740 1 -- 172.21.15.52:0/3927310192 --> [v2:172.21.15.102:6835/3333451504,v1:172.21.15.102:6837/3333451504] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563a0c3443c0 con 0x563a0c7120d0 2024-03-12T17:06:33.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.529+0000 7f59cbe00640 1 --2- 172.21.15.52:0/3927310192 >> [v2:172.21.15.102:6835/3333451504,v1:172.21.15.102:6837/3333451504] conn(0x563a0c7120d0 0x563a0c737af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:33.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.530+0000 7f59cbe00640 1 --2- 172.21.15.52:0/3927310192 >> [v2:172.21.15.102:6835/3333451504,v1:172.21.15.102:6837/3333451504] conn(0x563a0c7120d0 0x563a0c737af0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22a367d3a15fb01 server_cookie=d24168d43f907316 in_seq=0 out_seq=0 2024-03-12T17:06:33.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.530+0000 7f59c0c00640 10 client.4416 ms_handle_connect on v2:172.21.15.102:6835/3333451504 2024-03-12T17:06:33.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.531+0000 7f59c0c00640 1 -- 172.21.15.52:0/3927310192 <== mds.0 v2:172.21.15.102:6835/3333451504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x563a0c3443c0 con 0x563a0c7120d0 2024-03-12T17:06:33.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.531+0000 7f59c0c00640 10 client.4416 handle_command_reply: tid=0 2024-03-12T17:06:33.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.531+0000 7f59dd1da740 1 client.4416 shutdown 2024-03-12T17:06:33.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.531+0000 7f59dd1da740 1 -- 172.21.15.52:0/3927310192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a0c6ea070 msgr2=0x563a0c6e5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:33.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.531+0000 7f59dd1da740 1 --2- 172.21.15.52:0/3927310192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a0c6ea070 0x563a0c6e5ae0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f59c400d7d0 tx=0x7f59bc030000 comp rx=0 tx=0).stop 2024-03-12T17:06:33.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.531+0000 7f59dd1da740 1 -- 172.21.15.52:0/3927310192 shutdown_connections 2024-03-12T17:06:33.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.531+0000 7f59dd1da740 1 --2- 172.21.15.52:0/3927310192 >> [v2:172.21.15.102:6835/3333451504,v1:172.21.15.102:6837/3333451504] conn(0x563a0c7120d0 0x563a0c737af0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.531+0000 7f59dd1da740 1 --2- 172.21.15.52:0/3927310192 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x563a0c733ff0 0x563a0c79c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.531+0000 7f59dd1da740 1 --2- 172.21.15.52:0/3927310192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a0c711600 0x563a0c6fe990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.531+0000 7f59dd1da740 1 --2- 172.21.15.52:0/3927310192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a0c6ea070 0x563a0c6e5ae0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.531+0000 7f59c0c00640 0 client.4416 ms_handle_reset on v2:172.21.15.102:6835/3333451504 2024-03-12T17:06:33.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.531+0000 7f59dd1da740 1 -- 172.21.15.52:0/3927310192 >> 172.21.15.52:0/3927310192 conn(0x563a0c6e6510 msgr2=0x563a0c6ebd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:33.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.531+0000 7f59dd1da740 1 -- 172.21.15.52:0/3927310192 shutdown_connections 2024-03-12T17:06:33.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.531+0000 7f59dd1da740 1 -- 172.21.15.52:0/3927310192 wait complete. 2024-03-12T17:06:33.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.532+0000 7f59dd1da740 20 client.4416 trim_cache size 0 max 16384 2024-03-12T17:06:33.527 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-03-12T17:06:33.527 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3", 2024-03-12T17:06:33.527 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 0, 2024-03-12T17:06:33.527 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 4264, 2024-03-12T17:06:33.528 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-03-12T17:06:33.528 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-03-12T17:06:33.528 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-03-12T17:06:33.528 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 7.6191170289999999, 2024-03-12T17:06:33.528 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 7, 2024-03-12T17:06:33.528 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 25, 2024-03-12T17:06:33.528 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 24, 2024-03-12T17:06:33.528 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 15.159892038000001 2024-03-12T17:06:33.528 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-03-12T17:06:33.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.532+0000 7f59da400640 1 -- 172.21.15.52:0/2584406900 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f59b006ac00 msgr2=0x7f59b006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:33.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.532+0000 7f59da400640 1 --2- 172.21.15.52:0/2584406900 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f59b006ac00 0x7f59b006d0c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f59bc036000 tx=0x7f59bc03c040 comp rx=0 tx=0).stop 2024-03-12T17:06:33.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.532+0000 7f59da400640 1 -- 172.21.15.52:0/2584406900 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59d40b1be0 msgr2=0x7f59d413c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:33.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.532+0000 7f59da400640 1 --2- 172.21.15.52:0/2584406900 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59d40b1be0 0x7f59d413c500 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f59c400d800 tx=0x7f59c400dcc0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.532+0000 7f59da400640 1 -- 172.21.15.52:0/2584406900 shutdown_connections 2024-03-12T17:06:33.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.532+0000 7f59da400640 1 --2- 172.21.15.52:0/2584406900 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f59b006ac00 0x7f59b006d0c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.532+0000 7f59da400640 1 --2- 172.21.15.52:0/2584406900 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f59d413faa0 0x7f59d4154c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.532+0000 7f59da400640 1 --2- 172.21.15.52:0/2584406900 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59d4160800 0x7f59d4141470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.532+0000 7f59da400640 1 --2- 172.21.15.52:0/2584406900 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59d40b1be0 0x7f59d413c500 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.532+0000 7f59da400640 1 -- 172.21.15.52:0/2584406900 >> 172.21.15.52:0/2584406900 conn(0x7f59d406a3d0 msgr2=0x7f59d40b0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:33.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.532+0000 7f59da400640 1 -- 172.21.15.52:0/2584406900 shutdown_connections 2024-03-12T17:06:33.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.532+0000 7f59da400640 1 -- 172.21.15.52:0/2584406900 wait complete. 2024-03-12T17:06:33.541 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-12T17:06:33.541 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-12T17:06:33.542 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-12T17:06:33.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.756+0000 7f3c9d800640 1 Processor -- start 2024-03-12T17:06:33.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.756+0000 7f3c9d800640 1 -- start start 2024-03-12T17:06:33.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.758+0000 7f3c9d800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3c981571d0 0x7f3c980b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:33.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.758+0000 7f3c9d800640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f3c980cdc00 0x7f3c9815aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:33.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.758+0000 7f3c9d800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c9815afe0 0x7f3c980cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:33.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.758+0000 7f3c9d800640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c981575b0 con 0x7f3c980cdc00 2024-03-12T17:06:33.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.758+0000 7f3c9d800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c98159f20 con 0x7f3c981571d0 2024-03-12T17:06:33.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.758+0000 7f3c9d800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9815a200 con 0x7f3c9815afe0 2024-03-12T17:06:33.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.758+0000 7f3c8fe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f3c980cdc00 0x7f3c9815aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:33.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.758+0000 7f3c8fe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f3c980cdc00 0x7f3c9815aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52196/0 (socket says 172.21.15.52:52196) 2024-03-12T17:06:33.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.758+0000 7f3c8fe00640 1 -- 172.21.15.52:0/1366911452 learned_addr learned my addr 172.21.15.52:0/1366911452 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:33.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.759+0000 7f3c9ce00640 1 --2- 172.21.15.52:0/1366911452 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c9815afe0 0x7f3c980cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:33.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.759+0000 7f3c97e00640 1 --2- 172.21.15.52:0/1366911452 >> v2:172.21.15.102:3300/0 conn(0x7f3c981571d0 0x7f3c980b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:33.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.759+0000 7f3c8fe00640 1 -- 172.21.15.52:0/1366911452 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c9815afe0 msgr2=0x7f3c980cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:33.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.759+0000 7f3c8fe00640 1 --2- 172.21.15.52:0/1366911452 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c9815afe0 0x7f3c980cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.759+0000 7f3c8fe00640 1 -- 172.21.15.52:0/1366911452 >> v2:172.21.15.102:3300/0 conn(0x7f3c981571d0 msgr2=0x7f3c980b35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:33.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.759+0000 7f3c8fe00640 1 --2- 172.21.15.52:0/1366911452 >> v2:172.21.15.102:3300/0 conn(0x7f3c981571d0 0x7f3c980b35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.759+0000 7f3c8fe00640 1 -- 172.21.15.52:0/1366911452 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c9815a4e0 con 0x7f3c980cdc00 2024-03-12T17:06:33.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.760+0000 7f3c8fe00640 1 --2- 172.21.15.52:0/1366911452 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3c980cdc00 0x7f3c9815aaa0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3c980712e0 tx=0x7f3c8002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f779bb15d3f02658 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:33.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.760+0000 7f3c97400640 1 -- 172.21.15.52:0/1366911452 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c80031870 con 0x7f3c980cdc00 2024-03-12T17:06:33.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.760+0000 7f3c97400640 1 -- 172.21.15.52:0/1366911452 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c800319d0 con 0x7f3c980cdc00 2024-03-12T17:06:33.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.760+0000 7f3c97400640 1 -- 172.21.15.52:0/1366911452 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c80031d60 con 0x7f3c980cdc00 2024-03-12T17:06:33.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.761+0000 7f3c9d800640 1 -- 172.21.15.52:0/1366911452 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3c980cdc00 msgr2=0x7f3c9815aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:33.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.761+0000 7f3c9d800640 1 --2- 172.21.15.52:0/1366911452 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3c980cdc00 0x7f3c9815aaa0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3c980712e0 tx=0x7f3c8002b910 comp rx=0 tx=0).stop 2024-03-12T17:06:33.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.761+0000 7f3c9d800640 1 -- 172.21.15.52:0/1366911452 shutdown_connections 2024-03-12T17:06:33.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.761+0000 7f3c9d800640 1 --2- 172.21.15.52:0/1366911452 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c9815afe0 0x7f3c980cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.761+0000 7f3c9d800640 1 --2- 172.21.15.52:0/1366911452 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3c980cdc00 0x7f3c9815aaa0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.761+0000 7f3c9d800640 1 --2- 172.21.15.52:0/1366911452 >> v2:172.21.15.102:3300/0 conn(0x7f3c981571d0 0x7f3c980b35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.761+0000 7f3c9d800640 1 -- 172.21.15.52:0/1366911452 >> 172.21.15.52:0/1366911452 conn(0x7f3c98065df0 msgr2=0x7f3c98069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:33.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.761+0000 7f3c9d800640 1 -- 172.21.15.52:0/1366911452 shutdown_connections 2024-03-12T17:06:33.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.761+0000 7f3c9d800640 1 -- 172.21.15.52:0/1366911452 wait complete. 2024-03-12T17:06:33.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.763+0000 7f3c9d800640 1 Processor -- start 2024-03-12T17:06:33.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.763+0000 7f3c9d800640 1 -- start start 2024-03-12T17:06:33.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.764+0000 7f3c9d800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3c980cdc00 0x7f3c98155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:33.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.764+0000 7f3c9d800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c981571d0 0x7f3c980ac260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:33.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.764+0000 7f3c9d800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c9815afe0 0x7f3c9806d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:33.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.764+0000 7f3c9d800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c980acab0 con 0x7f3c980cdc00 2024-03-12T17:06:33.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.764+0000 7f3c9d800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c98155e80 con 0x7f3c9815afe0 2024-03-12T17:06:33.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.764+0000 7f3c9d800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9806d520 con 0x7f3c981571d0 2024-03-12T17:06:33.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.764+0000 7f3c97e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3c980cdc00 0x7f3c98155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:33.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.764+0000 7f3c97e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3c980cdc00 0x7f3c98155940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52202/0 (socket says 172.21.15.52:52202) 2024-03-12T17:06:33.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.764+0000 7f3c97e00640 1 -- 172.21.15.52:0/644444383 learned_addr learned my addr 172.21.15.52:0/644444383 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:33.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.765+0000 7f3c9ce00640 1 --2- 172.21.15.52:0/644444383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c9815afe0 0x7f3c9806d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:33.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.765+0000 7f3c8fe00640 1 --2- 172.21.15.52:0/644444383 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c981571d0 0x7f3c980ac260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:33.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.765+0000 7f3c97e00640 1 -- 172.21.15.52:0/644444383 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c981571d0 msgr2=0x7f3c980ac260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:33.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.765+0000 7f3c97e00640 1 --2- 172.21.15.52:0/644444383 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c981571d0 0x7f3c980ac260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.765+0000 7f3c97e00640 1 -- 172.21.15.52:0/644444383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c9815afe0 msgr2=0x7f3c9806d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:33.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.765+0000 7f3c97e00640 1 --2- 172.21.15.52:0/644444383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c9815afe0 0x7f3c9806d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:33.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.765+0000 7f3c97e00640 1 -- 172.21.15.52:0/644444383 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c800094d0 con 0x7f3c980cdc00 2024-03-12T17:06:33.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.765+0000 7f3c97e00640 1 --2- 172.21.15.52:0/644444383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3c980cdc00 0x7f3c98155940 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3c8800d830 tx=0x7f3c8800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:33.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.765+0000 7f3c94c00640 1 -- 172.21.15.52:0/644444383 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c880096c0 con 0x7f3c980cdc00 2024-03-12T17:06:33.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.765+0000 7f3c94c00640 1 -- 172.21.15.52:0/644444383 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c88009840 con 0x7f3c980cdc00 2024-03-12T17:06:33.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.765+0000 7f3c94c00640 1 -- 172.21.15.52:0/644444383 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c88009bf0 con 0x7f3c980cdc00 2024-03-12T17:06:33.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.765+0000 7f3c9d800640 1 -- 172.21.15.52:0/644444383 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c9806cc30 con 0x7f3c980cdc00 2024-03-12T17:06:33.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.766+0000 7f3c9d800640 1 -- 172.21.15.52:0/644444383 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c9806d1b0 con 0x7f3c980cdc00 2024-03-12T17:06:33.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.767+0000 7f3c9d800640 1 -- 172.21.15.52:0/644444383 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c9804e030 con 0x7f3c980cdc00 2024-03-12T17:06:33.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.768+0000 7f3c94c00640 1 -- 172.21.15.52:0/644444383 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3c88014410 con 0x7f3c980cdc00 2024-03-12T17:06:33.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.769+0000 7f3c94c00640 1 --2- 172.21.15.52:0/644444383 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f3c7406ac00 0x7f3c7406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:33.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.769+0000 7f3c94c00640 1 -- 172.21.15.52:0/644444383 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3c88089ff0 con 0x7f3c980cdc00 2024-03-12T17:06:33.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.769+0000 7f3c8fe00640 1 --2- 172.21.15.52:0/644444383 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f3c7406ac00 0x7f3c7406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:33.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.770+0000 7f3c8fe00640 1 --2- 172.21.15.52:0/644444383 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f3c7406ac00 0x7f3c7406d0c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3c80036000 tx=0x7f3c8003c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:33.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:33.774+0000 7f3c94c00640 1 -- 172.21.15.52:0/644444383 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f3c88054ee0 con 0x7f3c980cdc00 2024-03-12T17:06:34.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:34.018+0000 7f3c9d800640 1 -- 172.21.15.52:0/644444383 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3c980ca520 con 0x7f3c980cdc00 2024-03-12T17:06:34.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:34.019+0000 7f3c94c00640 1 -- 172.21.15.52:0/644444383 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f3c88018070 con 0x7f3c980cdc00 2024-03-12T17:06:34.012 INFO:teuthology.orchestra.run.smithi052.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-12T17:06:34.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:34.022+0000 7f3c9d800640 1 -- 172.21.15.52:0/644444383 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f3c7406ac00 msgr2=0x7f3c7406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:34.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:34.022+0000 7f3c9d800640 1 --2- 172.21.15.52:0/644444383 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f3c7406ac00 0x7f3c7406d0c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3c80036000 tx=0x7f3c8003c040 comp rx=0 tx=0).stop 2024-03-12T17:06:34.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:34.022+0000 7f3c9d800640 1 -- 172.21.15.52:0/644444383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3c980cdc00 msgr2=0x7f3c98155940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:34.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:34.022+0000 7f3c9d800640 1 --2- 172.21.15.52:0/644444383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3c980cdc00 0x7f3c98155940 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3c8800d830 tx=0x7f3c8800dcf0 comp rx=0 tx=0).stop 2024-03-12T17:06:34.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:34.022+0000 7f3c9d800640 1 -- 172.21.15.52:0/644444383 shutdown_connections 2024-03-12T17:06:34.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:34.022+0000 7f3c9d800640 1 --2- 172.21.15.52:0/644444383 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f3c7406ac00 0x7f3c7406d0c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:34.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:34.022+0000 7f3c9d800640 1 --2- 172.21.15.52:0/644444383 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c981571d0 0x7f3c980ac260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:34.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:34.022+0000 7f3c9d800640 1 --2- 172.21.15.52:0/644444383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c9815afe0 0x7f3c9806d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:34.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:34.022+0000 7f3c9d800640 1 --2- 172.21.15.52:0/644444383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3c980cdc00 0x7f3c98155940 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:34.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:34.022+0000 7f3c9d800640 1 -- 172.21.15.52:0/644444383 >> 172.21.15.52:0/644444383 conn(0x7f3c98065df0 msgr2=0x7f3c98069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:34.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:34.022+0000 7f3c9d800640 1 -- 172.21.15.52:0/644444383 shutdown_connections 2024-03-12T17:06:34.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:34.022+0000 7f3c9d800640 1 -- 172.21.15.52:0/644444383 wait complete. 2024-03-12T17:06:34.035 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-12T17:06:34.035 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-12T17:06:34.035 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi052.front.sepia.ceph.com 2024-03-12T17:06:34.035 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-03-12T17:06:34.035 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-12T17:06:34.035 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-12T17:06:34.035 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-12T17:06:34.035 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-12T17:06:34.035 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-03-12T17:06:34.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:06:34.036 DEBUG:teuthology.orchestra.run.smithi052:> ip addr 2024-03-12T17:06:34.087 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-12T17:06:34.087 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-12T17:06:34.087 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-03-12T17:06:34.087 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:06:34.087 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-03-12T17:06:34.087 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:06:34.087 INFO:teuthology.orchestra.run.smithi052.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-12T17:06:34.087 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d2 brd ff:ff:ff:ff:ff:ff 2024-03-12T17:06:34.087 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f0 2024-03-12T17:06:34.087 INFO:teuthology.orchestra.run.smithi052.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-12T17:06:34.087 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d3 brd ff:ff:ff:ff:ff:ff 2024-03-12T17:06:34.087 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f1 2024-03-12T17:06:34.088 INFO:teuthology.orchestra.run.smithi052.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-12T17:06:34.088 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:de brd ff:ff:ff:ff:ff:ff 2024-03-12T17:06:34.088 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f0 2024-03-12T17:06:34.088 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.21.15.52/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-12T17:06:34.088 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft 42600sec preferred_lft 42600sec 2024-03-12T17:06:34.088 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::ec4:7aff:febd:15de/64 scope link noprefixroute 2024-03-12T17:06:34.088 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:06:34.088 INFO:teuthology.orchestra.run.smithi052.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-12T17:06:34.088 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:df brd ff:ff:ff:ff:ff:ff 2024-03-12T17:06:34.088 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f1 2024-03-12T17:06:34.089 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-12T17:06:34.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:06:34.090 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec bash -c ' 2024-03-12T17:06:34.090 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-03-12T17:06:34.090 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link add name ceph-brx type bridge 2024-03-12T17:06:34.090 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip addr flush dev ceph-brx 2024-03-12T17:06:34.090 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set ceph-brx up 2024-03-12T17:06:34.090 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-12T17:06:34.090 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-03-12T17:06:34.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:06:34.554 DEBUG:teuthology.orchestra.run.smithi052:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-12T17:06:34.636 INFO:teuthology.orchestra.run.smithi052.stdout:1 2024-03-12T17:06:34.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:06:34.640 DEBUG:teuthology.orchestra.run.smithi052:> route 2024-03-12T17:06:34.699 INFO:teuthology.orchestra.run.smithi052.stdout:Kernel IP routing table 2024-03-12T17:06:34.700 INFO:teuthology.orchestra.run.smithi052.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-12T17:06:34.700 INFO:teuthology.orchestra.run.smithi052.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-12T17:06:34.700 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-12T17:06:34.700 INFO:teuthology.orchestra.run.smithi052.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-12T17:06:34.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:06:34.701 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec bash -c ' 2024-03-12T17:06:34.701 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-03-12T17:06:34.701 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-12T17:06:34.701 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-12T17:06:34.702 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-12T17:06:34.702 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-03-12T17:06:35.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:06:35.415 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-03-12T17:06:35.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:06:35.474 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list-id 2024-03-12T17:06:35.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:06:35.532 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec bash -c ' 2024-03-12T17:06:35.532 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-03-12T17:06:35.532 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-12T17:06:35.532 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-03-12T17:06:35.532 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-03-12T17:06:35.683 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-03-12T17:06:35.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:06:35.683 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec bash -c ' 2024-03-12T17:06:35.683 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-03-12T17:06:35.683 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-03-12T17:06:35.684 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-12T17:06:35.684 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-03-12T17:06:35.684 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-03-12T17:06:35.684 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-03-12T17:06:35.684 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-03-12T17:06:35.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:06:35.972 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec bash -c ' 2024-03-12T17:06:35.972 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-03-12T17:06:35.972 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set brx.0 up 2024-03-12T17:06:35.972 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set dev brx.0 master ceph-brx 2024-03-12T17:06:35.972 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-03-12T17:06:36.145 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-03-12T17:06:36.146 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-12T17:06:36.146 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-03-12T17:06:36.198 INFO:teuthology.orchestra.run.smithi052.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-03-12T17:06:36.199 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-12T17:06:36.199 DEBUG:teuthology.orchestra.run.smithi052:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-03-12T17:06:36.250 DEBUG:teuthology.orchestra.run.smithi052:> sudo modprobe fuse 2024-03-12T17:06:36.347 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-12T17:06:36.347 DEBUG:teuthology.orchestra.run.smithi052:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-12T17:06:36.441 INFO:teuthology.orchestra.run.smithi052.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-12T17:06:36.446 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-12T17:06:36.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:06:36.447 DEBUG:teuthology.orchestra.run.smithi052:> ls /sys/fs/fuse/connections 2024-03-12T17:06:36.501 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-12T17:06:36.502 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-03-12T17:06:36.506 DEBUG:teuthology.orchestra.run.smithi052:> sudo modprobe fuse 2024-03-12T17:06:36.580 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-12T17:06:36.581 DEBUG:teuthology.orchestra.run.smithi052:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-12T17:06:36.649 INFO:teuthology.orchestra.run.smithi052.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-12T17:06:36.652 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-12T17:06:36.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:06:36.652 DEBUG:teuthology.orchestra.run.smithi052:> ls /sys/fs/fuse/connections 2024-03-12T17:06:36.691 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi052.stderr:2024-03-12T17:06:36.698+0000 7fe65ac45480 -1 init, newargv = 0x55f2b74ac440 newargc=15 2024-03-12T17:06:36.691 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi052.stderr:ceph-fuse[42396]: starting ceph client 2024-03-12T17:06:36.705 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi052.stderr:ceph-fuse[42396]: starting fuse 2024-03-12T17:06:37.796 DEBUG:teuthology.orchestra.run.smithi052:> sudo modprobe fuse 2024-03-12T17:06:37.905 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-12T17:06:37.906 DEBUG:teuthology.orchestra.run.smithi052:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-12T17:06:37.999 INFO:teuthology.orchestra.run.smithi052.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-12T17:06:38.004 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-12T17:06:38.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:06:38.004 DEBUG:teuthology.orchestra.run.smithi052:> ls /sys/fs/fuse/connections 2024-03-12T17:06:38.059 INFO:teuthology.orchestra.run.smithi052.stdout:40 2024-03-12T17:06:38.059 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2024-03-12T17:06:38.059 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-12T17:06:38.059 DEBUG:teuthology.orchestra.run.smithi052:> import glob 2024-03-12T17:06:38.059 DEBUG:teuthology.orchestra.run.smithi052:> import re 2024-03-12T17:06:38.060 DEBUG:teuthology.orchestra.run.smithi052:> import os 2024-03-12T17:06:38.060 DEBUG:teuthology.orchestra.run.smithi052:> import subprocess 2024-03-12T17:06:38.060 DEBUG:teuthology.orchestra.run.smithi052:> 2024-03-12T17:06:38.060 DEBUG:teuthology.orchestra.run.smithi052:> def _find_admin_socket(client_name): 2024-03-12T17:06:38.060 DEBUG:teuthology.orchestra.run.smithi052:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-03-12T17:06:38.060 DEBUG:teuthology.orchestra.run.smithi052:> files = glob.glob(asok_path) 2024-03-12T17:06:38.060 DEBUG:teuthology.orchestra.run.smithi052:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-03-12T17:06:38.060 DEBUG:teuthology.orchestra.run.smithi052:> 2024-03-12T17:06:38.060 DEBUG:teuthology.orchestra.run.smithi052:> # Given a non-glob path, it better be there 2024-03-12T17:06:38.060 DEBUG:teuthology.orchestra.run.smithi052:> if "*" not in asok_path: 2024-03-12T17:06:38.060 DEBUG:teuthology.orchestra.run.smithi052:> assert(len(files) == 1) 2024-03-12T17:06:38.061 DEBUG:teuthology.orchestra.run.smithi052:> return files[0] 2024-03-12T17:06:38.061 DEBUG:teuthology.orchestra.run.smithi052:> 2024-03-12T17:06:38.061 DEBUG:teuthology.orchestra.run.smithi052:> for f in files: 2024-03-12T17:06:38.061 DEBUG:teuthology.orchestra.run.smithi052:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-12T17:06:38.061 DEBUG:teuthology.orchestra.run.smithi052:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-12T17:06:38.061 DEBUG:teuthology.orchestra.run.smithi052:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-12T17:06:38.061 DEBUG:teuthology.orchestra.run.smithi052:> contents = proc_f.read() 2024-03-12T17:06:38.061 DEBUG:teuthology.orchestra.run.smithi052:> if mountpoint in contents: 2024-03-12T17:06:38.061 DEBUG:teuthology.orchestra.run.smithi052:> return f 2024-03-12T17:06:38.061 DEBUG:teuthology.orchestra.run.smithi052:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-12T17:06:38.061 DEBUG:teuthology.orchestra.run.smithi052:> 2024-03-12T17:06:38.061 DEBUG:teuthology.orchestra.run.smithi052:> print(_find_admin_socket("client.admin")) 2024-03-12T17:06:38.061 DEBUG:teuthology.orchestra.run.smithi052:> ' 2024-03-12T17:06:38.243 INFO:teuthology.orchestra.run.smithi052.stdout:/var/run/ceph/ceph-client.admin.42396.asok 2024-03-12T17:06:38.423 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.42396.asok 2024-03-12T17:06:38.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:06:38.423 DEBUG:teuthology.orchestra.run.smithi052:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.42396.asok status 2024-03-12T17:06:38.671 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-03-12T17:06:38.672 INFO:teuthology.orchestra.run.smithi052.stdout: "metadata": { 2024-03-12T17:06:38.672 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_sha1": "49e4a6794d5502bd66908faf714fc54896c51712", 2024-03-12T17:06:38.672 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_version": "ceph version 18.2.2-797-g49e4a679 (49e4a6794d5502bd66908faf714fc54896c51712) reef (stable)", 2024-03-12T17:06:38.672 INFO:teuthology.orchestra.run.smithi052.stdout: "entity_id": "admin", 2024-03-12T17:06:38.672 INFO:teuthology.orchestra.run.smithi052.stdout: "hostname": "smithi052", 2024-03-12T17:06:38.672 INFO:teuthology.orchestra.run.smithi052.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-03-12T17:06:38.672 INFO:teuthology.orchestra.run.smithi052.stdout: "pid": "42396", 2024-03-12T17:06:38.672 INFO:teuthology.orchestra.run.smithi052.stdout: "root": "/" 2024-03-12T17:06:38.672 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-12T17:06:38.672 INFO:teuthology.orchestra.run.smithi052.stdout: "dentry_count": 0, 2024-03-12T17:06:38.672 INFO:teuthology.orchestra.run.smithi052.stdout: "dentry_pinned_count": 0, 2024-03-12T17:06:38.672 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 4431, 2024-03-12T17:06:38.673 INFO:teuthology.orchestra.run.smithi052.stdout: "inst": { 2024-03-12T17:06:38.673 INFO:teuthology.orchestra.run.smithi052.stdout: "name": { 2024-03-12T17:06:38.673 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "client", 2024-03-12T17:06:38.673 INFO:teuthology.orchestra.run.smithi052.stdout: "num": 4431 2024-03-12T17:06:38.673 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-12T17:06:38.673 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": { 2024-03-12T17:06:38.673 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v1", 2024-03-12T17:06:38.673 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "192.168.0.1:0", 2024-03-12T17:06:38.673 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 1226487595 2024-03-12T17:06:38.673 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-12T17:06:38.673 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-12T17:06:38.673 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": { 2024-03-12T17:06:38.673 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v1", 2024-03-12T17:06:38.673 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "192.168.0.1:0", 2024-03-12T17:06:38.673 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 1226487595 2024-03-12T17:06:38.674 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-12T17:06:38.674 INFO:teuthology.orchestra.run.smithi052.stdout: "inst_str": "client.4431 192.168.0.1:0/1226487595", 2024-03-12T17:06:38.674 INFO:teuthology.orchestra.run.smithi052.stdout: "addr_str": "192.168.0.1:0/1226487595", 2024-03-12T17:06:38.674 INFO:teuthology.orchestra.run.smithi052.stdout: "inode_count": 1, 2024-03-12T17:06:38.674 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_epoch": 7, 2024-03-12T17:06:38.674 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_epoch": 25, 2024-03-12T17:06:38.674 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_epoch_barrier": 0, 2024-03-12T17:06:38.674 INFO:teuthology.orchestra.run.smithi052.stdout: "blocklisted": false, 2024-03-12T17:06:38.674 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs" 2024-03-12T17:06:38.674 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-03-12T17:06:38.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:06:38.687 DEBUG:teuthology.orchestra.run.smithi052:> stat --file-system '--printf=%T 2024-03-12T17:06:38.687 DEBUG:teuthology.orchestra.run.smithi052:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-03-12T17:06:38.747 INFO:teuthology.orchestra.run.smithi052.stdout:fuseblk 2024-03-12T17:06:38.747 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-03-12T17:06:38.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:06:38.747 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-03-12T17:06:38.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:06:38.856 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-03-12T17:06:38.955 INFO:teuthology.orchestra.run.smithi052.stdout:# file: . 2024-03-12T17:06:38.955 INFO:teuthology.orchestra.run.smithi052.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-03-12T17:06:38.955 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:06:38.955 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:06:38.956 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-12T17:06:39.013 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:06:39.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.241+0000 7fa383200640 1 Processor -- start 2024-03-12T17:06:39.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.241+0000 7fa383200640 1 -- start start 2024-03-12T17:06:39.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.242+0000 7fa383200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa3840cf5c0 0x7fa3840b8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:39.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.242+0000 7fa383200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa3840d0c20 0x7fa3840d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:39.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.242+0000 7fa383200640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa38415ce90 con 0x7fa38415cab0 2024-03-12T17:06:39.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.242+0000 7fa383200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3840cddf0 con 0x7fa3840d0c20 2024-03-12T17:06:39.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.242+0000 7fa383200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3840cd570 con 0x7fa3840cf5c0 2024-03-12T17:06:39.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.243+0000 7fa381e00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fa38415cab0 0x7fa3840ceeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:40972/0 (socket says 172.21.15.52:40972) 2024-03-12T17:06:39.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.243+0000 7fa381e00640 1 -- 172.21.15.52:0/1594054598 learned_addr learned my addr 172.21.15.52:0/1594054598 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:06:39.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.243+0000 7fa381400640 1 --2- 172.21.15.52:0/1594054598 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa3840cf5c0 0x7fa3840b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:39.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.243+0000 7fa382800640 1 --2- 172.21.15.52:0/1594054598 >> v2:172.21.15.102:3300/0 conn(0x7fa3840d0c20 0x7fa3840d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:39.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.243+0000 7fa380a00640 1 -- 172.21.15.52:0/1594054598 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 552765283 0 0) 0x7fa38415ce90 con 0x7fa38415cab0 2024-03-12T17:06:39.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.243+0000 7fa380a00640 1 -- 172.21.15.52:0/1594054598 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa368003520 con 0x7fa38415cab0 2024-03-12T17:06:39.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.244+0000 7fa380a00640 1 -- 172.21.15.52:0/1594054598 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 87044727 0 0) 0x7fa368003520 con 0x7fa38415cab0 2024-03-12T17:06:39.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.244+0000 7fa380a00640 1 -- 172.21.15.52:0/1594054598 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa3840cf5c0 msgr2=0x7fa3840b8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:39.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.244+0000 7fa380a00640 1 --2- 172.21.15.52:0/1594054598 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa3840cf5c0 0x7fa3840b8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:39.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.244+0000 7fa380a00640 1 -- 172.21.15.52:0/1594054598 >> v2:172.21.15.102:3300/0 conn(0x7fa3840d0c20 msgr2=0x7fa3840d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:39.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.244+0000 7fa380a00640 1 --2- 172.21.15.52:0/1594054598 >> v2:172.21.15.102:3300/0 conn(0x7fa3840d0c20 0x7fa3840d10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:39.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.244+0000 7fa380a00640 1 -- 172.21.15.52:0/1594054598 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa384161370 con 0x7fa38415cab0 2024-03-12T17:06:39.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.244+0000 7fa380a00640 1 -- 172.21.15.52:0/1594054598 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7fa374001670 con 0x7fa38415cab0 2024-03-12T17:06:39.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.244+0000 7fa380a00640 1 -- 172.21.15.52:0/1594054598 >> v1:172.21.15.52:6789/0 conn(0x7fa38415cab0 legacy=0x7fa3840ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:39.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.244+0000 7fa380a00640 1 --2- 172.21.15.52:0/1594054598 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa3840cf5c0 0x7fa368003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:39.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.244+0000 7fa380a00640 1 --2- 172.21.15.52:0/1594054598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa368003ef0 0x7fa3680047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:39.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.244+0000 7fa380a00640 1 --2- 172.21.15.52:0/1594054598 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa368004d00 0x7fa368005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:39.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.244+0000 7fa380a00640 1 -- 172.21.15.52:0/1594054598 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3840cddf0 con 0x7fa368004d00 2024-03-12T17:06:39.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.244+0000 7fa380a00640 1 -- 172.21.15.52:0/1594054598 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3840cd570 con 0x7fa368003ef0 2024-03-12T17:06:39.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.244+0000 7fa380a00640 1 -- 172.21.15.52:0/1594054598 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa36800b980 con 0x7fa3840cf5c0 2024-03-12T17:06:39.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.244+0000 7fa382800640 1 --2- 172.21.15.52:0/1594054598 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa368004d00 0x7fa368005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:39.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.244+0000 7fa381400640 1 --2- 172.21.15.52:0/1594054598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa368003ef0 0x7fa3680047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:39.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.244+0000 7fa381e00640 1 --2- 172.21.15.52:0/1594054598 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa3840cf5c0 0x7fa368003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:39.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.245+0000 7fa382800640 1 -- 172.21.15.52:0/1594054598 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa3840cf5c0 msgr2=0x7fa368003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:39.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.245+0000 7fa382800640 1 --2- 172.21.15.52:0/1594054598 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa3840cf5c0 0x7fa368003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:39.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.245+0000 7fa382800640 1 -- 172.21.15.52:0/1594054598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa368003ef0 msgr2=0x7fa3680047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:39.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.245+0000 7fa382800640 1 --2- 172.21.15.52:0/1594054598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa368003ef0 0x7fa3680047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:39.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.245+0000 7fa382800640 1 -- 172.21.15.52:0/1594054598 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa368003520 con 0x7fa368004d00 2024-03-12T17:06:39.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.245+0000 7fa381400640 1 --2- 172.21.15.52:0/1594054598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa368003ef0 0x7fa3680047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:06:39.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.246+0000 7fa382800640 1 --2- 172.21.15.52:0/1594054598 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa368004d00 0x7fa368005180 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa37c009fc0 tx=0x7fa3680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c16cef3caa7d32 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:39.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.246+0000 7fa380a00640 1 -- 172.21.15.52:0/1594054598 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37c02cc40 con 0x7fa368004d00 2024-03-12T17:06:39.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.246+0000 7fa380a00640 1 -- 172.21.15.52:0/1594054598 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa37c00f040 con 0x7fa368004d00 2024-03-12T17:06:39.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.246+0000 7fa380a00640 1 -- 172.21.15.52:0/1594054598 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37c034440 con 0x7fa368004d00 2024-03-12T17:06:39.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.247+0000 7fa383200640 1 -- 172.21.15.52:0/1594054598 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa368004d00 msgr2=0x7fa368005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:39.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.247+0000 7fa383200640 1 --2- 172.21.15.52:0/1594054598 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa368004d00 0x7fa368005180 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa37c009fc0 tx=0x7fa3680057f0 comp rx=0 tx=0).stop 2024-03-12T17:06:39.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.247+0000 7fa382800640 1 -- 172.21.15.52:0/1594054598 reap_dead start 2024-03-12T17:06:39.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.247+0000 7fa383200640 1 -- 172.21.15.52:0/1594054598 shutdown_connections 2024-03-12T17:06:39.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.247+0000 7fa383200640 1 -- 172.21.15.52:0/1594054598 >> 172.21.15.52:0/1594054598 conn(0x7fa38406ae90 msgr2=0x7fa38406b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:39.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.247+0000 7fa383200640 1 -- 172.21.15.52:0/1594054598 shutdown_connections 2024-03-12T17:06:39.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.247+0000 7fa383200640 1 -- 172.21.15.52:0/1594054598 wait complete. 2024-03-12T17:06:39.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.249+0000 7fa383200640 1 Processor -- start 2024-03-12T17:06:39.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.249+0000 7fa383200640 1 -- start start 2024-03-12T17:06:39.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.250+0000 7fa383200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3840cb6b0 0x7fa38415eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:39.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.250+0000 7fa383200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3840cba90 0x7fa3840c6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:39.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.250+0000 7fa383200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa384154fc0 0x7fa384156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:39.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.250+0000 7fa383200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38415f940 con 0x7fa3840cba90 2024-03-12T17:06:39.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.250+0000 7fa383200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38415f380 con 0x7fa3840cb6b0 2024-03-12T17:06:39.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.250+0000 7fa383200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa38415f660 con 0x7fa384154fc0 2024-03-12T17:06:39.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.251+0000 7fa381400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3840cba90 0x7fa3840c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:39.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.251+0000 7fa381400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3840cba90 0x7fa3840c6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46610/0 (socket says 172.21.15.52:46610) 2024-03-12T17:06:39.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.251+0000 7fa381400640 1 -- 172.21.15.52:0/1303637519 learned_addr learned my addr 172.21.15.52:0/1303637519 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:39.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.251+0000 7fa382800640 1 --2- 172.21.15.52:0/1303637519 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa384154fc0 0x7fa384156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:39.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.251+0000 7fa381e00640 1 --2- 172.21.15.52:0/1303637519 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3840cb6b0 0x7fa38415eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:39.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.251+0000 7fa381400640 1 -- 172.21.15.52:0/1303637519 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa384154fc0 msgr2=0x7fa384156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:39.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.251+0000 7fa381400640 1 --2- 172.21.15.52:0/1303637519 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa384154fc0 0x7fa384156620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:39.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.251+0000 7fa381400640 1 -- 172.21.15.52:0/1303637519 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3840cb6b0 msgr2=0x7fa38415eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:39.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.251+0000 7fa381400640 1 --2- 172.21.15.52:0/1303637519 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3840cb6b0 0x7fa38415eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:39.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.251+0000 7fa381400640 1 -- 172.21.15.52:0/1303637519 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3840c9910 con 0x7fa3840cba90 2024-03-12T17:06:39.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.252+0000 7fa381400640 1 --2- 172.21.15.52:0/1303637519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3840cba90 0x7fa3840c6850 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa370011540 tx=0x7fa370011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:39.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.252+0000 7fa37a000640 1 -- 172.21.15.52:0/1303637519 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa370015870 con 0x7fa3840cba90 2024-03-12T17:06:39.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.252+0000 7fa37a000640 1 -- 172.21.15.52:0/1303637519 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3700159d0 con 0x7fa3840cba90 2024-03-12T17:06:39.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.252+0000 7fa37a000640 1 -- 172.21.15.52:0/1303637519 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa370015d40 con 0x7fa3840cba90 2024-03-12T17:06:39.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.252+0000 7fa383200640 1 -- 172.21.15.52:0/1303637519 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3840c9bf0 con 0x7fa3840cba90 2024-03-12T17:06:39.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.252+0000 7fa383200640 1 -- 172.21.15.52:0/1303637519 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3840c6d90 con 0x7fa3840cba90 2024-03-12T17:06:39.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.254+0000 7fa383200640 1 -- 172.21.15.52:0/1303637519 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3840c9750 con 0x7fa3840cba90 2024-03-12T17:06:39.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.255+0000 7fa37a000640 1 -- 172.21.15.52:0/1303637519 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa37000f460 con 0x7fa3840cba90 2024-03-12T17:06:39.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.255+0000 7fa37a000640 1 --2- 172.21.15.52:0/1303637519 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa36406ac00 0x7fa36406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:39.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.256+0000 7fa37a000640 1 -- 172.21.15.52:0/1303637519 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa370089f20 con 0x7fa3840cba90 2024-03-12T17:06:39.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.256+0000 7fa381e00640 1 --2- 172.21.15.52:0/1303637519 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa36406ac00 0x7fa36406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:39.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.257+0000 7fa381e00640 1 --2- 172.21.15.52:0/1303637519 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa36406ac00 0x7fa36406d0c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa374001eb0 tx=0x7fa374009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:39.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.261+0000 7fa37a000640 1 -- 172.21.15.52:0/1303637519 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fa370054d70 con 0x7fa3840cba90 2024-03-12T17:06:39.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.459+0000 7fa383200640 1 -- 172.21.15.52:0/1303637519 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3840c9d80 con 0x7fa3840cba90 2024-03-12T17:06:39.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.462+0000 7fa37a000640 1 -- 172.21.15.52:0/1303637519 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7fa370058390 con 0x7fa3840cba90 2024-03-12T17:06:39.456 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:06:39.456 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":25,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:06:30.468476+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T17:06:19.218543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T17:06:23.525474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T17:06:24.911574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-12T17:06:28.017172+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.733031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.775178+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.591522+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.820838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.829252+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.582481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:06:39.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.465+0000 7fa383200640 1 -- 172.21.15.52:0/1303637519 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa36406ac00 msgr2=0x7fa36406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:39.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.465+0000 7fa383200640 1 --2- 172.21.15.52:0/1303637519 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa36406ac00 0x7fa36406d0c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa374001eb0 tx=0x7fa374009040 comp rx=0 tx=0).stop 2024-03-12T17:06:39.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.465+0000 7fa383200640 1 -- 172.21.15.52:0/1303637519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3840cba90 msgr2=0x7fa3840c6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:39.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.465+0000 7fa383200640 1 --2- 172.21.15.52:0/1303637519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3840cba90 0x7fa3840c6850 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa370011540 tx=0x7fa370011a00 comp rx=0 tx=0).stop 2024-03-12T17:06:39.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.465+0000 7fa383200640 1 -- 172.21.15.52:0/1303637519 shutdown_connections 2024-03-12T17:06:39.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.465+0000 7fa383200640 1 --2- 172.21.15.52:0/1303637519 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa36406ac00 0x7fa36406d0c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:39.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.465+0000 7fa383200640 1 --2- 172.21.15.52:0/1303637519 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa384154fc0 0x7fa384156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:39.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.465+0000 7fa383200640 1 --2- 172.21.15.52:0/1303637519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3840cba90 0x7fa3840c6850 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:39.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.465+0000 7fa383200640 1 --2- 172.21.15.52:0/1303637519 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3840cb6b0 0x7fa38415eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:39.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.465+0000 7fa383200640 1 -- 172.21.15.52:0/1303637519 >> 172.21.15.52:0/1303637519 conn(0x7fa38406ae90 msgr2=0x7fa384160470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:39.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.466+0000 7fa383200640 1 -- 172.21.15.52:0/1303637519 shutdown_connections 2024-03-12T17:06:39.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.466+0000 7fa383200640 1 -- 172.21.15.52:0/1303637519 wait complete. 2024-03-12T17:06:39.476 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T17:06:19.218543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-12T17:06:23.525474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-12T17:06:24.911574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-12T17:06:28.017172+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-12T17:06:39.477 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T17:06:39.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.709+0000 7f97b8a00640 1 Processor -- start 2024-03-12T17:06:39.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.709+0000 7f97b8a00640 1 -- start start 2024-03-12T17:06:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.711+0000 7f97b8a00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f97b415c820 0x7f97b40cd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.711+0000 7f97b8a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f97b40cf7c0 0x7f97b40d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.711+0000 7f97b8a00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b40ccd90 con 0x7f97b4160e90 2024-03-12T17:06:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.711+0000 7f97b8a00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97b40cdd70 con 0x7f97b415c820 2024-03-12T17:06:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.711+0000 7f97b8a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97b40b7fc0 con 0x7f97b40cf7c0 2024-03-12T17:06:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.711+0000 7f97b3400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f97b415c820 0x7f97b40cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.711+0000 7f97b3400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f97b415c820 0x7f97b40cd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46614/0 (socket says 172.21.15.52:46614) 2024-03-12T17:06:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.711+0000 7f97b3400640 1 -- 172.21.15.52:0/1014779941 learned_addr learned my addr 172.21.15.52:0/1014779941 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.711+0000 7f97abe00640 1 --1- 172.21.15.52:0/1014779941 >> v1:172.21.15.52:6789/0 conn(0x7f97b4160e90 0x7f97b40cf0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:40982/0 (socket says 172.21.15.52:40982) 2024-03-12T17:06:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.711+0000 7f97b3e00640 1 --2- 172.21.15.52:0/1014779941 >> v2:172.21.15.102:3300/0 conn(0x7f97b40cf7c0 0x7f97b40d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:39.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.711+0000 7f97b3400640 1 -- 172.21.15.52:0/1014779941 >> v2:172.21.15.102:3300/0 conn(0x7f97b40cf7c0 msgr2=0x7f97b40d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:39.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.712+0000 7f97b3400640 1 --2- 172.21.15.52:0/1014779941 >> v2:172.21.15.102:3300/0 conn(0x7f97b40cf7c0 0x7f97b40d0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:39.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.712+0000 7f97b3400640 1 -- 172.21.15.52:0/1014779941 >> v1:172.21.15.52:6789/0 conn(0x7f97b4160e90 legacy=0x7f97b40cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:39.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.712+0000 7f97b3400640 1 -- 172.21.15.52:0/1014779941 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b40b82a0 con 0x7f97b415c820 2024-03-12T17:06:39.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.712+0000 7f97b2a00640 1 -- 172.21.15.52:0/1014779941 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 486845059 0 0) 0x7f97b40ccd90 con 0x7f97b4160e90 2024-03-12T17:06:39.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.712+0000 7f97b3e00640 1 --2- 172.21.15.52:0/1014779941 >> v2:172.21.15.102:3300/0 conn(0x7f97b40cf7c0 0x7f97b40d0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:39.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.713+0000 7f97b3400640 1 --2- 172.21.15.52:0/1014779941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97b415c820 0x7f97b40cd830 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f97b408b310 tx=0x7f97a402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517aafb0d55fb657 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:39.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.713+0000 7f97b2a00640 1 -- 172.21.15.52:0/1014779941 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a402f850 con 0x7f97b415c820 2024-03-12T17:06:39.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.713+0000 7f97b2a00640 1 -- 172.21.15.52:0/1014779941 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a402f9b0 con 0x7f97b415c820 2024-03-12T17:06:39.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.713+0000 7f97b2a00640 1 -- 172.21.15.52:0/1014779941 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a402fd40 con 0x7f97b415c820 2024-03-12T17:06:39.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.714+0000 7f97b8a00640 1 -- 172.21.15.52:0/1014779941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97b415c820 msgr2=0x7f97b40cd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:39.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.714+0000 7f97b8a00640 1 --2- 172.21.15.52:0/1014779941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97b415c820 0x7f97b40cd830 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f97b408b310 tx=0x7f97a402ba30 comp rx=0 tx=0).stop 2024-03-12T17:06:39.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.714+0000 7f97b8a00640 1 -- 172.21.15.52:0/1014779941 shutdown_connections 2024-03-12T17:06:39.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.714+0000 7f97b8a00640 1 --2- 172.21.15.52:0/1014779941 >> v2:172.21.15.102:3300/0 conn(0x7f97b40cf7c0 0x7f97b40d0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:39.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.714+0000 7f97b8a00640 1 --2- 172.21.15.52:0/1014779941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97b415c820 0x7f97b40cd830 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:39.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.714+0000 7f97b8a00640 1 -- 172.21.15.52:0/1014779941 >> 172.21.15.52:0/1014779941 conn(0x7f97b406b050 msgr2=0x7f97b406b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:39.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.714+0000 7f97b8a00640 1 -- 172.21.15.52:0/1014779941 shutdown_connections 2024-03-12T17:06:39.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.714+0000 7f97b8a00640 1 -- 172.21.15.52:0/1014779941 wait complete. 2024-03-12T17:06:39.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.716+0000 7f97b8a00640 1 Processor -- start 2024-03-12T17:06:39.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.716+0000 7f97b8a00640 1 -- start start 2024-03-12T17:06:39.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.717+0000 7f97b8a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f97b40cf7c0 0x7f97b41f2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:39.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.717+0000 7f97b8a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97b415c820 0x7f97b41f1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:39.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.717+0000 7f97b8a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97b4160e90 0x7f97b40c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:39.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.717+0000 7f97b8a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b40b4060 con 0x7f97b4160e90 2024-03-12T17:06:39.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.717+0000 7f97b8a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b41f3140 con 0x7f97b415c820 2024-03-12T17:06:39.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.717+0000 7f97b8a00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b40c7310 con 0x7f97b40cf7c0 2024-03-12T17:06:39.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.717+0000 7f97b3e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97b4160e90 0x7f97b40c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:39.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.717+0000 7f97b3e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97b4160e90 0x7f97b40c7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46622/0 (socket says 172.21.15.52:46622) 2024-03-12T17:06:39.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.717+0000 7f97b3e00640 1 -- 172.21.15.52:0/1118316044 learned_addr learned my addr 172.21.15.52:0/1118316044 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:39.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.717+0000 7f97b3400640 1 --2- 172.21.15.52:0/1118316044 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f97b40cf7c0 0x7f97b41f2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:39.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.717+0000 7f97abe00640 1 --2- 172.21.15.52:0/1118316044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97b415c820 0x7f97b41f1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:39.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.718+0000 7f97b3e00640 1 -- 172.21.15.52:0/1118316044 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f97b40cf7c0 msgr2=0x7f97b41f2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:39.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.718+0000 7f97b3e00640 1 --2- 172.21.15.52:0/1118316044 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f97b40cf7c0 0x7f97b41f2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:39.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.718+0000 7f97b3e00640 1 -- 172.21.15.52:0/1118316044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97b415c820 msgr2=0x7f97b41f1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:39.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.718+0000 7f97b3e00640 1 --2- 172.21.15.52:0/1118316044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97b415c820 0x7f97b41f1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:39.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.718+0000 7f97b3e00640 1 -- 172.21.15.52:0/1118316044 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a40094d0 con 0x7f97b4160e90 2024-03-12T17:06:39.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.718+0000 7f97b3e00640 1 --2- 172.21.15.52:0/1118316044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97b4160e90 0x7f97b40c7ec0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f97ac00cd80 tx=0x7f97ac00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:39.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.718+0000 7f97ab400640 1 -- 172.21.15.52:0/1118316044 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ac00f680 con 0x7f97b4160e90 2024-03-12T17:06:39.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.718+0000 7f97ab400640 1 -- 172.21.15.52:0/1118316044 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ac00f800 con 0x7f97b4160e90 2024-03-12T17:06:39.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.718+0000 7f97ab400640 1 -- 172.21.15.52:0/1118316044 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ac00fbb0 con 0x7f97b4160e90 2024-03-12T17:06:39.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.718+0000 7f97b8a00640 1 -- 172.21.15.52:0/1118316044 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b40c6270 con 0x7f97b4160e90 2024-03-12T17:06:39.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.718+0000 7f97b8a00640 1 -- 172.21.15.52:0/1118316044 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b40c6830 con 0x7f97b4160e90 2024-03-12T17:06:39.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.720+0000 7f97b8a00640 1 -- 172.21.15.52:0/1118316044 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b40d2480 con 0x7f97b4160e90 2024-03-12T17:06:39.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.721+0000 7f97ab400640 1 -- 172.21.15.52:0/1118316044 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f97ac00b040 con 0x7f97b4160e90 2024-03-12T17:06:39.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.722+0000 7f97ab400640 1 --2- 172.21.15.52:0/1118316044 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f979006ac00 0x7f979006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:39.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.722+0000 7f97ab400640 1 -- 172.21.15.52:0/1118316044 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f97ac089f40 con 0x7f97b4160e90 2024-03-12T17:06:39.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.723+0000 7f97b3400640 1 --2- 172.21.15.52:0/1118316044 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f979006ac00 0x7f979006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:39.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.724+0000 7f97b3400640 1 --2- 172.21.15.52:0/1118316044 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f979006ac00 0x7f979006d0c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f97a4036000 tx=0x7f97a4008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:39.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.727+0000 7f97ab400640 1 -- 172.21.15.52:0/1118316044 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f97ac054ec0 con 0x7f97b4160e90 2024-03-12T17:06:39.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.926+0000 7f97b8a00640 1 -- 172.21.15.52:0/1118316044 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f97b404e030 con 0x7f97b4160e90 2024-03-12T17:06:39.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.927+0000 7f97ab400640 1 -- 172.21.15.52:0/1118316044 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f97ac01d020 con 0x7f97b4160e90 2024-03-12T17:06:39.920 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-03-12T17:06:39.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.930+0000 7f97b8a00640 1 -- 172.21.15.52:0/1118316044 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f979006ac00 msgr2=0x7f979006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:39.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.930+0000 7f97b8a00640 1 --2- 172.21.15.52:0/1118316044 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f979006ac00 0x7f979006d0c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f97a4036000 tx=0x7f97a4008030 comp rx=0 tx=0).stop 2024-03-12T17:06:39.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.930+0000 7f97b8a00640 1 -- 172.21.15.52:0/1118316044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97b4160e90 msgr2=0x7f97b40c7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:39.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.930+0000 7f97b8a00640 1 --2- 172.21.15.52:0/1118316044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97b4160e90 0x7f97b40c7ec0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f97ac00cd80 tx=0x7f97ac00a650 comp rx=0 tx=0).stop 2024-03-12T17:06:39.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.930+0000 7f97b8a00640 1 -- 172.21.15.52:0/1118316044 shutdown_connections 2024-03-12T17:06:39.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.930+0000 7f97b8a00640 1 --2- 172.21.15.52:0/1118316044 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f979006ac00 0x7f979006d0c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:39.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.930+0000 7f97b8a00640 1 --2- 172.21.15.52:0/1118316044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97b4160e90 0x7f97b40c7ec0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:39.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.930+0000 7f97b8a00640 1 --2- 172.21.15.52:0/1118316044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97b415c820 0x7f97b41f1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:39.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.930+0000 7f97b8a00640 1 --2- 172.21.15.52:0/1118316044 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f97b40cf7c0 0x7f97b41f2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:39.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.930+0000 7f97b8a00640 1 -- 172.21.15.52:0/1118316044 >> 172.21.15.52:0/1118316044 conn(0x7f97b406b050 msgr2=0x7f97b40cb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:39.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.930+0000 7f97b8a00640 1 -- 172.21.15.52:0/1118316044 shutdown_connections 2024-03-12T17:06:39.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:39.930+0000 7f97b8a00640 1 -- 172.21.15.52:0/1118316044 wait complete. 2024-03-12T17:06:39.943 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-12T17:06:40.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.173+0000 7fb3cd200640 1 Processor -- start 2024-03-12T17:06:40.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.173+0000 7fb3cd200640 1 -- start start 2024-03-12T17:06:40.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.175+0000 7fb3cd200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb3c8143460 0x7fb3c815aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:40.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.175+0000 7fb3cd200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb3c8143930 0x7fb3c80cd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:40.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.175+0000 7fb3cd200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3c80cc6b0 con 0x7fb3c80cdc50 2024-03-12T17:06:40.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.175+0000 7fb3cd200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3c80cc950 con 0x7fb3c8143460 2024-03-12T17:06:40.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.175+0000 7fb3cd200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3c80ccc30 con 0x7fb3c8143930 2024-03-12T17:06:40.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.175+0000 7fb3c7400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb3c8143460 0x7fb3c815aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:40.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.176+0000 7fb3c7400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb3c8143460 0x7fb3c815aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46634/0 (socket says 172.21.15.52:46634) 2024-03-12T17:06:40.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.176+0000 7fb3c7400640 1 -- 172.21.15.52:0/2578916841 learned_addr learned my addr 172.21.15.52:0/2578916841 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:40.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.176+0000 7fb3c7400640 1 -- 172.21.15.52:0/2578916841 >> v2:172.21.15.102:3300/0 conn(0x7fb3c8143930 msgr2=0x7fb3c80cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:40.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.176+0000 7fb3bfe00640 1 --2- 172.21.15.52:0/2578916841 >> v2:172.21.15.102:3300/0 conn(0x7fb3c8143930 0x7fb3c80cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:40.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.176+0000 7fb3c7400640 1 --2- 172.21.15.52:0/2578916841 >> v2:172.21.15.102:3300/0 conn(0x7fb3c8143930 0x7fb3c80cd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:40.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.176+0000 7fb3c7400640 1 -- 172.21.15.52:0/2578916841 >> v1:172.21.15.102:6789/0 conn(0x7fb3c80cdc50 legacy=0x7fb3c815e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:40.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.176+0000 7fb3c7400640 1 -- 172.21.15.52:0/2578916841 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c80ccf10 con 0x7fb3c8143460 2024-03-12T17:06:40.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.176+0000 7fb3bfe00640 1 --2- 172.21.15.52:0/2578916841 >> v2:172.21.15.102:3300/0 conn(0x7fb3c8143930 0x7fb3c80cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:40.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.177+0000 7fb3c7400640 1 --2- 172.21.15.52:0/2578916841 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3c8143460 0x7fb3c815aba0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb3c806ace0 tx=0x7fb3b802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a87e91b9792ac625 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:40.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.178+0000 7fb3c6a00640 1 -- 172.21.15.52:0/2578916841 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8030860 con 0x7fb3c8143460 2024-03-12T17:06:40.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.178+0000 7fb3c6a00640 1 -- 172.21.15.52:0/2578916841 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b80309c0 con 0x7fb3c8143460 2024-03-12T17:06:40.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.178+0000 7fb3c6a00640 1 -- 172.21.15.52:0/2578916841 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8030d50 con 0x7fb3c8143460 2024-03-12T17:06:40.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.178+0000 7fb3cd200640 1 -- 172.21.15.52:0/2578916841 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3c8143460 msgr2=0x7fb3c815aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:40.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.178+0000 7fb3cd200640 1 --2- 172.21.15.52:0/2578916841 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3c8143460 0x7fb3c815aba0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb3c806ace0 tx=0x7fb3b802b9a0 comp rx=0 tx=0).stop 2024-03-12T17:06:40.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.178+0000 7fb3cd200640 1 -- 172.21.15.52:0/2578916841 shutdown_connections 2024-03-12T17:06:40.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.178+0000 7fb3cd200640 1 --2- 172.21.15.52:0/2578916841 >> v2:172.21.15.102:3300/0 conn(0x7fb3c8143930 0x7fb3c80cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:40.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.178+0000 7fb3cd200640 1 --2- 172.21.15.52:0/2578916841 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3c8143460 0x7fb3c815aba0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:40.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.178+0000 7fb3cd200640 1 -- 172.21.15.52:0/2578916841 >> 172.21.15.52:0/2578916841 conn(0x7fb3c8065df0 msgr2=0x7fb3c8069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:40.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.179+0000 7fb3cd200640 1 -- 172.21.15.52:0/2578916841 shutdown_connections 2024-03-12T17:06:40.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.179+0000 7fb3cd200640 1 -- 172.21.15.52:0/2578916841 wait complete. 2024-03-12T17:06:40.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.180+0000 7fb3cd200640 1 Processor -- start 2024-03-12T17:06:40.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.180+0000 7fb3cd200640 1 -- start start 2024-03-12T17:06:40.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.181+0000 7fb3cd200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3c80cdc50 0x7fb3c813dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:40.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.181+0000 7fb3cd200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3c8143460 0x7fb3c815b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:40.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.181+0000 7fb3cd200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb3c8143930 0x7fb3c815b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:40.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.181+0000 7fb3cd200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c813bec0 con 0x7fb3c8143460 2024-03-12T17:06:40.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.181+0000 7fb3cd200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c815d680 con 0x7fb3c80cdc50 2024-03-12T17:06:40.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.181+0000 7fb3cd200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c815c020 con 0x7fb3c8143930 2024-03-12T17:06:40.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.182+0000 7fb3bfe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3c8143460 0x7fb3c815b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:40.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.182+0000 7fb3bfe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3c8143460 0x7fb3c815b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46642/0 (socket says 172.21.15.52:46642) 2024-03-12T17:06:40.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.182+0000 7fb3bfe00640 1 -- 172.21.15.52:0/2151669220 learned_addr learned my addr 172.21.15.52:0/2151669220 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:40.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.182+0000 7fb3c7e00640 1 --2- 172.21.15.52:0/2151669220 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb3c8143930 0x7fb3c815b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:40.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.182+0000 7fb3c7400640 1 --2- 172.21.15.52:0/2151669220 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3c80cdc50 0x7fb3c813dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:40.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.182+0000 7fb3bfe00640 1 -- 172.21.15.52:0/2151669220 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb3c8143930 msgr2=0x7fb3c815b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:40.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.182+0000 7fb3bfe00640 1 --2- 172.21.15.52:0/2151669220 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb3c8143930 0x7fb3c815b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:40.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.182+0000 7fb3bfe00640 1 -- 172.21.15.52:0/2151669220 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3c80cdc50 msgr2=0x7fb3c813dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:40.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.182+0000 7fb3bfe00640 1 --2- 172.21.15.52:0/2151669220 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3c80cdc50 0x7fb3c813dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:40.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.182+0000 7fb3bfe00640 1 -- 172.21.15.52:0/2151669220 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b80094d0 con 0x7fb3c8143460 2024-03-12T17:06:40.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.182+0000 7fb3c7400640 1 --2- 172.21.15.52:0/2151669220 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3c80cdc50 0x7fb3c813dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:40.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.182+0000 7fb3bfe00640 1 --2- 172.21.15.52:0/2151669220 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3c8143460 0x7fb3c815b480 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb3b000c7f0 tx=0x7fb3b000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:40.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.183+0000 7fb3bf400640 1 -- 172.21.15.52:0/2151669220 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0007820 con 0x7fb3c8143460 2024-03-12T17:06:40.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.183+0000 7fb3bf400640 1 -- 172.21.15.52:0/2151669220 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b0007980 con 0x7fb3c8143460 2024-03-12T17:06:40.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.183+0000 7fb3bf400640 1 -- 172.21.15.52:0/2151669220 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0007cf0 con 0x7fb3c8143460 2024-03-12T17:06:40.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.183+0000 7fb3cd200640 1 -- 172.21.15.52:0/2151669220 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3c815c300 con 0x7fb3c8143460 2024-03-12T17:06:40.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.183+0000 7fb3cd200640 1 -- 172.21.15.52:0/2151669220 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3c813bb50 con 0x7fb3c8143460 2024-03-12T17:06:40.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.184+0000 7fb3cd200640 1 -- 172.21.15.52:0/2151669220 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3c813c070 con 0x7fb3c8143460 2024-03-12T17:06:40.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.185+0000 7fb3bf400640 1 -- 172.21.15.52:0/2151669220 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb3b00113f0 con 0x7fb3c8143460 2024-03-12T17:06:40.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.186+0000 7fb3bf400640 1 --2- 172.21.15.52:0/2151669220 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb3a406ac00 0x7fb3a406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:40.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.186+0000 7fb3bf400640 1 -- 172.21.15.52:0/2151669220 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb3b00899c0 con 0x7fb3c8143460 2024-03-12T17:06:40.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.187+0000 7fb3c7400640 1 --2- 172.21.15.52:0/2151669220 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb3a406ac00 0x7fb3a406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:40.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.188+0000 7fb3c7400640 1 --2- 172.21.15.52:0/2151669220 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb3a406ac00 0x7fb3a406d0c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8036000 tx=0x7fb3b803b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:40.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.190+0000 7fb3bf400640 1 -- 172.21.15.52:0/2151669220 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fb3c813c070 con 0x7fb3c8143460 2024-03-12T17:06:40.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.395+0000 7fb3cd200640 1 -- 172.21.15.52:0/2151669220 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb3c8144050 con 0x7fb3c8143460 2024-03-12T17:06:40.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.396+0000 7fb3bf400640 1 -- 172.21.15.52:0/2151669220 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb3b00548a0 con 0x7fb3c8143460 2024-03-12T17:06:40.389 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-12T17:06:40.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.400+0000 7fb3cd200640 1 -- 172.21.15.52:0/2151669220 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb3a406ac00 msgr2=0x7fb3a406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:40.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.400+0000 7fb3cd200640 1 --2- 172.21.15.52:0/2151669220 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb3a406ac00 0x7fb3a406d0c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8036000 tx=0x7fb3b803b040 comp rx=0 tx=0).stop 2024-03-12T17:06:40.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.400+0000 7fb3cd200640 1 -- 172.21.15.52:0/2151669220 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3c8143460 msgr2=0x7fb3c815b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:40.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.400+0000 7fb3cd200640 1 --2- 172.21.15.52:0/2151669220 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3c8143460 0x7fb3c815b480 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb3b000c7f0 tx=0x7fb3b000ccb0 comp rx=0 tx=0).stop 2024-03-12T17:06:40.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.400+0000 7fb3cd200640 1 -- 172.21.15.52:0/2151669220 shutdown_connections 2024-03-12T17:06:40.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.400+0000 7fb3cd200640 1 --2- 172.21.15.52:0/2151669220 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb3a406ac00 0x7fb3a406d0c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:40.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.400+0000 7fb3cd200640 1 --2- 172.21.15.52:0/2151669220 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb3c8143930 0x7fb3c815b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:40.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.400+0000 7fb3cd200640 1 --2- 172.21.15.52:0/2151669220 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3c8143460 0x7fb3c815b480 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:40.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.400+0000 7fb3cd200640 1 --2- 172.21.15.52:0/2151669220 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3c80cdc50 0x7fb3c813dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:40.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.400+0000 7fb3cd200640 1 -- 172.21.15.52:0/2151669220 >> 172.21.15.52:0/2151669220 conn(0x7fb3c8065df0 msgr2=0x7fb3c8069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:40.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.400+0000 7fb3cd200640 1 -- 172.21.15.52:0/2151669220 shutdown_connections 2024-03-12T17:06:40.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.400+0000 7fb3cd200640 1 -- 172.21.15.52:0/2151669220 wait complete. 2024-03-12T17:06:40.412 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-12T17:06:40.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.639+0000 7f8686000640 1 Processor -- start 2024-03-12T17:06:40.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.639+0000 7f8686000640 1 -- start start 2024-03-12T17:06:40.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.641+0000 7f8686000640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f86800cf7c0 0x7f86800cd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:40.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.641+0000 7f8686000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8680160e90 0x7f86800d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:40.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.641+0000 7f8686000640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86800ccd90 con 0x7f868015c820 2024-03-12T17:06:40.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.641+0000 7f8686000640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86800cdd70 con 0x7f86800cf7c0 2024-03-12T17:06:40.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.641+0000 7f8686000640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86800b7fc0 con 0x7f8680160e90 2024-03-12T17:06:40.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.641+0000 7f867fe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f86800cf7c0 0x7f86800cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:40.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.641+0000 7f8684c00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f868015c820 0x7f86800cf0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:40986/0 (socket says 172.21.15.52:40986) 2024-03-12T17:06:40.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.641+0000 7f867fe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f86800cf7c0 0x7f86800cd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46658/0 (socket says 172.21.15.52:46658) 2024-03-12T17:06:40.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.641+0000 7f8684c00640 1 -- 172.21.15.52:0/2445012800 learned_addr learned my addr 172.21.15.52:0/2445012800 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:06:40.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.642+0000 7f8685600640 1 --2- 172.21.15.52:0/2445012800 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8680160e90 0x7f86800d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:40.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.642+0000 7f867fe00640 1 -- 172.21.15.52:0/2445012800 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8680160e90 msgr2=0x7f86800d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:40.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.642+0000 7f867fe00640 1 --2- 172.21.15.52:0/2445012800 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8680160e90 0x7f86800d0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:40.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.642+0000 7f867fe00640 1 -- 172.21.15.52:0/2445012800 >> v1:172.21.15.52:6789/0 conn(0x7f868015c820 legacy=0x7f86800cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:40.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.642+0000 7f867fe00640 1 -- 172.21.15.52:0/2445012800 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86800b82a0 con 0x7f86800cf7c0 2024-03-12T17:06:40.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.642+0000 7f867f400640 1 -- 172.21.15.52:0/2445012800 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1833984394 0 0) 0x7f86800ccd90 con 0x7f868015c820 2024-03-12T17:06:40.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.643+0000 7f867fe00640 1 --2- 172.21.15.52:0/2445012800 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86800cf7c0 0x7f86800cd830 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f86680097a0 tx=0x7f866802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a830d3e19cf935 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:40.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.643+0000 7f867f400640 1 -- 172.21.15.52:0/2445012800 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8668030610 con 0x7f86800cf7c0 2024-03-12T17:06:40.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.643+0000 7f867f400640 1 -- 172.21.15.52:0/2445012800 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8668030770 con 0x7f86800cf7c0 2024-03-12T17:06:40.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.643+0000 7f867f400640 1 -- 172.21.15.52:0/2445012800 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8668030b00 con 0x7f86800cf7c0 2024-03-12T17:06:40.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.644+0000 7f8686000640 1 -- 172.21.15.52:0/2445012800 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86800cf7c0 msgr2=0x7f86800cd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:40.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.644+0000 7f8686000640 1 --2- 172.21.15.52:0/2445012800 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86800cf7c0 0x7f86800cd830 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f86680097a0 tx=0x7f866802b960 comp rx=0 tx=0).stop 2024-03-12T17:06:40.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.644+0000 7f8686000640 1 -- 172.21.15.52:0/2445012800 shutdown_connections 2024-03-12T17:06:40.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.644+0000 7f8686000640 1 --2- 172.21.15.52:0/2445012800 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86800cf7c0 0x7f86800cd830 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:40.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.644+0000 7f8686000640 1 --2- 172.21.15.52:0/2445012800 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8680160e90 0x7f86800d0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:40.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.644+0000 7f8686000640 1 -- 172.21.15.52:0/2445012800 >> 172.21.15.52:0/2445012800 conn(0x7f868006b050 msgr2=0x7f868006b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:40.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.644+0000 7f8686000640 1 -- 172.21.15.52:0/2445012800 shutdown_connections 2024-03-12T17:06:40.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.644+0000 7f8686000640 1 -- 172.21.15.52:0/2445012800 wait complete. 2024-03-12T17:06:40.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.646+0000 7f8686000640 1 Processor -- start 2024-03-12T17:06:40.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.646+0000 7f8686000640 1 -- start start 2024-03-12T17:06:40.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.647+0000 7f8686000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86800cf7c0 0x7f86800a99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:40.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.647+0000 7f8686000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f868015c820 0x7f868014f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:40.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.647+0000 7f8686000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8680160e90 0x7f868015f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:40.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.647+0000 7f8686000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868015ef50 con 0x7f86800cf7c0 2024-03-12T17:06:40.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.647+0000 7f8686000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86800a9fa0 con 0x7f868015c820 2024-03-12T17:06:40.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.647+0000 7f8686000640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f868015e900 con 0x7f8680160e90 2024-03-12T17:06:40.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.647+0000 7f8684c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86800cf7c0 0x7f86800a99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:40.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.648+0000 7f8684c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86800cf7c0 0x7f86800a99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46672/0 (socket says 172.21.15.52:46672) 2024-03-12T17:06:40.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.648+0000 7f8684c00640 1 -- 172.21.15.52:0/4254913299 learned_addr learned my addr 172.21.15.52:0/4254913299 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:40.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.648+0000 7f8685600640 1 --2- 172.21.15.52:0/4254913299 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8680160e90 0x7f868015f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:40.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.648+0000 7f867fe00640 1 --2- 172.21.15.52:0/4254913299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f868015c820 0x7f868014f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:40.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.648+0000 7f8684c00640 1 -- 172.21.15.52:0/4254913299 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8680160e90 msgr2=0x7f868015f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:40.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.648+0000 7f8684c00640 1 --2- 172.21.15.52:0/4254913299 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8680160e90 0x7f868015f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:40.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.648+0000 7f8684c00640 1 -- 172.21.15.52:0/4254913299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f868015c820 msgr2=0x7f868014f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:40.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.648+0000 7f8684c00640 1 --2- 172.21.15.52:0/4254913299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f868015c820 0x7f868014f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:40.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.648+0000 7f8684c00640 1 -- 172.21.15.52:0/4254913299 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8668009400 con 0x7f86800cf7c0 2024-03-12T17:06:40.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.648+0000 7f867fe00640 1 --2- 172.21.15.52:0/4254913299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f868015c820 0x7f868014f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:40.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.648+0000 7f8684c00640 1 --2- 172.21.15.52:0/4254913299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86800cf7c0 0x7f86800a99a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f867000c4d0 tx=0x7f867000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:40.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.649+0000 7f867cc00640 1 -- 172.21.15.52:0/4254913299 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86700074e0 con 0x7f86800cf7c0 2024-03-12T17:06:40.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.649+0000 7f867cc00640 1 -- 172.21.15.52:0/4254913299 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8670007660 con 0x7f86800cf7c0 2024-03-12T17:06:40.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.649+0000 7f8686000640 1 -- 172.21.15.52:0/4254913299 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f868015ebe0 con 0x7f86800cf7c0 2024-03-12T17:06:40.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.649+0000 7f867cc00640 1 -- 172.21.15.52:0/4254913299 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8670004a20 con 0x7f86800cf7c0 2024-03-12T17:06:40.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.649+0000 7f8686000640 1 -- 172.21.15.52:0/4254913299 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f868015e5d0 con 0x7f86800cf7c0 2024-03-12T17:06:40.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.650+0000 7f8686000640 1 -- 172.21.15.52:0/4254913299 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86800d2480 con 0x7f86800cf7c0 2024-03-12T17:06:40.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.651+0000 7f867cc00640 1 -- 172.21.15.52:0/4254913299 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8670004b80 con 0x7f86800cf7c0 2024-03-12T17:06:40.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.652+0000 7f867cc00640 1 --2- 172.21.15.52:0/4254913299 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f865c06ac30 0x7f865c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:40.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.653+0000 7f867cc00640 1 -- 172.21.15.52:0/4254913299 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8670088bb0 con 0x7f86800cf7c0 2024-03-12T17:06:40.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.653+0000 7f867fe00640 1 --2- 172.21.15.52:0/4254913299 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f865c06ac30 0x7f865c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:40.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.654+0000 7f867fe00640 1 --2- 172.21.15.52:0/4254913299 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f865c06ac30 0x7f865c06d0f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8668036000 tx=0x7f866803b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:40.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.657+0000 7f867cc00640 1 -- 172.21.15.52:0/4254913299 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f8670053a00 con 0x7f86800cf7c0 2024-03-12T17:06:40.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.856+0000 7f8686000640 1 -- 172.21.15.52:0/4254913299 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f86800ce270 con 0x7f86800cf7c0 2024-03-12T17:06:40.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.857+0000 7f867cc00640 1 -- 172.21.15.52:0/4254913299 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8670018020 con 0x7f86800cf7c0 2024-03-12T17:06:40.850 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-03-12T17:06:40.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.859+0000 7f8686000640 1 -- 172.21.15.52:0/4254913299 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f865c06ac30 msgr2=0x7f865c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:40.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.860+0000 7f8686000640 1 --2- 172.21.15.52:0/4254913299 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f865c06ac30 0x7f865c06d0f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8668036000 tx=0x7f866803b040 comp rx=0 tx=0).stop 2024-03-12T17:06:40.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.860+0000 7f8686000640 1 -- 172.21.15.52:0/4254913299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86800cf7c0 msgr2=0x7f86800a99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:40.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.860+0000 7f8686000640 1 --2- 172.21.15.52:0/4254913299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86800cf7c0 0x7f86800a99a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f867000c4d0 tx=0x7f867000c990 comp rx=0 tx=0).stop 2024-03-12T17:06:40.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.860+0000 7f8686000640 1 -- 172.21.15.52:0/4254913299 shutdown_connections 2024-03-12T17:06:40.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.860+0000 7f8686000640 1 --2- 172.21.15.52:0/4254913299 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f865c06ac30 0x7f865c06d0f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:40.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.860+0000 7f8686000640 1 --2- 172.21.15.52:0/4254913299 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8680160e90 0x7f868015f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:40.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.860+0000 7f8686000640 1 --2- 172.21.15.52:0/4254913299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f868015c820 0x7f868014f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:40.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.860+0000 7f8686000640 1 --2- 172.21.15.52:0/4254913299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86800cf7c0 0x7f86800a99a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:40.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.860+0000 7f8686000640 1 -- 172.21.15.52:0/4254913299 >> 172.21.15.52:0/4254913299 conn(0x7f868006b050 msgr2=0x7f86800b76f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:40.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.860+0000 7f8686000640 1 -- 172.21.15.52:0/4254913299 shutdown_connections 2024-03-12T17:06:40.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:40.860+0000 7f8686000640 1 -- 172.21.15.52:0/4254913299 wait complete. 2024-03-12T17:06:40.871 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-12T17:06:41.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.108+0000 7f1cee200640 1 Processor -- start 2024-03-12T17:06:41.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.109+0000 7f1cee200640 1 -- start start 2024-03-12T17:06:41.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.110+0000 7f1cee200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f1ce80cd740 0x7f1ce80b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:41.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.110+0000 7f1cee200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1ce80b5aa0 0x7f1ce80b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:41.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.110+0000 7f1cee200640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80d20c0 con 0x7f1ce80d0d40 2024-03-12T17:06:41.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.110+0000 7f1cee200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80d14b0 con 0x7f1ce80cd740 2024-03-12T17:06:41.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.110+0000 7f1cee200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80d1790 con 0x7f1ce80b5aa0 2024-03-12T17:06:41.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.111+0000 7f1ce7400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f1ce80cd740 0x7f1ce80b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:41.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.111+0000 7f1ce7400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f1ce80cd740 0x7f1ce80b84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46676/0 (socket says 172.21.15.52:46676) 2024-03-12T17:06:41.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.111+0000 7f1ce7400640 1 -- 172.21.15.52:0/3724299983 learned_addr learned my addr 172.21.15.52:0/3724299983 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:41.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.111+0000 7f1ce7e00640 1 --2- 172.21.15.52:0/3724299983 >> v2:172.21.15.102:3300/0 conn(0x7f1ce80b5aa0 0x7f1ce80b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:41.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.111+0000 7f1ce7400640 1 -- 172.21.15.52:0/3724299983 >> v2:172.21.15.102:3300/0 conn(0x7f1ce80b5aa0 msgr2=0x7f1ce80b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:41.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.111+0000 7f1ce7400640 1 --2- 172.21.15.52:0/3724299983 >> v2:172.21.15.102:3300/0 conn(0x7f1ce80b5aa0 0x7f1ce80b5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:41.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.111+0000 7f1ce7400640 1 -- 172.21.15.52:0/3724299983 >> v1:172.21.15.52:6789/0 conn(0x7f1ce80d0d40 legacy=0x7f1ce80cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:41.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.111+0000 7f1ce7400640 1 -- 172.21.15.52:0/3724299983 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce80d1a70 con 0x7f1ce80cd740 2024-03-12T17:06:41.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.113+0000 7f1ce7400640 1 --2- 172.21.15.52:0/3724299983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ce80cd740 0x7f1ce80b84b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1ce80c99c0 tx=0x7f1cd802bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ef1379bc14273c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:41.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.113+0000 7f1ce6000640 1 -- 172.21.15.52:0/3724299983 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd802cb10 con 0x7f1ce80cd740 2024-03-12T17:06:41.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.113+0000 7f1ce6000640 1 -- 172.21.15.52:0/3724299983 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd802cc70 con 0x7f1ce80cd740 2024-03-12T17:06:41.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.113+0000 7f1ce6000640 1 -- 172.21.15.52:0/3724299983 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd80343f0 con 0x7f1ce80cd740 2024-03-12T17:06:41.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.113+0000 7f1cee200640 1 -- 172.21.15.52:0/3724299983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ce80cd740 msgr2=0x7f1ce80b84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:41.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.113+0000 7f1cee200640 1 --2- 172.21.15.52:0/3724299983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ce80cd740 0x7f1ce80b84b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1ce80c99c0 tx=0x7f1cd802bf70 comp rx=0 tx=0).stop 2024-03-12T17:06:41.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.114+0000 7f1cee200640 1 -- 172.21.15.52:0/3724299983 shutdown_connections 2024-03-12T17:06:41.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.114+0000 7f1cee200640 1 --2- 172.21.15.52:0/3724299983 >> v2:172.21.15.102:3300/0 conn(0x7f1ce80b5aa0 0x7f1ce80b5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:41.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.114+0000 7f1cee200640 1 --2- 172.21.15.52:0/3724299983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ce80cd740 0x7f1ce80b84b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:41.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.114+0000 7f1cee200640 1 -- 172.21.15.52:0/3724299983 >> 172.21.15.52:0/3724299983 conn(0x7f1ce80cf5d0 msgr2=0x7f1ce80cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:41.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.114+0000 7f1cee200640 1 -- 172.21.15.52:0/3724299983 shutdown_connections 2024-03-12T17:06:41.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.114+0000 7f1cee200640 1 -- 172.21.15.52:0/3724299983 wait complete. 2024-03-12T17:06:41.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.115+0000 7f1cee200640 1 Processor -- start 2024-03-12T17:06:41.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.116+0000 7f1cee200640 1 -- start start 2024-03-12T17:06:41.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.116+0000 7f1cee200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ce80b5aa0 0x7f1ce8154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:41.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.117+0000 7f1cee200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ce80cd740 0x7f1ce815b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:41.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.117+0000 7f1cee200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ce80d0d40 0x7f1ce8156e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:41.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.117+0000 7f1cee200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce81579d0 con 0x7f1ce80b5aa0 2024-03-12T17:06:41.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.117+0000 7f1cee200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce815a240 con 0x7f1ce80cd740 2024-03-12T17:06:41.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.117+0000 7f1cee200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce815a520 con 0x7f1ce80d0d40 2024-03-12T17:06:41.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.117+0000 7f1ce7400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ce80b5aa0 0x7f1ce8154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:41.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.117+0000 7f1ce7400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ce80b5aa0 0x7f1ce8154540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46686/0 (socket says 172.21.15.52:46686) 2024-03-12T17:06:41.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.117+0000 7f1ce7400640 1 -- 172.21.15.52:0/2350337385 learned_addr learned my addr 172.21.15.52:0/2350337385 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:41.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.117+0000 7f1ce7e00640 1 --2- 172.21.15.52:0/2350337385 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ce80d0d40 0x7f1ce8156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:41.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.117+0000 7f1ce6a00640 1 --2- 172.21.15.52:0/2350337385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ce80cd740 0x7f1ce815b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:41.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.117+0000 7f1ce7400640 1 -- 172.21.15.52:0/2350337385 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ce80d0d40 msgr2=0x7f1ce8156e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:41.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.117+0000 7f1ce7400640 1 --2- 172.21.15.52:0/2350337385 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ce80d0d40 0x7f1ce8156e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:41.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.117+0000 7f1ce7400640 1 -- 172.21.15.52:0/2350337385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ce80cd740 msgr2=0x7f1ce815b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:41.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.117+0000 7f1ce7400640 1 --2- 172.21.15.52:0/2350337385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ce80cd740 0x7f1ce815b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:41.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.117+0000 7f1ce7400640 1 -- 172.21.15.52:0/2350337385 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd8009b30 con 0x7f1ce80b5aa0 2024-03-12T17:06:41.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.117+0000 7f1ce7e00640 1 --2- 172.21.15.52:0/2350337385 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ce80d0d40 0x7f1ce8156e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:41.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.117+0000 7f1ce6a00640 1 --2- 172.21.15.52:0/2350337385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ce80cd740 0x7f1ce815b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:41.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.118+0000 7f1ce7400640 1 --2- 172.21.15.52:0/2350337385 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ce80b5aa0 0x7f1ce8154540 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8000f80 tx=0x7f1cd8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:41.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.118+0000 7f1cdf400640 1 -- 172.21.15.52:0/2350337385 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd8009510 con 0x7f1ce80b5aa0 2024-03-12T17:06:41.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.118+0000 7f1cdf400640 1 -- 172.21.15.52:0/2350337385 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd803e450 con 0x7f1ce80b5aa0 2024-03-12T17:06:41.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.118+0000 7f1cee200640 1 -- 172.21.15.52:0/2350337385 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ce815a800 con 0x7f1ce80b5aa0 2024-03-12T17:06:41.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.118+0000 7f1cdf400640 1 -- 172.21.15.52:0/2350337385 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd803e800 con 0x7f1ce80b5aa0 2024-03-12T17:06:41.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.118+0000 7f1cee200640 1 -- 172.21.15.52:0/2350337385 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce81f6480 con 0x7f1ce80b5aa0 2024-03-12T17:06:41.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.119+0000 7f1cee200640 1 -- 172.21.15.52:0/2350337385 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ce8073390 con 0x7f1ce80b5aa0 2024-03-12T17:06:41.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.120+0000 7f1cdf400640 1 -- 172.21.15.52:0/2350337385 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1cd803e960 con 0x7f1ce80b5aa0 2024-03-12T17:06:41.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.121+0000 7f1cdf400640 1 --2- 172.21.15.52:0/2350337385 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1cc406ac00 0x7f1cc406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:41.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.121+0000 7f1cdf400640 1 -- 172.21.15.52:0/2350337385 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1cd80ac510 con 0x7f1ce80b5aa0 2024-03-12T17:06:41.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.122+0000 7f1ce6a00640 1 --2- 172.21.15.52:0/2350337385 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1cc406ac00 0x7f1cc406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:41.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.123+0000 7f1ce6a00640 1 --2- 172.21.15.52:0/2350337385 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1cc406ac00 0x7f1cc406d0c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1ce80cff70 tx=0x7f1cd0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:41.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.126+0000 7f1cdf400640 1 -- 172.21.15.52:0/2350337385 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f1cd8077360 con 0x7f1ce80b5aa0 2024-03-12T17:06:41.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.330+0000 7f1cee200640 1 -- 172.21.15.52:0/2350337385 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1ce80cec90 con 0x7f1ce80b5aa0 2024-03-12T17:06:41.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.331+0000 7f1cdf400640 1 -- 172.21.15.52:0/2350337385 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1cd807a980 con 0x7f1ce80b5aa0 2024-03-12T17:06:41.324 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-12T17:06:41.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.334+0000 7f1cee200640 1 -- 172.21.15.52:0/2350337385 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1cc406ac00 msgr2=0x7f1cc406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:41.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.334+0000 7f1cee200640 1 --2- 172.21.15.52:0/2350337385 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1cc406ac00 0x7f1cc406d0c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1ce80cff70 tx=0x7f1cd0009040 comp rx=0 tx=0).stop 2024-03-12T17:06:41.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.334+0000 7f1cee200640 1 -- 172.21.15.52:0/2350337385 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ce80b5aa0 msgr2=0x7f1ce8154540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:41.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.334+0000 7f1cee200640 1 --2- 172.21.15.52:0/2350337385 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ce80b5aa0 0x7f1ce8154540 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8000f80 tx=0x7f1cd8004a40 comp rx=0 tx=0).stop 2024-03-12T17:06:41.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.334+0000 7f1cee200640 1 -- 172.21.15.52:0/2350337385 shutdown_connections 2024-03-12T17:06:41.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.334+0000 7f1cee200640 1 --2- 172.21.15.52:0/2350337385 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1cc406ac00 0x7f1cc406d0c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:41.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.334+0000 7f1cee200640 1 --2- 172.21.15.52:0/2350337385 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ce80d0d40 0x7f1ce8156e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:41.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.334+0000 7f1cee200640 1 --2- 172.21.15.52:0/2350337385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ce80cd740 0x7f1ce815b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:41.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.334+0000 7f1cee200640 1 --2- 172.21.15.52:0/2350337385 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ce80b5aa0 0x7f1ce8154540 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:41.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.334+0000 7f1cee200640 1 -- 172.21.15.52:0/2350337385 >> 172.21.15.52:0/2350337385 conn(0x7f1ce80cf5d0 msgr2=0x7f1ce80b4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:41.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.334+0000 7f1cee200640 1 -- 172.21.15.52:0/2350337385 shutdown_connections 2024-03-12T17:06:41.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.334+0000 7f1cee200640 1 -- 172.21.15.52:0/2350337385 wait complete. 2024-03-12T17:06:41.346 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-12T17:06:41.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.578+0000 7ff149800640 1 Processor -- start 2024-03-12T17:06:41.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.578+0000 7ff149800640 1 -- start start 2024-03-12T17:06:41.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.580+0000 7ff149800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff144146070 0x7ff1440b2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:41.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.580+0000 7ff149800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff14415cd00 0x7ff14406cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:41.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.580+0000 7ff149800640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff1440700d0 0x7ff144070530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:41.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.580+0000 7ff149800640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff14406d130 con 0x7ff1440700d0 2024-03-12T17:06:41.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.580+0000 7ff149800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff144070a70 con 0x7ff144146070 2024-03-12T17:06:41.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.580+0000 7ff149800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1440b2f70 con 0x7ff14415cd00 2024-03-12T17:06:41.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.580+0000 7ff148e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff1440700d0 0x7ff144070530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:41.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.580+0000 7ff148e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff1440700d0 0x7ff144070530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46690/0 (socket says 172.21.15.52:46690) 2024-03-12T17:06:41.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.580+0000 7ff148e00640 1 -- 172.21.15.52:0/1172901646 learned_addr learned my addr 172.21.15.52:0/1172901646 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:41.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.581+0000 7ff143400640 1 --2- 172.21.15.52:0/1172901646 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff14415cd00 0x7ff14406cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:41.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.581+0000 7ff143e00640 1 --2- 172.21.15.52:0/1172901646 >> v2:172.21.15.102:3300/0 conn(0x7ff144146070 0x7ff1440b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:41.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.581+0000 7ff148e00640 1 -- 172.21.15.52:0/1172901646 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff14415cd00 msgr2=0x7ff14406cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:41.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.581+0000 7ff148e00640 1 --2- 172.21.15.52:0/1172901646 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff14415cd00 0x7ff14406cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:41.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.581+0000 7ff148e00640 1 -- 172.21.15.52:0/1172901646 >> v2:172.21.15.102:3300/0 conn(0x7ff144146070 msgr2=0x7ff1440b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:41.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.581+0000 7ff148e00640 1 --2- 172.21.15.52:0/1172901646 >> v2:172.21.15.102:3300/0 conn(0x7ff144146070 0x7ff1440b2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:41.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.581+0000 7ff148e00640 1 -- 172.21.15.52:0/1172901646 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1440b3250 con 0x7ff1440700d0 2024-03-12T17:06:41.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.581+0000 7ff143400640 1 --2- 172.21.15.52:0/1172901646 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff14415cd00 0x7ff14406cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:06:41.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.583+0000 7ff148e00640 1 --2- 172.21.15.52:0/1172901646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1440700d0 0x7ff144070530 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff144146640 tx=0x7ff13c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6035c2715240449 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:41.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.583+0000 7ff142a00640 1 -- 172.21.15.52:0/1172901646 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c02c9e0 con 0x7ff1440700d0 2024-03-12T17:06:41.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.583+0000 7ff142a00640 1 -- 172.21.15.52:0/1172901646 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff13c02cb40 con 0x7ff1440700d0 2024-03-12T17:06:41.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.583+0000 7ff142a00640 1 -- 172.21.15.52:0/1172901646 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c0343f0 con 0x7ff1440700d0 2024-03-12T17:06:41.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.584+0000 7ff149800640 1 -- 172.21.15.52:0/1172901646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1440700d0 msgr2=0x7ff144070530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:41.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.584+0000 7ff149800640 1 --2- 172.21.15.52:0/1172901646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1440700d0 0x7ff144070530 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff144146640 tx=0x7ff13c02be40 comp rx=0 tx=0).stop 2024-03-12T17:06:41.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.584+0000 7ff149800640 1 -- 172.21.15.52:0/1172901646 shutdown_connections 2024-03-12T17:06:41.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.584+0000 7ff149800640 1 --2- 172.21.15.52:0/1172901646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1440700d0 0x7ff144070530 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:41.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.584+0000 7ff149800640 1 --2- 172.21.15.52:0/1172901646 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff14415cd00 0x7ff14406cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:41.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.584+0000 7ff149800640 1 --2- 172.21.15.52:0/1172901646 >> v2:172.21.15.102:3300/0 conn(0x7ff144146070 0x7ff1440b2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:41.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.584+0000 7ff149800640 1 -- 172.21.15.52:0/1172901646 >> 172.21.15.52:0/1172901646 conn(0x7ff144065df0 msgr2=0x7ff144069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:41.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.584+0000 7ff149800640 1 -- 172.21.15.52:0/1172901646 shutdown_connections 2024-03-12T17:06:41.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.584+0000 7ff149800640 1 -- 172.21.15.52:0/1172901646 wait complete. 2024-03-12T17:06:41.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.586+0000 7ff149800640 1 Processor -- start 2024-03-12T17:06:41.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.586+0000 7ff149800640 1 -- start start 2024-03-12T17:06:41.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.587+0000 7ff149800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1440700d0 0x7ff144157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:41.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.587+0000 7ff149800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff144146070 0x7ff14413e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:41.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.587+0000 7ff149800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff14415cd00 0x7ff14413edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:41.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.587+0000 7ff149800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14413fb20 con 0x7ff144146070 2024-03-12T17:06:41.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.587+0000 7ff149800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14413f2f0 con 0x7ff1440700d0 2024-03-12T17:06:41.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.587+0000 7ff149800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff14413f560 con 0x7ff14415cd00 2024-03-12T17:06:41.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.587+0000 7ff143400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff144146070 0x7ff14413e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:41.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.587+0000 7ff143400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff144146070 0x7ff14413e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46696/0 (socket says 172.21.15.52:46696) 2024-03-12T17:06:41.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.587+0000 7ff143400640 1 -- 172.21.15.52:0/3998826808 learned_addr learned my addr 172.21.15.52:0/3998826808 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:41.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.587+0000 7ff148e00640 1 --2- 172.21.15.52:0/3998826808 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff14415cd00 0x7ff14413edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:41.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.587+0000 7ff143e00640 1 --2- 172.21.15.52:0/3998826808 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1440700d0 0x7ff144157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:41.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.587+0000 7ff143400640 1 -- 172.21.15.52:0/3998826808 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff14415cd00 msgr2=0x7ff14413edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:41.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.587+0000 7ff143400640 1 --2- 172.21.15.52:0/3998826808 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff14415cd00 0x7ff14413edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:41.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.587+0000 7ff143400640 1 -- 172.21.15.52:0/3998826808 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1440700d0 msgr2=0x7ff144157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:41.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.587+0000 7ff143400640 1 --2- 172.21.15.52:0/3998826808 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1440700d0 0x7ff144157fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:41.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.587+0000 7ff143400640 1 -- 172.21.15.52:0/3998826808 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13c009b30 con 0x7ff144146070 2024-03-12T17:06:41.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.587+0000 7ff148e00640 1 --2- 172.21.15.52:0/3998826808 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff14415cd00 0x7ff14413edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:41.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.588+0000 7ff143400640 1 --2- 172.21.15.52:0/3998826808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff144146070 0x7ff14413e870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff13000d800 tx=0x7ff13000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:41.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.588+0000 7ff13be00640 1 -- 172.21.15.52:0/3998826808 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff130004030 con 0x7ff144146070 2024-03-12T17:06:41.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.588+0000 7ff13be00640 1 -- 172.21.15.52:0/3998826808 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff130004190 con 0x7ff144146070 2024-03-12T17:06:41.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.588+0000 7ff13be00640 1 -- 172.21.15.52:0/3998826808 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff130004520 con 0x7ff144146070 2024-03-12T17:06:41.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.588+0000 7ff149800640 1 -- 172.21.15.52:0/3998826808 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff14413f840 con 0x7ff144146070 2024-03-12T17:06:41.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.588+0000 7ff149800640 1 -- 172.21.15.52:0/3998826808 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff14412dbb0 con 0x7ff144146070 2024-03-12T17:06:41.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.590+0000 7ff149800640 1 -- 172.21.15.52:0/3998826808 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff14412d730 con 0x7ff144146070 2024-03-12T17:06:41.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.590+0000 7ff13be00640 1 -- 172.21.15.52:0/3998826808 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff13000b6c0 con 0x7ff144146070 2024-03-12T17:06:41.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.592+0000 7ff13be00640 1 --2- 172.21.15.52:0/3998826808 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff12406ac00 0x7ff12406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:41.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.592+0000 7ff13be00640 1 -- 172.21.15.52:0/3998826808 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff130089750 con 0x7ff144146070 2024-03-12T17:06:41.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.592+0000 7ff143e00640 1 --2- 172.21.15.52:0/3998826808 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff12406ac00 0x7ff12406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:41.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.594+0000 7ff143e00640 1 --2- 172.21.15.52:0/3998826808 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff12406ac00 0x7ff12406d0c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff134006fd0 tx=0x7ff134008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:41.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.596+0000 7ff13be00640 1 -- 172.21.15.52:0/3998826808 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7ff14412d730 con 0x7ff144146070 2024-03-12T17:06:41.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.811+0000 7ff149800640 1 -- 172.21.15.52:0/3998826808 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff14412d730 con 0x7ff144146070 2024-03-12T17:06:41.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.812+0000 7ff13be00640 1 -- 172.21.15.52:0/3998826808 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff14412d730 con 0x7ff144146070 2024-03-12T17:06:41.804 INFO:teuthology.orchestra.run.smithi052.stderr:listed 0 entries 2024-03-12T17:06:41.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.814+0000 7ff149800640 1 -- 172.21.15.52:0/3998826808 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff12406ac00 msgr2=0x7ff12406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:41.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.814+0000 7ff149800640 1 --2- 172.21.15.52:0/3998826808 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff12406ac00 0x7ff12406d0c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff134006fd0 tx=0x7ff134008040 comp rx=0 tx=0).stop 2024-03-12T17:06:41.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.814+0000 7ff149800640 1 -- 172.21.15.52:0/3998826808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff144146070 msgr2=0x7ff14413e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:41.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.814+0000 7ff149800640 1 --2- 172.21.15.52:0/3998826808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff144146070 0x7ff14413e870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff13000d800 tx=0x7ff13000dcc0 comp rx=0 tx=0).stop 2024-03-12T17:06:41.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.814+0000 7ff149800640 1 -- 172.21.15.52:0/3998826808 shutdown_connections 2024-03-12T17:06:41.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.814+0000 7ff149800640 1 --2- 172.21.15.52:0/3998826808 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff12406ac00 0x7ff12406d0c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:41.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.814+0000 7ff149800640 1 --2- 172.21.15.52:0/3998826808 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff14415cd00 0x7ff14413edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:41.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.814+0000 7ff149800640 1 --2- 172.21.15.52:0/3998826808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff144146070 0x7ff14413e870 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:41.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.814+0000 7ff149800640 1 --2- 172.21.15.52:0/3998826808 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1440700d0 0x7ff144157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:41.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.814+0000 7ff149800640 1 -- 172.21.15.52:0/3998826808 >> 172.21.15.52:0/3998826808 conn(0x7ff144065df0 msgr2=0x7ff14415c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:41.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.815+0000 7ff149800640 1 -- 172.21.15.52:0/3998826808 shutdown_connections 2024-03-12T17:06:41.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:41.815+0000 7ff149800640 1 -- 172.21.15.52:0/3998826808 wait complete. 2024-03-12T17:06:41.826 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:06:41.826 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-12T17:06:41.880 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:06:42.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.104+0000 7fdb3f200640 1 Processor -- start 2024-03-12T17:06:42.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.105+0000 7fdb3f200640 1 -- start start 2024-03-12T17:06:42.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.106+0000 7fdb3f200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fdb4015cab0 0x7fdb400b8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:42.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.106+0000 7fdb3f200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdb400ceeb0 0x7fdb400cf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:42.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.106+0000 7fdb3f200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb400cf850 con 0x7fdb400b4120 2024-03-12T17:06:42.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.106+0000 7fdb3f200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb4015ce90 con 0x7fdb400ceeb0 2024-03-12T17:06:42.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.106+0000 7fdb3f200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb400cddf0 con 0x7fdb4015cab0 2024-03-12T17:06:42.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.107+0000 7fdb35400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdb400ceeb0 0x7fdb400cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:42.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.107+0000 7fdb35400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdb400ceeb0 0x7fdb400cf310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46698/0 (socket says 172.21.15.52:46698) 2024-03-12T17:06:42.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.107+0000 7fdb35400640 1 -- 172.21.15.52:0/1737564707 learned_addr learned my addr 172.21.15.52:0/1737564707 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:42.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.107+0000 7fdb3de00640 1 --2- 172.21.15.52:0/1737564707 >> v2:172.21.15.102:3300/0 conn(0x7fdb4015cab0 0x7fdb400b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:42.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.107+0000 7fdb35400640 1 -- 172.21.15.52:0/1737564707 >> v2:172.21.15.102:3300/0 conn(0x7fdb4015cab0 msgr2=0x7fdb400b8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:42.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.107+0000 7fdb35400640 1 --2- 172.21.15.52:0/1737564707 >> v2:172.21.15.102:3300/0 conn(0x7fdb4015cab0 0x7fdb400b8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:42.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.107+0000 7fdb35400640 1 -- 172.21.15.52:0/1737564707 >> v1:172.21.15.102:6789/0 conn(0x7fdb400b4120 legacy=0x7fdb400b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:42.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.107+0000 7fdb35400640 1 -- 172.21.15.52:0/1737564707 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb400cd570 con 0x7fdb400ceeb0 2024-03-12T17:06:42.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.108+0000 7fdb35400640 1 --2- 172.21.15.52:0/1737564707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb400ceeb0 0x7fdb400cf310 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdb4008b100 tx=0x7fdb2802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd1cc5f1039eed8c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:42.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.109+0000 7fdb3d400640 1 -- 172.21.15.52:0/1737564707 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb28031870 con 0x7fdb400ceeb0 2024-03-12T17:06:42.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.109+0000 7fdb3d400640 1 -- 172.21.15.52:0/1737564707 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb280319d0 con 0x7fdb400ceeb0 2024-03-12T17:06:42.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.109+0000 7fdb3d400640 1 -- 172.21.15.52:0/1737564707 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb28031d60 con 0x7fdb400ceeb0 2024-03-12T17:06:42.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.109+0000 7fdb3f200640 1 -- 172.21.15.52:0/1737564707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb400ceeb0 msgr2=0x7fdb400cf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:42.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.109+0000 7fdb3f200640 1 --2- 172.21.15.52:0/1737564707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb400ceeb0 0x7fdb400cf310 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdb4008b100 tx=0x7fdb2802b910 comp rx=0 tx=0).stop 2024-03-12T17:06:42.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.109+0000 7fdb3f200640 1 -- 172.21.15.52:0/1737564707 shutdown_connections 2024-03-12T17:06:42.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.109+0000 7fdb3f200640 1 --2- 172.21.15.52:0/1737564707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb400ceeb0 0x7fdb400cf310 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:42.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.109+0000 7fdb3f200640 1 --2- 172.21.15.52:0/1737564707 >> v2:172.21.15.102:3300/0 conn(0x7fdb4015cab0 0x7fdb400b8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:42.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.109+0000 7fdb3f200640 1 -- 172.21.15.52:0/1737564707 >> 172.21.15.52:0/1737564707 conn(0x7fdb4006ae90 msgr2=0x7fdb4006b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:42.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.110+0000 7fdb3f200640 1 -- 172.21.15.52:0/1737564707 shutdown_connections 2024-03-12T17:06:42.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.110+0000 7fdb3f200640 1 -- 172.21.15.52:0/1737564707 wait complete. 2024-03-12T17:06:42.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.111+0000 7fdb3f200640 1 Processor -- start 2024-03-12T17:06:42.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.111+0000 7fdb3f200640 1 -- start start 2024-03-12T17:06:42.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.112+0000 7fdb3f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb400b4120 0x7fdb401f2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:42.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.112+0000 7fdb3f200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb400ceeb0 0x7fdb40154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:42.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.112+0000 7fdb3f200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdb4015cab0 0x7fdb40156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:42.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.112+0000 7fdb3f200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb401f3010 con 0x7fdb400ceeb0 2024-03-12T17:06:42.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.112+0000 7fdb3f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb40155330 con 0x7fdb400b4120 2024-03-12T17:06:42.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.112+0000 7fdb3f200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb400b5d90 con 0x7fdb4015cab0 2024-03-12T17:06:42.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.113+0000 7fdb35400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb400ceeb0 0x7fdb40154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:42.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.113+0000 7fdb35400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb400ceeb0 0x7fdb40154df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46700/0 (socket says 172.21.15.52:46700) 2024-03-12T17:06:42.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.113+0000 7fdb35400640 1 -- 172.21.15.52:0/2004404715 learned_addr learned my addr 172.21.15.52:0/2004404715 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:42.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.113+0000 7fdb3de00640 1 --2- 172.21.15.52:0/2004404715 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb400b4120 0x7fdb401f2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:42.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.113+0000 7fdb3e800640 1 --2- 172.21.15.52:0/2004404715 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdb4015cab0 0x7fdb40156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:42.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.113+0000 7fdb35400640 1 -- 172.21.15.52:0/2004404715 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdb4015cab0 msgr2=0x7fdb40156ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:42.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.113+0000 7fdb35400640 1 --2- 172.21.15.52:0/2004404715 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdb4015cab0 0x7fdb40156ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:42.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.113+0000 7fdb35400640 1 -- 172.21.15.52:0/2004404715 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb400b4120 msgr2=0x7fdb401f2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:42.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.113+0000 7fdb35400640 1 --2- 172.21.15.52:0/2004404715 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb400b4120 0x7fdb401f2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:42.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.113+0000 7fdb35400640 1 -- 172.21.15.52:0/2004404715 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb280094d0 con 0x7fdb400ceeb0 2024-03-12T17:06:42.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.113+0000 7fdb3e800640 1 --2- 172.21.15.52:0/2004404715 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdb4015cab0 0x7fdb40156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:42.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.114+0000 7fdb35400640 1 --2- 172.21.15.52:0/2004404715 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb400ceeb0 0x7fdb40154df0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdb28000c00 tx=0x7fdb280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:42.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.114+0000 7fdb36a00640 1 -- 172.21.15.52:0/2004404715 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2803abe0 con 0x7fdb400ceeb0 2024-03-12T17:06:42.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.114+0000 7fdb36a00640 1 -- 172.21.15.52:0/2004404715 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2803ad40 con 0x7fdb400ceeb0 2024-03-12T17:06:42.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.114+0000 7fdb36a00640 1 -- 172.21.15.52:0/2004404715 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb28044610 con 0x7fdb400ceeb0 2024-03-12T17:06:42.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.114+0000 7fdb3f200640 1 -- 172.21.15.52:0/2004404715 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb400b6070 con 0x7fdb400ceeb0 2024-03-12T17:06:42.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.114+0000 7fdb3f200640 1 -- 172.21.15.52:0/2004404715 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb400b6630 con 0x7fdb400ceeb0 2024-03-12T17:06:42.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.115+0000 7fdb3f200640 1 -- 172.21.15.52:0/2004404715 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb4004e030 con 0x7fdb400ceeb0 2024-03-12T17:06:42.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.117+0000 7fdb36a00640 1 -- 172.21.15.52:0/2004404715 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fdb2804d450 con 0x7fdb400ceeb0 2024-03-12T17:06:42.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.117+0000 7fdb36a00640 1 --2- 172.21.15.52:0/2004404715 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fdb1c06ac00 0x7fdb1c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:42.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.118+0000 7fdb36a00640 1 -- 172.21.15.52:0/2004404715 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdb2802d040 con 0x7fdb400ceeb0 2024-03-12T17:06:42.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.118+0000 7fdb3de00640 1 --2- 172.21.15.52:0/2004404715 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fdb1c06ac00 0x7fdb1c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:42.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.119+0000 7fdb3de00640 1 --2- 172.21.15.52:0/2004404715 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fdb1c06ac00 0x7fdb1c06d0c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdb300099c0 tx=0x7fdb30003990 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:42.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.124+0000 7fdb36a00640 1 -- 172.21.15.52:0/2004404715 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fdb28076bf0 con 0x7fdb400ceeb0 2024-03-12T17:06:42.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.327+0000 7fdb3f200640 1 -- 172.21.15.52:0/2004404715 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb400d02d0 con 0x7fdb400ceeb0 2024-03-12T17:06:42.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.329+0000 7fdb36a00640 1 -- 172.21.15.52:0/2004404715 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7fdb28035020 con 0x7fdb400ceeb0 2024-03-12T17:06:42.322 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:06:42.323 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":25,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:06:30.468476+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T17:06:19.218543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T17:06:23.525474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T17:06:24.911574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-12T17:06:28.017172+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.733031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.775178+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.591522+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.820838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.829252+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.582481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:06:42.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.332+0000 7fdb3f200640 1 -- 172.21.15.52:0/2004404715 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fdb1c06ac00 msgr2=0x7fdb1c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:42.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.332+0000 7fdb3f200640 1 --2- 172.21.15.52:0/2004404715 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fdb1c06ac00 0x7fdb1c06d0c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdb300099c0 tx=0x7fdb30003990 comp rx=0 tx=0).stop 2024-03-12T17:06:42.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.332+0000 7fdb3f200640 1 -- 172.21.15.52:0/2004404715 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb400ceeb0 msgr2=0x7fdb40154df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:42.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.332+0000 7fdb3f200640 1 --2- 172.21.15.52:0/2004404715 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb400ceeb0 0x7fdb40154df0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdb28000c00 tx=0x7fdb280046c0 comp rx=0 tx=0).stop 2024-03-12T17:06:42.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.332+0000 7fdb3f200640 1 -- 172.21.15.52:0/2004404715 shutdown_connections 2024-03-12T17:06:42.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.332+0000 7fdb3f200640 1 --2- 172.21.15.52:0/2004404715 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fdb1c06ac00 0x7fdb1c06d0c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:42.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.332+0000 7fdb3f200640 1 --2- 172.21.15.52:0/2004404715 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdb4015cab0 0x7fdb40156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:42.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.332+0000 7fdb3f200640 1 --2- 172.21.15.52:0/2004404715 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb400ceeb0 0x7fdb40154df0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:42.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.332+0000 7fdb3f200640 1 --2- 172.21.15.52:0/2004404715 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb400b4120 0x7fdb401f2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:42.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.332+0000 7fdb3f200640 1 -- 172.21.15.52:0/2004404715 >> 172.21.15.52:0/2004404715 conn(0x7fdb4006ae90 msgr2=0x7fdb4006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:42.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.332+0000 7fdb3f200640 1 -- 172.21.15.52:0/2004404715 shutdown_connections 2024-03-12T17:06:42.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.333+0000 7fdb3f200640 1 -- 172.21.15.52:0/2004404715 wait complete. 2024-03-12T17:06:42.344 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T17:06:19.218543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-12T17:06:23.525474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-12T17:06:24.911574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-12T17:06:28.017172+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-12T17:06:42.345 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T17:06:42.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.573+0000 7fb41d000640 1 Processor -- start 2024-03-12T17:06:42.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.574+0000 7fb41d000640 1 -- start start 2024-03-12T17:06:42.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.575+0000 7fb41d000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb4180af5e0 0x7fb4180b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:42.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.575+0000 7fb41d000640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb418099130 con 0x7fb4180995c0 2024-03-12T17:06:42.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.576+0000 7fb41d000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb41806b180 con 0x7fb4181610a0 2024-03-12T17:06:42.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.576+0000 7fb41d000640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb418098ae0 con 0x7fb4180af5e0 2024-03-12T17:06:42.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.576+0000 7fb417e00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fb4180995c0 0x7fb4180c34f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:41010/0 (socket says 172.21.15.52:41010) 2024-03-12T17:06:42.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.576+0000 7fb417e00640 1 -- 172.21.15.52:0/1172568335 learned_addr learned my addr 172.21.15.52:0/1172568335 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:06:42.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.576+0000 7fb416a00640 1 --2- 172.21.15.52:0/1172568335 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb4180af5e0 0x7fb4180b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:42.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.577+0000 7fb416000640 1 -- 172.21.15.52:0/1172568335 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 363890450 0 0) 0x7fb418099130 con 0x7fb4180995c0 2024-03-12T17:06:42.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.577+0000 7fb416000640 1 -- 172.21.15.52:0/1172568335 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3fc003520 con 0x7fb4180995c0 2024-03-12T17:06:42.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.577+0000 7fb416a00640 1 -- 172.21.15.52:0/1172568335 >> v1:172.21.15.102:6789/0 conn(0x7fb4181610a0 legacy=0x7fb4180cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:42.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.577+0000 7fb416a00640 1 -- 172.21.15.52:0/1172568335 >> v1:172.21.15.52:6789/0 conn(0x7fb4180995c0 legacy=0x7fb4180c34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:42.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.577+0000 7fb416a00640 1 -- 172.21.15.52:0/1172568335 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb418098dc0 con 0x7fb4180af5e0 2024-03-12T17:06:42.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.578+0000 7fb416a00640 1 --2- 172.21.15.52:0/1172568335 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb4180af5e0 0x7fb4180b2b30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb4000097d0 tx=0x7fb40002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a187315cdf594d1d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:42.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.579+0000 7fb416000640 1 -- 172.21.15.52:0/1172568335 <== mon.2 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb400031800 con 0x7fb4180af5e0 2024-03-12T17:06:42.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.579+0000 7fb416000640 1 -- 172.21.15.52:0/1172568335 <== mon.2 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb400031960 con 0x7fb4180af5e0 2024-03-12T17:06:42.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.579+0000 7fb416000640 1 -- 172.21.15.52:0/1172568335 <== mon.2 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb400031cf0 con 0x7fb4180af5e0 2024-03-12T17:06:42.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.579+0000 7fb41d000640 1 -- 172.21.15.52:0/1172568335 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb4180af5e0 msgr2=0x7fb4180b2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:42.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.579+0000 7fb41d000640 1 --2- 172.21.15.52:0/1172568335 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb4180af5e0 0x7fb4180b2b30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb4000097d0 tx=0x7fb40002b8a0 comp rx=0 tx=0).stop 2024-03-12T17:06:42.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.580+0000 7fb41d000640 1 -- 172.21.15.52:0/1172568335 shutdown_connections 2024-03-12T17:06:42.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.580+0000 7fb41d000640 1 --2- 172.21.15.52:0/1172568335 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb4180af5e0 0x7fb4180b2b30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:42.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.580+0000 7fb41d000640 1 -- 172.21.15.52:0/1172568335 >> 172.21.15.52:0/1172568335 conn(0x7fb4180b2010 msgr2=0x7fb4180b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:42.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.580+0000 7fb41d000640 1 -- 172.21.15.52:0/1172568335 shutdown_connections 2024-03-12T17:06:42.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.580+0000 7fb41d000640 1 -- 172.21.15.52:0/1172568335 wait complete. 2024-03-12T17:06:42.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.581+0000 7fb41d000640 1 Processor -- start 2024-03-12T17:06:42.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.582+0000 7fb41d000640 1 -- start start 2024-03-12T17:06:42.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.583+0000 7fb41d000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb4180995c0 0x7fb4180b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:42.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.583+0000 7fb41d000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4180af5e0 0x7fb4180b3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:42.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.583+0000 7fb41d000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4181610a0 0x7fb41815caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:42.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.583+0000 7fb41d000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4180c8910 con 0x7fb4181610a0 2024-03-12T17:06:42.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.583+0000 7fb41d000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4180b3680 con 0x7fb4180af5e0 2024-03-12T17:06:42.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.583+0000 7fb41d000640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4180b3900 con 0x7fb4180995c0 2024-03-12T17:06:42.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.583+0000 7fb417e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4181610a0 0x7fb41815caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:42.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.583+0000 7fb417e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4181610a0 0x7fb41815caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46706/0 (socket says 172.21.15.52:46706) 2024-03-12T17:06:42.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.583+0000 7fb417e00640 1 -- 172.21.15.52:0/1638359851 learned_addr learned my addr 172.21.15.52:0/1638359851 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:42.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.583+0000 7fb416a00640 1 --2- 172.21.15.52:0/1638359851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4180af5e0 0x7fb4180b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:42.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.583+0000 7fb417400640 1 --2- 172.21.15.52:0/1638359851 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb4180995c0 0x7fb4180b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:42.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.583+0000 7fb417e00640 1 -- 172.21.15.52:0/1638359851 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb4180995c0 msgr2=0x7fb4180b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:42.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.583+0000 7fb417e00640 1 --2- 172.21.15.52:0/1638359851 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb4180995c0 0x7fb4180b5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:42.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.583+0000 7fb417e00640 1 -- 172.21.15.52:0/1638359851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4180af5e0 msgr2=0x7fb4180b3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:42.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.583+0000 7fb417e00640 1 --2- 172.21.15.52:0/1638359851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4180af5e0 0x7fb4180b3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:42.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.583+0000 7fb417e00640 1 -- 172.21.15.52:0/1638359851 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb400009400 con 0x7fb4181610a0 2024-03-12T17:06:42.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.583+0000 7fb416a00640 1 --2- 172.21.15.52:0/1638359851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4180af5e0 0x7fb4180b3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:42.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.583+0000 7fb417400640 1 --2- 172.21.15.52:0/1638359851 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb4180995c0 0x7fb4180b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:42.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.584+0000 7fb417e00640 1 --2- 172.21.15.52:0/1638359851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4181610a0 0x7fb41815caf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb41000c430 tx=0x7fb41000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:42.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.584+0000 7fb40f400640 1 -- 172.21.15.52:0/1638359851 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb410005ce0 con 0x7fb4181610a0 2024-03-12T17:06:42.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.584+0000 7fb40f400640 1 -- 172.21.15.52:0/1638359851 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb410005e40 con 0x7fb4181610a0 2024-03-12T17:06:42.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.584+0000 7fb40f400640 1 -- 172.21.15.52:0/1638359851 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb410007310 con 0x7fb4181610a0 2024-03-12T17:06:42.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.584+0000 7fb41d000640 1 -- 172.21.15.52:0/1638359851 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4180b3be0 con 0x7fb4181610a0 2024-03-12T17:06:42.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.584+0000 7fb41d000640 1 -- 172.21.15.52:0/1638359851 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4180b41a0 con 0x7fb4181610a0 2024-03-12T17:06:42.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.586+0000 7fb41d000640 1 -- 172.21.15.52:0/1638359851 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4180c8a80 con 0x7fb4181610a0 2024-03-12T17:06:42.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.586+0000 7fb40f400640 1 -- 172.21.15.52:0/1638359851 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb410007470 con 0x7fb4181610a0 2024-03-12T17:06:42.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.587+0000 7fb40f400640 1 --2- 172.21.15.52:0/1638359851 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb3f406ac30 0x7fb3f406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:42.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.587+0000 7fb40f400640 1 -- 172.21.15.52:0/1638359851 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb410088cd0 con 0x7fb4181610a0 2024-03-12T17:06:42.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.588+0000 7fb417400640 1 --2- 172.21.15.52:0/1638359851 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb3f406ac30 0x7fb3f406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:42.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.589+0000 7fb417400640 1 --2- 172.21.15.52:0/1638359851 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb3f406ac30 0x7fb3f406d0f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb4080012e0 tx=0x7fb408009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:42.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.592+0000 7fb40f400640 1 -- 172.21.15.52:0/1638359851 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fb410053bb0 con 0x7fb4181610a0 2024-03-12T17:06:42.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.792+0000 7fb41d000640 1 -- 172.21.15.52:0/1638359851 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb41804e030 con 0x7fb4181610a0 2024-03-12T17:06:42.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.793+0000 7fb40f400640 1 -- 172.21.15.52:0/1638359851 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb4100170a0 con 0x7fb4181610a0 2024-03-12T17:06:42.786 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-03-12T17:06:42.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.795+0000 7fb41d000640 1 -- 172.21.15.52:0/1638359851 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb3f406ac30 msgr2=0x7fb3f406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:42.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.795+0000 7fb41d000640 1 --2- 172.21.15.52:0/1638359851 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb3f406ac30 0x7fb3f406d0f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb4080012e0 tx=0x7fb408009040 comp rx=0 tx=0).stop 2024-03-12T17:06:42.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.795+0000 7fb41d000640 1 -- 172.21.15.52:0/1638359851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4181610a0 msgr2=0x7fb41815caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:42.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.795+0000 7fb41d000640 1 --2- 172.21.15.52:0/1638359851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4181610a0 0x7fb41815caf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb41000c430 tx=0x7fb41000c8f0 comp rx=0 tx=0).stop 2024-03-12T17:06:42.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.796+0000 7fb41d000640 1 -- 172.21.15.52:0/1638359851 shutdown_connections 2024-03-12T17:06:42.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.796+0000 7fb41d000640 1 --2- 172.21.15.52:0/1638359851 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb3f406ac30 0x7fb3f406d0f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:42.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.796+0000 7fb41d000640 1 --2- 172.21.15.52:0/1638359851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4181610a0 0x7fb41815caf0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:42.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.796+0000 7fb41d000640 1 --2- 172.21.15.52:0/1638359851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4180af5e0 0x7fb4180b3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:42.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.796+0000 7fb41d000640 1 --2- 172.21.15.52:0/1638359851 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb4180995c0 0x7fb4180b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:42.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.796+0000 7fb41d000640 1 -- 172.21.15.52:0/1638359851 >> 172.21.15.52:0/1638359851 conn(0x7fb4180b2010 msgr2=0x7fb4180c34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:42.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.796+0000 7fb41d000640 1 -- 172.21.15.52:0/1638359851 shutdown_connections 2024-03-12T17:06:42.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:42.796+0000 7fb41d000640 1 -- 172.21.15.52:0/1638359851 wait complete. 2024-03-12T17:06:42.806 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-12T17:06:43.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.038+0000 7fd8aca00640 1 Processor -- start 2024-03-12T17:06:43.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.038+0000 7fd8aca00640 1 -- start start 2024-03-12T17:06:43.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.040+0000 7fd8aca00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd8a80cdc00 0x7fd8a80b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:43.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.040+0000 7fd8aca00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8a80cae50 0x7fd8a80cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:43.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.040+0000 7fd8aca00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8a81575b0 con 0x7fd8a81571d0 2024-03-12T17:06:43.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.040+0000 7fd8aca00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8a80cb820 con 0x7fd8a80cdc00 2024-03-12T17:06:43.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.040+0000 7fd8aca00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a8159f20 con 0x7fd8a80cae50 2024-03-12T17:06:43.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.040+0000 7fd8a6800640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fd8a81571d0 0x7fd8a815aaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:41018/0 (socket says 172.21.15.52:41018) 2024-03-12T17:06:43.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.040+0000 7fd8a6800640 1 -- 172.21.15.52:0/2832153706 learned_addr learned my addr 172.21.15.52:0/2832153706 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:06:43.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.041+0000 7fd8a5e00640 1 --2- 172.21.15.52:0/2832153706 >> v2:172.21.15.102:3300/0 conn(0x7fd8a80cdc00 0x7fd8a80b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:43.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.041+0000 7fd8a7200640 1 --2- 172.21.15.52:0/2832153706 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8a80cae50 0x7fd8a80cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:43.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.041+0000 7fd8a5400640 1 -- 172.21.15.52:0/2832153706 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1233392281 0 0) 0x7fd8a81575b0 con 0x7fd8a81571d0 2024-03-12T17:06:43.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.041+0000 7fd8a5400640 1 -- 172.21.15.52:0/2832153706 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd88c003520 con 0x7fd8a81571d0 2024-03-12T17:06:43.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.041+0000 7fd8a5400640 1 -- 172.21.15.52:0/2832153706 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4205806335 0 0) 0x7fd88c003520 con 0x7fd8a81571d0 2024-03-12T17:06:43.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.041+0000 7fd8a5400640 1 -- 172.21.15.52:0/2832153706 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8a80cae50 msgr2=0x7fd8a80cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:43.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.041+0000 7fd8a5400640 1 --2- 172.21.15.52:0/2832153706 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8a80cae50 0x7fd8a80cb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:43.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.041+0000 7fd8a5400640 1 -- 172.21.15.52:0/2832153706 >> v2:172.21.15.102:3300/0 conn(0x7fd8a80cdc00 msgr2=0x7fd8a80b35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:43.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.041+0000 7fd8a5400640 1 --2- 172.21.15.52:0/2832153706 >> v2:172.21.15.102:3300/0 conn(0x7fd8a80cdc00 0x7fd8a80b35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:43.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.041+0000 7fd8a5400640 1 -- 172.21.15.52:0/2832153706 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a815a200 con 0x7fd8a81571d0 2024-03-12T17:06:43.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.041+0000 7fd8a5400640 1 -- 172.21.15.52:0/2832153706 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7fd890001670 con 0x7fd8a81571d0 2024-03-12T17:06:43.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.041+0000 7fd8a5e00640 1 --2- 172.21.15.52:0/2832153706 >> v2:172.21.15.102:3300/0 conn(0x7fd8a80cdc00 0x7fd8a80b35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:06:43.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.041+0000 7fd8a5400640 1 -- 172.21.15.52:0/2832153706 >> v1:172.21.15.52:6789/0 conn(0x7fd8a81571d0 legacy=0x7fd8a815aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:43.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.041+0000 7fd8a5400640 1 --2- 172.21.15.52:0/2832153706 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8a80cae50 0x7fd88c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:43.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.041+0000 7fd8a5400640 1 --2- 172.21.15.52:0/2832153706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd88c003f40 0x7fd88c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:43.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.041+0000 7fd8a5400640 1 --2- 172.21.15.52:0/2832153706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd88c004d00 0x7fd88c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:43.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.041+0000 7fd8a5400640 1 -- 172.21.15.52:0/2832153706 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a80cb820 con 0x7fd88c004d00 2024-03-12T17:06:43.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.041+0000 7fd8a5400640 1 -- 172.21.15.52:0/2832153706 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a8159f20 con 0x7fd88c003f40 2024-03-12T17:06:43.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.041+0000 7fd8a5400640 1 -- 172.21.15.52:0/2832153706 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c00dc40 con 0x7fd8a80cae50 2024-03-12T17:06:43.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.042+0000 7fd8a7200640 1 --2- 172.21.15.52:0/2832153706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd88c004d00 0x7fd88c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:43.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.042+0000 7fd8a6800640 1 --2- 172.21.15.52:0/2832153706 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8a80cae50 0x7fd88c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:43.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.042+0000 7fd8a5e00640 1 --2- 172.21.15.52:0/2832153706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd88c003f40 0x7fd88c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:43.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.042+0000 7fd8a7200640 1 -- 172.21.15.52:0/2832153706 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8a80cae50 msgr2=0x7fd88c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:43.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.042+0000 7fd8a7200640 1 --2- 172.21.15.52:0/2832153706 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8a80cae50 0x7fd88c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:43.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.042+0000 7fd8a7200640 1 -- 172.21.15.52:0/2832153706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd88c003f40 msgr2=0x7fd88c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:43.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.042+0000 7fd8a7200640 1 --2- 172.21.15.52:0/2832153706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd88c003f40 0x7fd88c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:43.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.042+0000 7fd8a7200640 1 -- 172.21.15.52:0/2832153706 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd88c003520 con 0x7fd88c004d00 2024-03-12T17:06:43.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.042+0000 7fd8a5e00640 1 --2- 172.21.15.52:0/2832153706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd88c003f40 0x7fd88c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:43.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.043+0000 7fd8a7200640 1 --2- 172.21.15.52:0/2832153706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd88c004d00 0x7fd88c005180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0009fc0 tx=0x7fd88c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238711c7e324e1e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:43.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.043+0000 7fd8a5400640 1 -- 172.21.15.52:0/2832153706 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a002cc20 con 0x7fd88c004d00 2024-03-12T17:06:43.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.043+0000 7fd8a5400640 1 -- 172.21.15.52:0/2832153706 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a000f040 con 0x7fd88c004d00 2024-03-12T17:06:43.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.043+0000 7fd8a5400640 1 -- 172.21.15.52:0/2832153706 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0034440 con 0x7fd88c004d00 2024-03-12T17:06:43.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.044+0000 7fd8aca00640 1 -- 172.21.15.52:0/2832153706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd88c004d00 msgr2=0x7fd88c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:43.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.044+0000 7fd8aca00640 1 --2- 172.21.15.52:0/2832153706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd88c004d00 0x7fd88c005180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0009fc0 tx=0x7fd88c0057f0 comp rx=0 tx=0).stop 2024-03-12T17:06:43.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.044+0000 7fd8a7200640 1 -- 172.21.15.52:0/2832153706 reap_dead start 2024-03-12T17:06:43.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.044+0000 7fd8aca00640 1 -- 172.21.15.52:0/2832153706 shutdown_connections 2024-03-12T17:06:43.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.044+0000 7fd8aca00640 1 -- 172.21.15.52:0/2832153706 >> 172.21.15.52:0/2832153706 conn(0x7fd8a8065df0 msgr2=0x7fd8a8069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:43.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.044+0000 7fd8aca00640 1 -- 172.21.15.52:0/2832153706 shutdown_connections 2024-03-12T17:06:43.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.044+0000 7fd8aca00640 1 -- 172.21.15.52:0/2832153706 wait complete. 2024-03-12T17:06:43.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.046+0000 7fd8aca00640 1 Processor -- start 2024-03-12T17:06:43.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.046+0000 7fd8aca00640 1 -- start start 2024-03-12T17:06:43.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.047+0000 7fd8aca00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd88c004d00 0x7fd8a813f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:43.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.047+0000 7fd8aca00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8a81585f0 0x7fd8a8158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:43.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.047+0000 7fd8aca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8a806c080 0x7fd8a806c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:43.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.047+0000 7fd8aca00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a8158f90 con 0x7fd88c004d00 2024-03-12T17:06:43.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.047+0000 7fd8aca00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a81e4c00 con 0x7fd8a806c080 2024-03-12T17:06:43.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.047+0000 7fd8aca00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a8142570 con 0x7fd8a81585f0 2024-03-12T17:06:43.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.047+0000 7fd8a6800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd88c004d00 0x7fd8a813f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:43.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.047+0000 7fd8a6800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd88c004d00 0x7fd8a813f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46728/0 (socket says 172.21.15.52:46728) 2024-03-12T17:06:43.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.047+0000 7fd8a6800640 1 -- 172.21.15.52:0/4186547076 learned_addr learned my addr 172.21.15.52:0/4186547076 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:43.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.048+0000 7fd8a5e00640 1 --2- 172.21.15.52:0/4186547076 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8a81585f0 0x7fd8a8158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:43.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.048+0000 7fd8a7200640 1 --2- 172.21.15.52:0/4186547076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8a806c080 0x7fd8a806c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:43.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.048+0000 7fd8a6800640 1 -- 172.21.15.52:0/4186547076 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8a81585f0 msgr2=0x7fd8a8158a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:43.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.048+0000 7fd8a6800640 1 --2- 172.21.15.52:0/4186547076 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8a81585f0 0x7fd8a8158a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:43.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.048+0000 7fd8a6800640 1 -- 172.21.15.52:0/4186547076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8a806c080 msgr2=0x7fd8a806c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:43.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.048+0000 7fd8a6800640 1 --2- 172.21.15.52:0/4186547076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8a806c080 0x7fd8a806c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:43.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.048+0000 7fd8a6800640 1 -- 172.21.15.52:0/4186547076 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a81e3ff0 con 0x7fd88c004d00 2024-03-12T17:06:43.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.048+0000 7fd8a6800640 1 --2- 172.21.15.52:0/4186547076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd88c004d00 0x7fd8a813f3d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd890000fb0 tx=0x7fd89000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:43.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.048+0000 7fd89ea00640 1 -- 172.21.15.52:0/4186547076 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89000c840 con 0x7fd88c004d00 2024-03-12T17:06:43.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.048+0000 7fd89ea00640 1 -- 172.21.15.52:0/4186547076 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd89000c9a0 con 0x7fd88c004d00 2024-03-12T17:06:43.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.048+0000 7fd89ea00640 1 -- 172.21.15.52:0/4186547076 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89000cd30 con 0x7fd88c004d00 2024-03-12T17:06:43.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.048+0000 7fd8aca00640 1 -- 172.21.15.52:0/4186547076 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a81e42d0 con 0x7fd88c004d00 2024-03-12T17:06:43.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.049+0000 7fd8aca00640 1 -- 172.21.15.52:0/4186547076 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a81e4890 con 0x7fd88c004d00 2024-03-12T17:06:43.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.050+0000 7fd8aca00640 1 -- 172.21.15.52:0/4186547076 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8a81e4d90 con 0x7fd88c004d00 2024-03-12T17:06:43.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.051+0000 7fd89ea00640 1 -- 172.21.15.52:0/4186547076 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd89001d460 con 0x7fd88c004d00 2024-03-12T17:06:43.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.051+0000 7fd89ea00640 1 --2- 172.21.15.52:0/4186547076 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd88406ac00 0x7fd88406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:43.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.052+0000 7fd89ea00640 1 -- 172.21.15.52:0/4186547076 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd890089750 con 0x7fd88c004d00 2024-03-12T17:06:43.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.052+0000 7fd8a5e00640 1 --2- 172.21.15.52:0/4186547076 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd88406ac00 0x7fd88406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:43.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.053+0000 7fd8a5e00640 1 --2- 172.21.15.52:0/4186547076 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd88406ac00 0x7fd88406d0c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8158580 tx=0x7fd898008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:43.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.056+0000 7fd89ea00640 1 -- 172.21.15.52:0/4186547076 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fd890054510 con 0x7fd88c004d00 2024-03-12T17:06:43.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.253+0000 7fd8aca00640 1 -- 172.21.15.52:0/4186547076 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd8a80b26c0 con 0x7fd88c004d00 2024-03-12T17:06:43.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.254+0000 7fd89ea00640 1 -- 172.21.15.52:0/4186547076 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd890057b30 con 0x7fd88c004d00 2024-03-12T17:06:43.246 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-12T17:06:43.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.256+0000 7fd8aca00640 1 -- 172.21.15.52:0/4186547076 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd88406ac00 msgr2=0x7fd88406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:43.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.256+0000 7fd8aca00640 1 --2- 172.21.15.52:0/4186547076 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd88406ac00 0x7fd88406d0c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8158580 tx=0x7fd898008040 comp rx=0 tx=0).stop 2024-03-12T17:06:43.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.256+0000 7fd8aca00640 1 -- 172.21.15.52:0/4186547076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd88c004d00 msgr2=0x7fd8a813f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:43.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.256+0000 7fd8aca00640 1 --2- 172.21.15.52:0/4186547076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd88c004d00 0x7fd8a813f3d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd890000fb0 tx=0x7fd89000ec60 comp rx=0 tx=0).stop 2024-03-12T17:06:43.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.256+0000 7fd8aca00640 1 -- 172.21.15.52:0/4186547076 shutdown_connections 2024-03-12T17:06:43.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.257+0000 7fd8aca00640 1 --2- 172.21.15.52:0/4186547076 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd88406ac00 0x7fd88406d0c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:43.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.257+0000 7fd8aca00640 1 --2- 172.21.15.52:0/4186547076 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd8a81585f0 0x7fd8a8158a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:43.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.257+0000 7fd8aca00640 1 --2- 172.21.15.52:0/4186547076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8a806c080 0x7fd8a806c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:43.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.257+0000 7fd8aca00640 1 --2- 172.21.15.52:0/4186547076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd88c004d00 0x7fd8a813f3d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:43.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.257+0000 7fd8aca00640 1 -- 172.21.15.52:0/4186547076 >> 172.21.15.52:0/4186547076 conn(0x7fd8a8065df0 msgr2=0x7fd8a815d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:43.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.257+0000 7fd8aca00640 1 -- 172.21.15.52:0/4186547076 shutdown_connections 2024-03-12T17:06:43.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.257+0000 7fd8aca00640 1 -- 172.21.15.52:0/4186547076 wait complete. 2024-03-12T17:06:43.268 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-12T17:06:43.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.501+0000 7f96e7200640 1 Processor -- start 2024-03-12T17:06:43.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.501+0000 7f96e7200640 1 -- start start 2024-03-12T17:06:43.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.503+0000 7f96e7200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96e8159cd0 0x7f96e80ad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:43.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.503+0000 7f96e7200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f96e80cc200 0x7f96e80cab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:43.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.503+0000 7f96e7200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96e80c8c90 con 0x7f96e80ad7a0 2024-03-12T17:06:43.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.503+0000 7f96e7200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96e80cc6c0 con 0x7f96e80cc200 2024-03-12T17:06:43.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.503+0000 7f96e7200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96e80cc9a0 con 0x7f96e8159cd0 2024-03-12T17:06:43.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.503+0000 7f96dfe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f96e80cc200 0x7f96e80cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:43.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.504+0000 7f96dfe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f96e80cc200 0x7f96e80cab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46740/0 (socket says 172.21.15.52:46740) 2024-03-12T17:06:43.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.504+0000 7f96dfe00640 1 -- 172.21.15.52:0/2044639910 learned_addr learned my addr 172.21.15.52:0/2044639910 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:43.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.504+0000 7f96e5e00640 1 --2- 172.21.15.52:0/2044639910 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96e8159cd0 0x7f96e80ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:43.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.504+0000 7f96dfe00640 1 -- 172.21.15.52:0/2044639910 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96e8159cd0 msgr2=0x7f96e80ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:43.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.504+0000 7f96dfe00640 1 --2- 172.21.15.52:0/2044639910 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96e8159cd0 0x7f96e80ad260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:43.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.504+0000 7f96dfe00640 1 -- 172.21.15.52:0/2044639910 >> v1:172.21.15.102:6789/0 conn(0x7f96e80ad7a0 legacy=0x7f96e815a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:43.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.504+0000 7f96dfe00640 1 -- 172.21.15.52:0/2044639910 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e80ccc80 con 0x7f96e80cc200 2024-03-12T17:06:43.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.505+0000 7f96dfe00640 1 --2- 172.21.15.52:0/2044639910 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e80cc200 0x7f96e80cab30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f96d0009870 tx=0x7f96d002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6738757dbe467a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:43.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.506+0000 7f96e5400640 1 -- 172.21.15.52:0/2044639910 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d0030840 con 0x7f96e80cc200 2024-03-12T17:06:43.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.506+0000 7f96e5400640 1 -- 172.21.15.52:0/2044639910 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d00309a0 con 0x7f96e80cc200 2024-03-12T17:06:43.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.506+0000 7f96e5400640 1 -- 172.21.15.52:0/2044639910 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d0030d30 con 0x7f96e80cc200 2024-03-12T17:06:43.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.506+0000 7f96e7200640 1 -- 172.21.15.52:0/2044639910 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e80cc200 msgr2=0x7f96e80cab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:43.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.506+0000 7f96e7200640 1 --2- 172.21.15.52:0/2044639910 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e80cc200 0x7f96e80cab30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f96d0009870 tx=0x7f96d002b980 comp rx=0 tx=0).stop 2024-03-12T17:06:43.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.506+0000 7f96e7200640 1 -- 172.21.15.52:0/2044639910 shutdown_connections 2024-03-12T17:06:43.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.506+0000 7f96e7200640 1 --2- 172.21.15.52:0/2044639910 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e80cc200 0x7f96e80cab30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:43.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.507+0000 7f96e7200640 1 --2- 172.21.15.52:0/2044639910 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96e8159cd0 0x7f96e80ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:43.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.507+0000 7f96e7200640 1 -- 172.21.15.52:0/2044639910 >> 172.21.15.52:0/2044639910 conn(0x7f96e80b01b0 msgr2=0x7f96e80b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:43.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.507+0000 7f96e7200640 1 -- 172.21.15.52:0/2044639910 shutdown_connections 2024-03-12T17:06:43.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.507+0000 7f96e7200640 1 -- 172.21.15.52:0/2044639910 wait complete. 2024-03-12T17:06:43.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.508+0000 7f96e7200640 1 Processor -- start 2024-03-12T17:06:43.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.509+0000 7f96e7200640 1 -- start start 2024-03-12T17:06:43.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.509+0000 7f96e7200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96e80ad7a0 0x7f96e8097350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:43.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.510+0000 7f96e7200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96e80cc200 0x7f96e8150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:43.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.510+0000 7f96e7200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e8159cd0 0x7f96e806d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:43.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.510+0000 7f96e7200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e809c160 con 0x7f96e8159cd0 2024-03-12T17:06:43.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.510+0000 7f96e7200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e8150b70 con 0x7f96e80ad7a0 2024-03-12T17:06:43.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.510+0000 7f96e7200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96e815c400 con 0x7f96e80cc200 2024-03-12T17:06:43.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.510+0000 7f96e6800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e8159cd0 0x7f96e806d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:43.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.510+0000 7f96e6800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e8159cd0 0x7f96e806d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46750/0 (socket says 172.21.15.52:46750) 2024-03-12T17:06:43.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.510+0000 7f96e6800640 1 -- 172.21.15.52:0/2984785882 learned_addr learned my addr 172.21.15.52:0/2984785882 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:43.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.510+0000 7f96dfe00640 1 --2- 172.21.15.52:0/2984785882 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96e80cc200 0x7f96e8150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:43.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.510+0000 7f96e5e00640 1 --2- 172.21.15.52:0/2984785882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96e80ad7a0 0x7f96e8097350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:43.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.510+0000 7f96e6800640 1 -- 172.21.15.52:0/2984785882 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96e80cc200 msgr2=0x7f96e8150630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:43.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.510+0000 7f96e6800640 1 --2- 172.21.15.52:0/2984785882 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96e80cc200 0x7f96e8150630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:43.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.510+0000 7f96e6800640 1 -- 172.21.15.52:0/2984785882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96e80ad7a0 msgr2=0x7f96e8097350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:43.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.510+0000 7f96e6800640 1 --2- 172.21.15.52:0/2984785882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96e80ad7a0 0x7f96e8097350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:43.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.510+0000 7f96e6800640 1 -- 172.21.15.52:0/2984785882 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d00094d0 con 0x7f96e8159cd0 2024-03-12T17:06:43.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.511+0000 7f96e6800640 1 --2- 172.21.15.52:0/2984785882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e8159cd0 0x7f96e806d090 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f96e000c7a0 tx=0x7f96e000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:43.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.511+0000 7f96de000640 1 -- 172.21.15.52:0/2984785882 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e0003b70 con 0x7f96e8159cd0 2024-03-12T17:06:43.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.511+0000 7f96de000640 1 -- 172.21.15.52:0/2984785882 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e0003cf0 con 0x7f96e8159cd0 2024-03-12T17:06:43.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.511+0000 7f96de000640 1 -- 172.21.15.52:0/2984785882 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e0003ea0 con 0x7f96e8159cd0 2024-03-12T17:06:43.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.511+0000 7f96e7200640 1 -- 172.21.15.52:0/2984785882 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e815abe0 con 0x7f96e8159cd0 2024-03-12T17:06:43.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.511+0000 7f96e7200640 1 -- 172.21.15.52:0/2984785882 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e809bd80 con 0x7f96e8159cd0 2024-03-12T17:06:43.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.513+0000 7f96e7200640 1 -- 172.21.15.52:0/2984785882 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96e809c2a0 con 0x7f96e8159cd0 2024-03-12T17:06:43.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.514+0000 7f96de000640 1 -- 172.21.15.52:0/2984785882 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f96e000aa90 con 0x7f96e8159cd0 2024-03-12T17:06:43.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.515+0000 7f96de000640 1 --2- 172.21.15.52:0/2984785882 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f96c406ac00 0x7f96c406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:43.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.515+0000 7f96de000640 1 -- 172.21.15.52:0/2984785882 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f96e0089560 con 0x7f96e8159cd0 2024-03-12T17:06:43.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.515+0000 7f96e5e00640 1 --2- 172.21.15.52:0/2984785882 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f96c406ac00 0x7f96c406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:43.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.516+0000 7f96e5e00640 1 --2- 172.21.15.52:0/2984785882 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f96c406ac00 0x7f96c406d0c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f96d8006fd0 tx=0x7f96d8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:43.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.519+0000 7f96de000640 1 -- 172.21.15.52:0/2984785882 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f96e809c2a0 con 0x7f96e8159cd0 2024-03-12T17:06:43.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.715+0000 7f96e7200640 1 -- 172.21.15.52:0/2984785882 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f96e809c2a0 con 0x7f96e8159cd0 2024-03-12T17:06:43.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.716+0000 7f96de000640 1 -- 172.21.15.52:0/2984785882 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f96e809c2a0 con 0x7f96e8159cd0 2024-03-12T17:06:43.709 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-03-12T17:06:43.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.719+0000 7f96e7200640 1 -- 172.21.15.52:0/2984785882 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f96c406ac00 msgr2=0x7f96c406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:43.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.719+0000 7f96e7200640 1 --2- 172.21.15.52:0/2984785882 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f96c406ac00 0x7f96c406d0c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f96d8006fd0 tx=0x7f96d8008040 comp rx=0 tx=0).stop 2024-03-12T17:06:43.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.719+0000 7f96e7200640 1 -- 172.21.15.52:0/2984785882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e8159cd0 msgr2=0x7f96e806d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:43.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.719+0000 7f96e7200640 1 --2- 172.21.15.52:0/2984785882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e8159cd0 0x7f96e806d090 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f96e000c7a0 tx=0x7f96e000cc60 comp rx=0 tx=0).stop 2024-03-12T17:06:43.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.719+0000 7f96e7200640 1 -- 172.21.15.52:0/2984785882 shutdown_connections 2024-03-12T17:06:43.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.719+0000 7f96e7200640 1 --2- 172.21.15.52:0/2984785882 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f96c406ac00 0x7f96c406d0c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:43.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.719+0000 7f96e7200640 1 --2- 172.21.15.52:0/2984785882 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96e80cc200 0x7f96e8150630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:43.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.719+0000 7f96e7200640 1 --2- 172.21.15.52:0/2984785882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e8159cd0 0x7f96e806d090 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:43.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.719+0000 7f96e7200640 1 --2- 172.21.15.52:0/2984785882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96e80ad7a0 0x7f96e8097350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:43.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.719+0000 7f96e7200640 1 -- 172.21.15.52:0/2984785882 >> 172.21.15.52:0/2984785882 conn(0x7f96e80b01b0 msgr2=0x7f96e815b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:43.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.719+0000 7f96e7200640 1 -- 172.21.15.52:0/2984785882 shutdown_connections 2024-03-12T17:06:43.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.719+0000 7f96e7200640 1 -- 172.21.15.52:0/2984785882 wait complete. 2024-03-12T17:06:43.731 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-12T17:06:43.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.964+0000 7f3f8ec00640 1 Processor -- start 2024-03-12T17:06:43.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.965+0000 7f3f8ec00640 1 -- start start 2024-03-12T17:06:43.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.966+0000 7f3f8ec00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3f880cf5c0 0x7f3f880b8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:43.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.966+0000 7f3f8ec00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f3f880d0c20 0x7f3f880d1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:43.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.966+0000 7f3f8ec00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f8815ce90 con 0x7f3f8815cab0 2024-03-12T17:06:43.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.966+0000 7f3f8ec00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f880cddf0 con 0x7f3f880d0c20 2024-03-12T17:06:43.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.966+0000 7f3f8ec00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f880cd570 con 0x7f3f880cf5c0 2024-03-12T17:06:43.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.967+0000 7f3f8e200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f3f880d0c20 0x7f3f880d1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:43.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.967+0000 7f3f8d800640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f3f8815cab0 0x7f3f880ceeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:41020/0 (socket says 172.21.15.52:41020) 2024-03-12T17:06:43.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.967+0000 7f3f8d800640 1 -- 172.21.15.52:0/3562303993 learned_addr learned my addr 172.21.15.52:0/3562303993 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:06:43.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.967+0000 7f3f8ce00640 1 --2- 172.21.15.52:0/3562303993 >> v2:172.21.15.102:3300/0 conn(0x7f3f880cf5c0 0x7f3f880b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:43.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.967+0000 7f3f8e200640 1 -- 172.21.15.52:0/3562303993 >> v2:172.21.15.102:3300/0 conn(0x7f3f880cf5c0 msgr2=0x7f3f880b8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:43.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.967+0000 7f3f8e200640 1 --2- 172.21.15.52:0/3562303993 >> v2:172.21.15.102:3300/0 conn(0x7f3f880cf5c0 0x7f3f880b8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:43.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.967+0000 7f3f8e200640 1 -- 172.21.15.52:0/3562303993 >> v1:172.21.15.52:6789/0 conn(0x7f3f8815cab0 legacy=0x7f3f880ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:43.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.967+0000 7f3f8e200640 1 -- 172.21.15.52:0/3562303993 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f88161370 con 0x7f3f880d0c20 2024-03-12T17:06:43.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.967+0000 7f3f87e00640 1 -- 172.21.15.52:0/3562303993 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4061376680 0 0) 0x7f3f8815ce90 con 0x7f3f8815cab0 2024-03-12T17:06:43.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.968+0000 7f3f8e200640 1 --2- 172.21.15.52:0/3562303993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f880d0c20 0x7f3f880d1070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3f880d3750 tx=0x7f3f8002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6897f48b7e11c65b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:43.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.969+0000 7f3f87e00640 1 -- 172.21.15.52:0/3562303993 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f80031830 con 0x7f3f880d0c20 2024-03-12T17:06:43.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.969+0000 7f3f87e00640 1 -- 172.21.15.52:0/3562303993 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f80031990 con 0x7f3f880d0c20 2024-03-12T17:06:43.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.969+0000 7f3f87e00640 1 -- 172.21.15.52:0/3562303993 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f80031d20 con 0x7f3f880d0c20 2024-03-12T17:06:43.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.969+0000 7f3f8ec00640 1 -- 172.21.15.52:0/3562303993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f880d0c20 msgr2=0x7f3f880d1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:43.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.969+0000 7f3f8ec00640 1 --2- 172.21.15.52:0/3562303993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f880d0c20 0x7f3f880d1070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3f880d3750 tx=0x7f3f8002b8d0 comp rx=0 tx=0).stop 2024-03-12T17:06:43.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.970+0000 7f3f8ec00640 1 -- 172.21.15.52:0/3562303993 shutdown_connections 2024-03-12T17:06:43.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.970+0000 7f3f8ec00640 1 --2- 172.21.15.52:0/3562303993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f880d0c20 0x7f3f880d1070 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:43.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.970+0000 7f3f8ec00640 1 --2- 172.21.15.52:0/3562303993 >> v2:172.21.15.102:3300/0 conn(0x7f3f880cf5c0 0x7f3f880b8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:43.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.970+0000 7f3f8ec00640 1 -- 172.21.15.52:0/3562303993 >> 172.21.15.52:0/3562303993 conn(0x7f3f8806ae90 msgr2=0x7f3f8806b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:43.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.970+0000 7f3f8ec00640 1 -- 172.21.15.52:0/3562303993 shutdown_connections 2024-03-12T17:06:43.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.970+0000 7f3f8ec00640 1 -- 172.21.15.52:0/3562303993 wait complete. 2024-03-12T17:06:43.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.971+0000 7f3f8ec00640 1 Processor -- start 2024-03-12T17:06:43.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.972+0000 7f3f8ec00640 1 -- start start 2024-03-12T17:06:43.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.973+0000 7f3f8ec00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3f880cf5c0 0x7f3f881548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:43.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.973+0000 7f3f8ec00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f880d0c20 0x7f3f8815dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:43.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.973+0000 7f3f8ec00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f8815cab0 0x7f3f88156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:43.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.973+0000 7f3f8ec00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8815f210 con 0x7f3f880d0c20 2024-03-12T17:06:43.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.973+0000 7f3f8ec00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f88154e00 con 0x7f3f8815cab0 2024-03-12T17:06:43.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.973+0000 7f3f8ec00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8815e600 con 0x7f3f880cf5c0 2024-03-12T17:06:43.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.973+0000 7f3f8ce00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f880d0c20 0x7f3f8815dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:43.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.973+0000 7f3f8ce00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f880d0c20 0x7f3f8815dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46764/0 (socket says 172.21.15.52:46764) 2024-03-12T17:06:43.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.973+0000 7f3f8ce00640 1 -- 172.21.15.52:0/2406475382 learned_addr learned my addr 172.21.15.52:0/2406475382 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:43.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.973+0000 7f3f8e200640 1 --2- 172.21.15.52:0/2406475382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f8815cab0 0x7f3f88156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:43.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.973+0000 7f3f8d800640 1 --2- 172.21.15.52:0/2406475382 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3f880cf5c0 0x7f3f881548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:43.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.973+0000 7f3f8ce00640 1 -- 172.21.15.52:0/2406475382 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3f880cf5c0 msgr2=0x7f3f881548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:43.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.973+0000 7f3f8ce00640 1 --2- 172.21.15.52:0/2406475382 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3f880cf5c0 0x7f3f881548c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:43.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.973+0000 7f3f8ce00640 1 -- 172.21.15.52:0/2406475382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f8815cab0 msgr2=0x7f3f88156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:43.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.973+0000 7f3f8ce00640 1 --2- 172.21.15.52:0/2406475382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f8815cab0 0x7f3f88156790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:43.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.973+0000 7f3f8ce00640 1 -- 172.21.15.52:0/2406475382 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f80009400 con 0x7f3f880d0c20 2024-03-12T17:06:43.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.973+0000 7f3f8e200640 1 --2- 172.21.15.52:0/2406475382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f8815cab0 0x7f3f88156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:43.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.974+0000 7f3f8ce00640 1 --2- 172.21.15.52:0/2406475382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f880d0c20 0x7f3f8815dc20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3f78007ef0 tx=0x7f3f7800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:43.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.974+0000 7f3f85600640 1 -- 172.21.15.52:0/2406475382 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f78014620 con 0x7f3f880d0c20 2024-03-12T17:06:43.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.974+0000 7f3f85600640 1 -- 172.21.15.52:0/2406475382 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f78014780 con 0x7f3f880d0c20 2024-03-12T17:06:43.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.974+0000 7f3f85600640 1 -- 172.21.15.52:0/2406475382 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f78014b30 con 0x7f3f880d0c20 2024-03-12T17:06:43.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.974+0000 7f3f8ec00640 1 -- 172.21.15.52:0/2406475382 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f8815e8e0 con 0x7f3f880d0c20 2024-03-12T17:06:43.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.974+0000 7f3f8ec00640 1 -- 172.21.15.52:0/2406475382 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f8815eea0 con 0x7f3f880d0c20 2024-03-12T17:06:43.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.975+0000 7f3f8ec00640 1 -- 172.21.15.52:0/2406475382 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f8815f380 con 0x7f3f880d0c20 2024-03-12T17:06:43.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.976+0000 7f3f85600640 1 -- 172.21.15.52:0/2406475382 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3f78014c90 con 0x7f3f880d0c20 2024-03-12T17:06:43.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.977+0000 7f3f85600640 1 --2- 172.21.15.52:0/2406475382 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f3f6c06ac00 0x7f3f6c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:43.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.977+0000 7f3f85600640 1 -- 172.21.15.52:0/2406475382 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3f7808a0b0 con 0x7f3f880d0c20 2024-03-12T17:06:43.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.978+0000 7f3f8d800640 1 --2- 172.21.15.52:0/2406475382 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f3f6c06ac00 0x7f3f6c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:43.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.979+0000 7f3f8d800640 1 --2- 172.21.15.52:0/2406475382 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f3f6c06ac00 0x7f3f6c06d0c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c000f60 tx=0x7f3f7c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:43.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:43.982+0000 7f3f85600640 1 -- 172.21.15.52:0/2406475382 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f3f78054f90 con 0x7f3f880d0c20 2024-03-12T17:06:44.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.181+0000 7f3f8ec00640 1 -- 172.21.15.52:0/2406475382 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3f8804e030 con 0x7f3f880d0c20 2024-03-12T17:06:44.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.182+0000 7f3f85600640 1 -- 172.21.15.52:0/2406475382 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3f780585b0 con 0x7f3f880d0c20 2024-03-12T17:06:44.174 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-12T17:06:44.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.185+0000 7f3f8ec00640 1 -- 172.21.15.52:0/2406475382 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f3f6c06ac00 msgr2=0x7f3f6c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:44.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.185+0000 7f3f8ec00640 1 --2- 172.21.15.52:0/2406475382 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f3f6c06ac00 0x7f3f6c06d0c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c000f60 tx=0x7f3f7c009040 comp rx=0 tx=0).stop 2024-03-12T17:06:44.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.185+0000 7f3f8ec00640 1 -- 172.21.15.52:0/2406475382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f880d0c20 msgr2=0x7f3f8815dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:44.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.185+0000 7f3f8ec00640 1 --2- 172.21.15.52:0/2406475382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f880d0c20 0x7f3f8815dc20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3f78007ef0 tx=0x7f3f7800e4b0 comp rx=0 tx=0).stop 2024-03-12T17:06:44.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.185+0000 7f3f8ec00640 1 -- 172.21.15.52:0/2406475382 shutdown_connections 2024-03-12T17:06:44.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.185+0000 7f3f8ec00640 1 --2- 172.21.15.52:0/2406475382 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f3f6c06ac00 0x7f3f6c06d0c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:44.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.185+0000 7f3f8ec00640 1 --2- 172.21.15.52:0/2406475382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f8815cab0 0x7f3f88156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:44.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.185+0000 7f3f8ec00640 1 --2- 172.21.15.52:0/2406475382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f880d0c20 0x7f3f8815dc20 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:44.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.185+0000 7f3f8ec00640 1 --2- 172.21.15.52:0/2406475382 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3f880cf5c0 0x7f3f881548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:44.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.185+0000 7f3f8ec00640 1 -- 172.21.15.52:0/2406475382 >> 172.21.15.52:0/2406475382 conn(0x7f3f8806ae90 msgr2=0x7f3f8806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:44.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.185+0000 7f3f8ec00640 1 -- 172.21.15.52:0/2406475382 shutdown_connections 2024-03-12T17:06:44.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.185+0000 7f3f8ec00640 1 -- 172.21.15.52:0/2406475382 wait complete. 2024-03-12T17:06:44.195 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-12T17:06:44.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.417+0000 7fe653e00640 1 Processor -- start 2024-03-12T17:06:44.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.417+0000 7fe653e00640 1 -- start start 2024-03-12T17:06:44.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.418+0000 7fe653e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe65415b630 0x7fe6540700d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:44.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.418+0000 7fe653e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe654070610 0x7fe65406c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:44.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.418+0000 7fe653e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe65415b1a0 con 0x7fe6540ccd40 2024-03-12T17:06:44.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.418+0000 7fe653e00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe654070aa0 con 0x7fe65415b630 2024-03-12T17:06:44.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.418+0000 7fe653e00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe65406d940 con 0x7fe654070610 2024-03-12T17:06:44.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.419+0000 7fe652000640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe65415b630 0x7fe6540700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:44.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.419+0000 7fe652000640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe65415b630 0x7fe6540700d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46766/0 (socket says 172.21.15.52:46766) 2024-03-12T17:06:44.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.419+0000 7fe652000640 1 -- 172.21.15.52:0/2916432778 learned_addr learned my addr 172.21.15.52:0/2916432778 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:44.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.419+0000 7fe653400640 1 --2- 172.21.15.52:0/2916432778 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe654070610 0x7fe65406c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:44.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.419+0000 7fe652000640 1 -- 172.21.15.52:0/2916432778 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe654070610 msgr2=0x7fe65406c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:44.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.419+0000 7fe652000640 1 --2- 172.21.15.52:0/2916432778 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe654070610 0x7fe65406c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:44.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.419+0000 7fe652000640 1 -- 172.21.15.52:0/2916432778 >> v1:172.21.15.102:6789/0 conn(0x7fe6540ccd40 legacy=0x7fe65415e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:44.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.419+0000 7fe652000640 1 -- 172.21.15.52:0/2916432778 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe65415eba0 con 0x7fe65415b630 2024-03-12T17:06:44.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.420+0000 7fe652000640 1 --2- 172.21.15.52:0/2916432778 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe65415b630 0x7fe6540700d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe644009e50 tx=0x7fe64402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5157bb18681af89e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:44.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.421+0000 7fe651600640 1 -- 172.21.15.52:0/2916432778 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64402c9e0 con 0x7fe65415b630 2024-03-12T17:06:44.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.421+0000 7fe651600640 1 -- 172.21.15.52:0/2916432778 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe64402cb40 con 0x7fe65415b630 2024-03-12T17:06:44.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.421+0000 7fe651600640 1 -- 172.21.15.52:0/2916432778 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6440343f0 con 0x7fe65415b630 2024-03-12T17:06:44.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.421+0000 7fe653e00640 1 -- 172.21.15.52:0/2916432778 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe65415b630 msgr2=0x7fe6540700d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:44.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.421+0000 7fe653e00640 1 --2- 172.21.15.52:0/2916432778 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe65415b630 0x7fe6540700d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe644009e50 tx=0x7fe64402be40 comp rx=0 tx=0).stop 2024-03-12T17:06:44.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.421+0000 7fe653e00640 1 -- 172.21.15.52:0/2916432778 shutdown_connections 2024-03-12T17:06:44.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.421+0000 7fe653e00640 1 --2- 172.21.15.52:0/2916432778 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe654070610 0x7fe65406c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:44.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.421+0000 7fe653e00640 1 --2- 172.21.15.52:0/2916432778 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe65415b630 0x7fe6540700d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:44.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.421+0000 7fe653e00640 1 -- 172.21.15.52:0/2916432778 >> 172.21.15.52:0/2916432778 conn(0x7fe654065df0 msgr2=0x7fe654069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:44.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.421+0000 7fe653e00640 1 -- 172.21.15.52:0/2916432778 shutdown_connections 2024-03-12T17:06:44.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.422+0000 7fe653e00640 1 -- 172.21.15.52:0/2916432778 wait complete. 2024-03-12T17:06:44.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.423+0000 7fe653e00640 1 Processor -- start 2024-03-12T17:06:44.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.423+0000 7fe653e00640 1 -- start start 2024-03-12T17:06:44.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.424+0000 7fe653e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe654070610 0x7fe65414a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:44.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.424+0000 7fe653e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6540ccd40 0x7fe6541522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:44.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.424+0000 7fe653e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe65415b630 0x7fe654148920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:44.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.424+0000 7fe653e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6540c35f0 con 0x7fe6540ccd40 2024-03-12T17:06:44.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.424+0000 7fe653e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe654149ff0 con 0x7fe65415b630 2024-03-12T17:06:44.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.424+0000 7fe653e00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6541493e0 con 0x7fe654070610 2024-03-12T17:06:44.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.425+0000 7fe652000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6540ccd40 0x7fe6541522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:44.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.425+0000 7fe652000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6540ccd40 0x7fe6541522e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46774/0 (socket says 172.21.15.52:46774) 2024-03-12T17:06:44.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.425+0000 7fe652000640 1 -- 172.21.15.52:0/3466102716 learned_addr learned my addr 172.21.15.52:0/3466102716 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:44.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.425+0000 7fe653400640 1 --2- 172.21.15.52:0/3466102716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe65415b630 0x7fe654148920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:44.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.425+0000 7fe652a00640 1 --2- 172.21.15.52:0/3466102716 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe654070610 0x7fe65414a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:44.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.425+0000 7fe652000640 1 -- 172.21.15.52:0/3466102716 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe654070610 msgr2=0x7fe65414a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:44.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.425+0000 7fe652000640 1 --2- 172.21.15.52:0/3466102716 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe654070610 0x7fe65414a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:44.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.425+0000 7fe652000640 1 -- 172.21.15.52:0/3466102716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe65415b630 msgr2=0x7fe654148920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:44.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.425+0000 7fe652000640 1 --2- 172.21.15.52:0/3466102716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe65415b630 0x7fe654148920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:44.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.425+0000 7fe652000640 1 -- 172.21.15.52:0/3466102716 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe644009b30 con 0x7fe6540ccd40 2024-03-12T17:06:44.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.425+0000 7fe653400640 1 --2- 172.21.15.52:0/3466102716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe65415b630 0x7fe654148920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:44.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.426+0000 7fe652000640 1 --2- 172.21.15.52:0/3466102716 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6540ccd40 0x7fe6541522e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe644000f80 tx=0x7fe644004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:44.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.426+0000 7fe64aa00640 1 -- 172.21.15.52:0/3466102716 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe644040670 con 0x7fe6540ccd40 2024-03-12T17:06:44.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.426+0000 7fe64aa00640 1 -- 172.21.15.52:0/3466102716 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6440407d0 con 0x7fe6540ccd40 2024-03-12T17:06:44.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.426+0000 7fe64aa00640 1 -- 172.21.15.52:0/3466102716 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe644040b80 con 0x7fe6540ccd40 2024-03-12T17:06:44.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.426+0000 7fe653e00640 1 -- 172.21.15.52:0/3466102716 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6541496c0 con 0x7fe6540ccd40 2024-03-12T17:06:44.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.426+0000 7fe653e00640 1 -- 172.21.15.52:0/3466102716 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe654149c80 con 0x7fe6540ccd40 2024-03-12T17:06:44.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.427+0000 7fe653e00640 1 -- 172.21.15.52:0/3466102716 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe654071b10 con 0x7fe6540ccd40 2024-03-12T17:06:44.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.428+0000 7fe64aa00640 1 -- 172.21.15.52:0/3466102716 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe644040ce0 con 0x7fe6540ccd40 2024-03-12T17:06:44.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.429+0000 7fe64aa00640 1 --2- 172.21.15.52:0/3466102716 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe63006ac00 0x7fe63006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:44.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.429+0000 7fe64aa00640 1 -- 172.21.15.52:0/3466102716 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe6440ac700 con 0x7fe6540ccd40 2024-03-12T17:06:44.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.430+0000 7fe652a00640 1 --2- 172.21.15.52:0/3466102716 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe63006ac00 0x7fe63006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:44.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.431+0000 7fe652a00640 1 --2- 172.21.15.52:0/3466102716 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe63006ac00 0x7fe63006d0c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe63c003d90 tx=0x7fe63c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:44.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.434+0000 7fe64aa00640 1 -- 172.21.15.52:0/3466102716 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fe644077680 con 0x7fe6540ccd40 2024-03-12T17:06:44.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.644+0000 7fe653e00640 1 -- 172.21.15.52:0/3466102716 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe65404e110 con 0x7fe6540ccd40 2024-03-12T17:06:44.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.645+0000 7fe64aa00640 1 -- 172.21.15.52:0/3466102716 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe64407aca0 con 0x7fe6540ccd40 2024-03-12T17:06:44.639 INFO:teuthology.orchestra.run.smithi052.stderr:listed 0 entries 2024-03-12T17:06:44.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.648+0000 7fe653e00640 1 -- 172.21.15.52:0/3466102716 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe63006ac00 msgr2=0x7fe63006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:44.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.648+0000 7fe653e00640 1 --2- 172.21.15.52:0/3466102716 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe63006ac00 0x7fe63006d0c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe63c003d90 tx=0x7fe63c009040 comp rx=0 tx=0).stop 2024-03-12T17:06:44.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.648+0000 7fe653e00640 1 -- 172.21.15.52:0/3466102716 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6540ccd40 msgr2=0x7fe6541522e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:44.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.648+0000 7fe653e00640 1 --2- 172.21.15.52:0/3466102716 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6540ccd40 0x7fe6541522e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe644000f80 tx=0x7fe644004a40 comp rx=0 tx=0).stop 2024-03-12T17:06:44.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.648+0000 7fe653e00640 1 -- 172.21.15.52:0/3466102716 shutdown_connections 2024-03-12T17:06:44.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.648+0000 7fe653e00640 1 --2- 172.21.15.52:0/3466102716 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe63006ac00 0x7fe63006d0c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:44.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.648+0000 7fe653e00640 1 --2- 172.21.15.52:0/3466102716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe65415b630 0x7fe654148920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:44.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.648+0000 7fe653e00640 1 --2- 172.21.15.52:0/3466102716 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6540ccd40 0x7fe6541522e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:44.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.648+0000 7fe653e00640 1 --2- 172.21.15.52:0/3466102716 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe654070610 0x7fe65414a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:44.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.648+0000 7fe653e00640 1 -- 172.21.15.52:0/3466102716 >> 172.21.15.52:0/3466102716 conn(0x7fe654065df0 msgr2=0x7fe6540726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:44.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.648+0000 7fe653e00640 1 -- 172.21.15.52:0/3466102716 shutdown_connections 2024-03-12T17:06:44.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:44.648+0000 7fe653e00640 1 -- 172.21.15.52:0/3466102716 wait complete. 2024-03-12T17:06:44.660 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi052.front.sepia.ceph.com... 2024-03-12T17:06:44.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:06:44.661 DEBUG:teuthology.orchestra.run.smithi052:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-03-12T17:06:44.745 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi052.stderr:ceph-fuse[42396]: fuse finished with error 0 and tester_r 0 2024-03-12T17:06:44.750 INFO:teuthology.orchestra.run:waiting for 300 2024-03-12T17:06:50.852 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi052.front.sepia.ceph.com 2024-03-12T17:06:50.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:06:50.853 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-03-12T17:06:50.922 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:06:51.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.154+0000 7fdba7e00640 1 Processor -- start 2024-03-12T17:06:51.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.154+0000 7fdba7e00640 1 -- start start 2024-03-12T17:06:51.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.156+0000 7fdba7e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdba0089360 0x7fdba00897a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:51.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.156+0000 7fdba7e00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdba0089ce0 con 0x7fdba0169ec0 2024-03-12T17:06:51.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.156+0000 7fdba7e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdba016abf0 con 0x7fdba006d6f0 2024-03-12T17:06:51.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.156+0000 7fdba7e00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdba00cc9e0 con 0x7fdba0089360 2024-03-12T17:06:51.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.156+0000 7fdba4a00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdba0089360 0x7fdba00897a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:51.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.156+0000 7fdba4a00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdba0089360 0x7fdba00897a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53332/0 (socket says 172.21.15.52:53332) 2024-03-12T17:06:51.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.156+0000 7fdba4a00640 1 -- 172.21.15.52:0/3369202176 learned_addr learned my addr 172.21.15.52:0/3369202176 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:51.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.157+0000 7fdb9fe00640 1 -- 172.21.15.52:0/3369202176 <== mon.1 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 308429174 0 0) 0x7fdba016abf0 con 0x7fdba006d6f0 2024-03-12T17:06:51.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.156+0000 7fdba5e00640 1 --1- 172.21.15.52:0/3369202176 >> v1:172.21.15.52:6789/0 conn(0x7fdba0169ec0 0x7fdba016a310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:57886/0 (socket says 172.21.15.52:57886) 2024-03-12T17:06:51.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.160+0000 7fdb9fe00640 1 -- 172.21.15.52:0/3369202176 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb88003500 con 0x7fdba006d6f0 2024-03-12T17:06:51.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.160+0000 7fdb9fe00640 1 -- 172.21.15.52:0/3369202176 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 983263063 0 0) 0x7fdba0089ce0 con 0x7fdba0169ec0 2024-03-12T17:06:51.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.160+0000 7fdb9fe00640 1 -- 172.21.15.52:0/3369202176 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdba016abf0 con 0x7fdba0169ec0 2024-03-12T17:06:51.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.161+0000 7fdba4a00640 1 -- 172.21.15.52:0/3369202176 >> v1:172.21.15.102:6789/0 conn(0x7fdba006d6f0 legacy=0x7fdba00cd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:51.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.161+0000 7fdba4a00640 1 -- 172.21.15.52:0/3369202176 >> v1:172.21.15.52:6789/0 conn(0x7fdba0169ec0 legacy=0x7fdba016a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:51.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.161+0000 7fdb9fe00640 1 -- 172.21.15.52:0/3369202176 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3688836772 0 0) 0x7fdba016abf0 con 0x7fdba0169ec0 2024-03-12T17:06:51.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.161+0000 7fdba4a00640 1 -- 172.21.15.52:0/3369202176 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba00cccc0 con 0x7fdba0089360 2024-03-12T17:06:51.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.162+0000 7fdba4a00640 1 --2- 172.21.15.52:0/3369202176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba0089360 0x7fdba00897a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c008120 tx=0x7fdb8c02bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8dde1f75ac87765 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:51.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.162+0000 7fdb9fe00640 1 -- 172.21.15.52:0/3369202176 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c004230 con 0x7fdba0089360 2024-03-12T17:06:51.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.162+0000 7fdb9fe00640 1 -- 172.21.15.52:0/3369202176 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8c004390 con 0x7fdba0089360 2024-03-12T17:06:51.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.162+0000 7fdb9fe00640 1 -- 172.21.15.52:0/3369202176 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c02c970 con 0x7fdba0089360 2024-03-12T17:06:51.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.163+0000 7fdba7e00640 1 -- 172.21.15.52:0/3369202176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba0089360 msgr2=0x7fdba00897a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:51.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.163+0000 7fdba7e00640 1 --2- 172.21.15.52:0/3369202176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba0089360 0x7fdba00897a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c008120 tx=0x7fdb8c02bdd0 comp rx=0 tx=0).stop 2024-03-12T17:06:51.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.163+0000 7fdba7e00640 1 -- 172.21.15.52:0/3369202176 shutdown_connections 2024-03-12T17:06:51.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.163+0000 7fdba7e00640 1 --2- 172.21.15.52:0/3369202176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba0089360 0x7fdba00897a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:51.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.163+0000 7fdba7e00640 1 -- 172.21.15.52:0/3369202176 >> 172.21.15.52:0/3369202176 conn(0x7fdba00b03a0 msgr2=0x7fdba00b0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:51.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.163+0000 7fdba7e00640 1 -- 172.21.15.52:0/3369202176 shutdown_connections 2024-03-12T17:06:51.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.163+0000 7fdba7e00640 1 -- 172.21.15.52:0/3369202176 wait complete. 2024-03-12T17:06:51.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.165+0000 7fdba7e00640 1 Processor -- start 2024-03-12T17:06:51.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.165+0000 7fdba7e00640 1 -- start start 2024-03-12T17:06:51.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.166+0000 7fdba7e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba006d6f0 0x7fdba0158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:51.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.166+0000 7fdba7e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdba0089360 0x7fdba01559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:51.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.166+0000 7fdba7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdba0169ec0 0x7fdba01597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:51.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.166+0000 7fdba7e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba01592b0 con 0x7fdba006d6f0 2024-03-12T17:06:51.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.166+0000 7fdba7e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba0156670 con 0x7fdba0169ec0 2024-03-12T17:06:51.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.166+0000 7fdba7e00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba0156950 con 0x7fdba0089360 2024-03-12T17:06:51.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.166+0000 7fdba5400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba006d6f0 0x7fdba0158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:51.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.166+0000 7fdba5400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba006d6f0 0x7fdba0158d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53338/0 (socket says 172.21.15.52:53338) 2024-03-12T17:06:51.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.166+0000 7fdba5400640 1 -- 172.21.15.52:0/3996936263 learned_addr learned my addr 172.21.15.52:0/3996936263 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:51.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.166+0000 7fdba5e00640 1 --2- 172.21.15.52:0/3996936263 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdba0169ec0 0x7fdba01597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:51.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.166+0000 7fdba4a00640 1 --2- 172.21.15.52:0/3996936263 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdba0089360 0x7fdba01559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:51.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.166+0000 7fdba5400640 1 -- 172.21.15.52:0/3996936263 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdba0089360 msgr2=0x7fdba01559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:51.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.167+0000 7fdba5400640 1 --2- 172.21.15.52:0/3996936263 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdba0089360 0x7fdba01559f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:51.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.167+0000 7fdba5400640 1 -- 172.21.15.52:0/3996936263 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdba0169ec0 msgr2=0x7fdba01597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:51.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.167+0000 7fdba5400640 1 --2- 172.21.15.52:0/3996936263 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdba0169ec0 0x7fdba01597c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:51.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.167+0000 7fdba5400640 1 -- 172.21.15.52:0/3996936263 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb8c00a580 con 0x7fdba006d6f0 2024-03-12T17:06:51.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.167+0000 7fdba4a00640 1 --2- 172.21.15.52:0/3996936263 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdba0089360 0x7fdba01559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:51.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.167+0000 7fdba5400640 1 --2- 172.21.15.52:0/3996936263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba006d6f0 0x7fdba0158d70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdb9400c400 tx=0x7fdb9400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:51.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.167+0000 7fdb9d600640 1 -- 172.21.15.52:0/3996936263 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb940052d0 con 0x7fdba006d6f0 2024-03-12T17:06:51.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.167+0000 7fdb9d600640 1 -- 172.21.15.52:0/3996936263 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb94005430 con 0x7fdba006d6f0 2024-03-12T17:06:51.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.167+0000 7fdb9d600640 1 -- 172.21.15.52:0/3996936263 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb94003830 con 0x7fdba006d6f0 2024-03-12T17:06:51.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.167+0000 7fdba7e00640 1 -- 172.21.15.52:0/3996936263 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdba0156c30 con 0x7fdba006d6f0 2024-03-12T17:06:51.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.167+0000 7fdba7e00640 1 -- 172.21.15.52:0/3996936263 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba01571f0 con 0x7fdba006d6f0 2024-03-12T17:06:51.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.169+0000 7fdba7e00640 1 -- 172.21.15.52:0/3996936263 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdba006ea30 con 0x7fdba006d6f0 2024-03-12T17:06:51.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.169+0000 7fdb9d600640 1 -- 172.21.15.52:0/3996936263 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fdb94003990 con 0x7fdba006d6f0 2024-03-12T17:06:51.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.171+0000 7fdb9d600640 1 --2- 172.21.15.52:0/3996936263 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fdb8006ac00 0x7fdb8006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:51.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.171+0000 7fdb9d600640 1 -- 172.21.15.52:0/3996936263 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdb94088cf0 con 0x7fdba006d6f0 2024-03-12T17:06:51.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.171+0000 7fdba4a00640 1 --2- 172.21.15.52:0/3996936263 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fdb8006ac00 0x7fdb8006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:51.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.172+0000 7fdba4a00640 1 --2- 172.21.15.52:0/3996936263 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fdb8006ac00 0x7fdb8006d0c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c035000 tx=0x7fdb8c030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:51.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.176+0000 7fdb9d600640 1 -- 172.21.15.52:0/3996936263 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fdb9408d070 con 0x7fdba006d6f0 2024-03-12T17:06:51.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.379+0000 7fdba7e00640 1 -- 172.21.15.52:0/3996936263 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdba0073390 con 0x7fdba006d6f0 2024-03-12T17:06:51.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.382+0000 7fdb9d600640 1 -- 172.21.15.52:0/3996936263 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7fdb94017020 con 0x7fdba006d6f0 2024-03-12T17:06:51.376 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:06:51.376 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":25,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:06:30.468476+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T17:06:19.218543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T17:06:23.525474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T17:06:24.911574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-12T17:06:28.017172+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.733031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.775178+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.591522+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.820838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.829252+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.582481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:06:51.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.385+0000 7fdba7e00640 1 -- 172.21.15.52:0/3996936263 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fdb8006ac00 msgr2=0x7fdb8006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:51.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.385+0000 7fdba7e00640 1 --2- 172.21.15.52:0/3996936263 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fdb8006ac00 0x7fdb8006d0c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c035000 tx=0x7fdb8c030000 comp rx=0 tx=0).stop 2024-03-12T17:06:51.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.385+0000 7fdba7e00640 1 -- 172.21.15.52:0/3996936263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba006d6f0 msgr2=0x7fdba0158d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:51.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.385+0000 7fdba7e00640 1 --2- 172.21.15.52:0/3996936263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba006d6f0 0x7fdba0158d70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdb9400c400 tx=0x7fdb9400c8c0 comp rx=0 tx=0).stop 2024-03-12T17:06:51.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.385+0000 7fdba7e00640 1 -- 172.21.15.52:0/3996936263 shutdown_connections 2024-03-12T17:06:51.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.385+0000 7fdba7e00640 1 --2- 172.21.15.52:0/3996936263 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fdb8006ac00 0x7fdb8006d0c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:51.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.385+0000 7fdba7e00640 1 --2- 172.21.15.52:0/3996936263 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdba0089360 0x7fdba01559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:51.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.385+0000 7fdba7e00640 1 --2- 172.21.15.52:0/3996936263 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdba0169ec0 0x7fdba01597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:51.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.385+0000 7fdba7e00640 1 --2- 172.21.15.52:0/3996936263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba006d6f0 0x7fdba0158d70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:51.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.385+0000 7fdba7e00640 1 -- 172.21.15.52:0/3996936263 >> 172.21.15.52:0/3996936263 conn(0x7fdba00b03a0 msgr2=0x7fdba0166f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:51.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.385+0000 7fdba7e00640 1 -- 172.21.15.52:0/3996936263 shutdown_connections 2024-03-12T17:06:51.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.385+0000 7fdba7e00640 1 -- 172.21.15.52:0/3996936263 wait complete. 2024-03-12T17:06:51.400 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:06:51.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.624+0000 7f0f02200640 1 Processor -- start 2024-03-12T17:06:51.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.625+0000 7f0f02200640 1 -- start start 2024-03-12T17:06:51.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.626+0000 7f0f02200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0efc0cf5c0 0x7f0efc0b8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:51.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.626+0000 7f0f02200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f0efc0d0c20 0x7f0efc0d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:51.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.626+0000 7f0f02200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0efc15ce90 con 0x7f0efc15cab0 2024-03-12T17:06:51.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.626+0000 7f0f02200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0efc0cddf0 con 0x7f0efc0d0c20 2024-03-12T17:06:51.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.626+0000 7f0f02200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc0cd570 con 0x7f0efc0cf5c0 2024-03-12T17:06:51.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.627+0000 7f0f01800640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f0efc0d0c20 0x7f0efc0d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:51.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.627+0000 7f0f01800640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f0efc0d0c20 0x7f0efc0d10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53340/0 (socket says 172.21.15.52:53340) 2024-03-12T17:06:51.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.627+0000 7f0f01800640 1 -- 172.21.15.52:0/4211899542 learned_addr learned my addr 172.21.15.52:0/4211899542 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:51.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.627+0000 7f0efbe00640 1 --2- 172.21.15.52:0/4211899542 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0efc0cf5c0 0x7f0efc0b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:51.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.627+0000 7f0f01800640 1 -- 172.21.15.52:0/4211899542 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0efc0cf5c0 msgr2=0x7f0efc0b8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:51.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.627+0000 7f0f01800640 1 --2- 172.21.15.52:0/4211899542 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0efc0cf5c0 0x7f0efc0b8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:51.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.627+0000 7f0f01800640 1 -- 172.21.15.52:0/4211899542 >> v1:172.21.15.102:6789/0 conn(0x7f0efc15cab0 legacy=0x7f0efc0ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:51.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.627+0000 7f0f01800640 1 -- 172.21.15.52:0/4211899542 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0efc161370 con 0x7f0efc0d0c20 2024-03-12T17:06:51.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.628+0000 7f0f01800640 1 --2- 172.21.15.52:0/4211899542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0efc0d0c20 0x7f0efc0d10d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4009e50 tx=0x7f0ef402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab80f8f492d062b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:51.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.629+0000 7f0efb400640 1 -- 172.21.15.52:0/4211899542 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef402caf0 con 0x7f0efc0d0c20 2024-03-12T17:06:51.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.629+0000 7f0efb400640 1 -- 172.21.15.52:0/4211899542 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ef402cc50 con 0x7f0efc0d0c20 2024-03-12T17:06:51.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.629+0000 7f0efb400640 1 -- 172.21.15.52:0/4211899542 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef40343f0 con 0x7f0efc0d0c20 2024-03-12T17:06:51.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.629+0000 7f0f02200640 1 -- 172.21.15.52:0/4211899542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0efc0d0c20 msgr2=0x7f0efc0d10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:51.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.629+0000 7f0f02200640 1 --2- 172.21.15.52:0/4211899542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0efc0d0c20 0x7f0efc0d10d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4009e50 tx=0x7f0ef402bf50 comp rx=0 tx=0).stop 2024-03-12T17:06:51.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.629+0000 7f0f02200640 1 -- 172.21.15.52:0/4211899542 shutdown_connections 2024-03-12T17:06:51.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.629+0000 7f0f02200640 1 --2- 172.21.15.52:0/4211899542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0efc0d0c20 0x7f0efc0d10d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:51.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.629+0000 7f0f02200640 1 --2- 172.21.15.52:0/4211899542 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0efc0cf5c0 0x7f0efc0b8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:51.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.629+0000 7f0f02200640 1 -- 172.21.15.52:0/4211899542 >> 172.21.15.52:0/4211899542 conn(0x7f0efc06ae90 msgr2=0x7f0efc06b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:51.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.629+0000 7f0f02200640 1 -- 172.21.15.52:0/4211899542 shutdown_connections 2024-03-12T17:06:51.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.629+0000 7f0f02200640 1 -- 172.21.15.52:0/4211899542 wait complete. 2024-03-12T17:06:51.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.631+0000 7f0f02200640 1 Processor -- start 2024-03-12T17:06:51.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.631+0000 7f0f02200640 1 -- start start 2024-03-12T17:06:51.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.632+0000 7f0f02200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0efc0cf5c0 0x7f0efc157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:51.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.632+0000 7f0f02200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0efc0d0c20 0x7f0efc159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:51.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.632+0000 7f0f02200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0efc15cab0 0x7f0efc159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:51.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.632+0000 7f0f02200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc0cba80 con 0x7f0efc0cf5c0 2024-03-12T17:06:51.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.632+0000 7f0f02200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc0cbd60 con 0x7f0efc0d0c20 2024-03-12T17:06:51.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.632+0000 7f0f02200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc0cc040 con 0x7f0efc15cab0 2024-03-12T17:06:51.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.633+0000 7f0f00e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0efc0cf5c0 0x7f0efc157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:51.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.633+0000 7f0f00e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0efc0cf5c0 0x7f0efc157b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53352/0 (socket says 172.21.15.52:53352) 2024-03-12T17:06:51.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.633+0000 7f0f00e00640 1 -- 172.21.15.52:0/374444583 learned_addr learned my addr 172.21.15.52:0/374444583 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:51.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.633+0000 7f0efbe00640 1 --2- 172.21.15.52:0/374444583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0efc0d0c20 0x7f0efc159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:51.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.633+0000 7f0f01800640 1 --2- 172.21.15.52:0/374444583 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0efc15cab0 0x7f0efc159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:51.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.633+0000 7f0f00e00640 1 -- 172.21.15.52:0/374444583 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0efc15cab0 msgr2=0x7f0efc159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:51.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.633+0000 7f0f00e00640 1 --2- 172.21.15.52:0/374444583 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0efc15cab0 0x7f0efc159a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:51.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.633+0000 7f0f00e00640 1 -- 172.21.15.52:0/374444583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0efc0d0c20 msgr2=0x7f0efc159550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:51.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.633+0000 7f0f00e00640 1 --2- 172.21.15.52:0/374444583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0efc0d0c20 0x7f0efc159550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:51.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.633+0000 7f0f00e00640 1 -- 172.21.15.52:0/374444583 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef4009b30 con 0x7f0efc0cf5c0 2024-03-12T17:06:51.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.633+0000 7f0efbe00640 1 --2- 172.21.15.52:0/374444583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0efc0d0c20 0x7f0efc159550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:51.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.633+0000 7f0f00e00640 1 --2- 172.21.15.52:0/374444583 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0efc0cf5c0 0x7f0efc157b70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0ef000c400 tx=0x7f0ef000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:51.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.634+0000 7f0ef8c00640 1 -- 172.21.15.52:0/374444583 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef00052d0 con 0x7f0efc0cf5c0 2024-03-12T17:06:51.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.634+0000 7f0ef8c00640 1 -- 172.21.15.52:0/374444583 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ef0005430 con 0x7f0efc0cf5c0 2024-03-12T17:06:51.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.634+0000 7f0ef8c00640 1 -- 172.21.15.52:0/374444583 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef0003830 con 0x7f0efc0cf5c0 2024-03-12T17:06:51.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.634+0000 7f0f02200640 1 -- 172.21.15.52:0/374444583 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0efc0cc320 con 0x7f0efc0cf5c0 2024-03-12T17:06:51.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.634+0000 7f0f02200640 1 -- 172.21.15.52:0/374444583 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0efc0cad50 con 0x7f0efc0cf5c0 2024-03-12T17:06:51.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.635+0000 7f0f02200640 1 -- 172.21.15.52:0/374444583 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0efc04e030 con 0x7f0efc0cf5c0 2024-03-12T17:06:51.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.636+0000 7f0ef8c00640 1 -- 172.21.15.52:0/374444583 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0ef0003990 con 0x7f0efc0cf5c0 2024-03-12T17:06:51.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.637+0000 7f0ef8c00640 1 --2- 172.21.15.52:0/374444583 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f0edc06ac30 0x7f0edc06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:51.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.637+0000 7f0ef8c00640 1 -- 172.21.15.52:0/374444583 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0ef0088c50 con 0x7f0efc0cf5c0 2024-03-12T17:06:51.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.638+0000 7f0efbe00640 1 --2- 172.21.15.52:0/374444583 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f0edc06ac30 0x7f0edc06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:51.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.639+0000 7f0efbe00640 1 --2- 172.21.15.52:0/374444583 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f0edc06ac30 0x7f0edc06d0f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0efc0b28a0 tx=0x7f0ee8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:51.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.641+0000 7f0ef8c00640 1 -- 172.21.15.52:0/374444583 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f0ef0053aa0 con 0x7f0efc0cf5c0 2024-03-12T17:06:51.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.847+0000 7f0f02200640 1 -- 172.21.15.52:0/374444583 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0efc0c61d0 con 0x7f0efc0cf5c0 2024-03-12T17:06:51.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.850+0000 7f0ef8c00640 1 -- 172.21.15.52:0/374444583 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7f0ef00570c0 con 0x7f0efc0cf5c0 2024-03-12T17:06:51.844 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:06:51.845 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":25,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:06:30.468476+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T17:06:19.218543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T17:06:23.525474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T17:06:24.911574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-12T17:06:28.017172+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.733031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.775178+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.591522+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.820838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.829252+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.582481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:06:51.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.853+0000 7f0f02200640 1 -- 172.21.15.52:0/374444583 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f0edc06ac30 msgr2=0x7f0edc06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:51.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.853+0000 7f0f02200640 1 --2- 172.21.15.52:0/374444583 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f0edc06ac30 0x7f0edc06d0f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0efc0b28a0 tx=0x7f0ee8008040 comp rx=0 tx=0).stop 2024-03-12T17:06:51.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.853+0000 7f0f02200640 1 -- 172.21.15.52:0/374444583 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0efc0cf5c0 msgr2=0x7f0efc157b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:51.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.853+0000 7f0f02200640 1 --2- 172.21.15.52:0/374444583 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0efc0cf5c0 0x7f0efc157b70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0ef000c400 tx=0x7f0ef000c8c0 comp rx=0 tx=0).stop 2024-03-12T17:06:51.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.853+0000 7f0f02200640 1 -- 172.21.15.52:0/374444583 shutdown_connections 2024-03-12T17:06:51.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.853+0000 7f0f02200640 1 --2- 172.21.15.52:0/374444583 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f0edc06ac30 0x7f0edc06d0f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:51.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.853+0000 7f0f02200640 1 --2- 172.21.15.52:0/374444583 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0efc15cab0 0x7f0efc159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:51.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.853+0000 7f0f02200640 1 --2- 172.21.15.52:0/374444583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0efc0d0c20 0x7f0efc159550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:51.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.853+0000 7f0f02200640 1 --2- 172.21.15.52:0/374444583 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0efc0cf5c0 0x7f0efc157b70 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:51.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.853+0000 7f0f02200640 1 -- 172.21.15.52:0/374444583 >> 172.21.15.52:0/374444583 conn(0x7f0efc06ae90 msgr2=0x7f0efc06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:51.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.853+0000 7f0f02200640 1 -- 172.21.15.52:0/374444583 shutdown_connections 2024-03-12T17:06:51.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:51.853+0000 7f0f02200640 1 -- 172.21.15.52:0/374444583 wait complete. 2024-03-12T17:06:51.868 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-12T17:06:52.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.102+0000 7fa14b200640 1 Processor -- start 2024-03-12T17:06:52.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.102+0000 7fa14b200640 1 -- start start 2024-03-12T17:06:52.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.103+0000 7fa14b200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa14c06e2a0 0x7fa14c0ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:52.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.103+0000 7fa14b200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa14c06e700 0x7fa14c0cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:52.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.103+0000 7fa14b200640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa14c06f570 con 0x7fa14c15ca60 2024-03-12T17:06:52.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.103+0000 7fa14b200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa14c15ce40 con 0x7fa14c06e700 2024-03-12T17:06:52.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.103+0000 7fa14b200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c0ce5b0 con 0x7fa14c06e2a0 2024-03-12T17:06:52.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.104+0000 7fa149e00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fa14c15ca60 0x7fa14c06b6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:57900/0 (socket says 172.21.15.52:57900) 2024-03-12T17:06:52.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.104+0000 7fa149e00640 1 -- 172.21.15.52:0/452960561 learned_addr learned my addr 172.21.15.52:0/452960561 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:06:52.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.104+0000 7fa14a800640 1 --2- 172.21.15.52:0/452960561 >> v2:172.21.15.102:3300/0 conn(0x7fa14c06e700 0x7fa14c0cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:52.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.104+0000 7fa149400640 1 --2- 172.21.15.52:0/452960561 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa14c06e2a0 0x7fa14c0ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:52.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.104+0000 7fa148a00640 1 -- 172.21.15.52:0/452960561 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1110720957 0 0) 0x7fa14c06f570 con 0x7fa14c15ca60 2024-03-12T17:06:52.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.104+0000 7fa148a00640 1 -- 172.21.15.52:0/452960561 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa130003520 con 0x7fa14c15ca60 2024-03-12T17:06:52.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.105+0000 7fa148a00640 1 -- 172.21.15.52:0/452960561 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 904352088 0 0) 0x7fa130003520 con 0x7fa14c15ca60 2024-03-12T17:06:52.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.105+0000 7fa148a00640 1 -- 172.21.15.52:0/452960561 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa14c06e2a0 msgr2=0x7fa14c0ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:52.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.105+0000 7fa148a00640 1 --2- 172.21.15.52:0/452960561 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa14c06e2a0 0x7fa14c0ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:52.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.105+0000 7fa148a00640 1 -- 172.21.15.52:0/452960561 >> v2:172.21.15.102:3300/0 conn(0x7fa14c06e700 msgr2=0x7fa14c0cd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:52.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.105+0000 7fa148a00640 1 --2- 172.21.15.52:0/452960561 >> v2:172.21.15.102:3300/0 conn(0x7fa14c06e700 0x7fa14c0cd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:52.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.105+0000 7fa148a00640 1 -- 172.21.15.52:0/452960561 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14c06ecd0 con 0x7fa14c15ca60 2024-03-12T17:06:52.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.105+0000 7fa148a00640 1 -- 172.21.15.52:0/452960561 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7fa13c0019f0 con 0x7fa14c15ca60 2024-03-12T17:06:52.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.105+0000 7fa148a00640 1 -- 172.21.15.52:0/452960561 >> v1:172.21.15.52:6789/0 conn(0x7fa14c15ca60 legacy=0x7fa14c06b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:52.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.105+0000 7fa148a00640 1 --2- 172.21.15.52:0/452960561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa130003970 0x7fa130003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:52.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.105+0000 7fa148a00640 1 --2- 172.21.15.52:0/452960561 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa14c06e2a0 0x7fa1300047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:52.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.105+0000 7fa148a00640 1 --2- 172.21.15.52:0/452960561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa130004d00 0x7fa130005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:52.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.105+0000 7fa148a00640 1 -- 172.21.15.52:0/452960561 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c0ce5b0 con 0x7fa130003970 2024-03-12T17:06:52.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.105+0000 7fa148a00640 1 -- 172.21.15.52:0/452960561 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13000b980 con 0x7fa130004d00 2024-03-12T17:06:52.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.105+0000 7fa148a00640 1 -- 172.21.15.52:0/452960561 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa13000bac0 con 0x7fa14c06e2a0 2024-03-12T17:06:52.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.105+0000 7fa149e00640 1 --2- 172.21.15.52:0/452960561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa130003970 0x7fa130003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:52.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.106+0000 7fa149400640 1 --2- 172.21.15.52:0/452960561 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa14c06e2a0 0x7fa1300047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:52.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.106+0000 7fa14a800640 1 --2- 172.21.15.52:0/452960561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa130004d00 0x7fa130005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:52.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.106+0000 7fa149e00640 1 -- 172.21.15.52:0/452960561 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa14c06e2a0 msgr2=0x7fa1300047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:52.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.106+0000 7fa149e00640 1 --2- 172.21.15.52:0/452960561 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa14c06e2a0 0x7fa1300047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:52.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.106+0000 7fa149e00640 1 -- 172.21.15.52:0/452960561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa130004d00 msgr2=0x7fa130005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:52.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.106+0000 7fa149e00640 1 --2- 172.21.15.52:0/452960561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa130004d00 0x7fa130005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:52.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.106+0000 7fa149e00640 1 -- 172.21.15.52:0/452960561 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa130003520 con 0x7fa130003970 2024-03-12T17:06:52.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.107+0000 7fa149e00640 1 --2- 172.21.15.52:0/452960561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa130003970 0x7fa130003d90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00a790 tx=0x7fa1300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9953a3361ede887 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:52.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.107+0000 7fa148a00640 1 -- 172.21.15.52:0/452960561 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c034630 con 0x7fa130003970 2024-03-12T17:06:52.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.107+0000 7fa148a00640 1 -- 172.21.15.52:0/452960561 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13c02f070 con 0x7fa130003970 2024-03-12T17:06:52.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.107+0000 7fa148a00640 1 -- 172.21.15.52:0/452960561 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c0349f0 con 0x7fa130003970 2024-03-12T17:06:52.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.108+0000 7fa14b200640 1 -- 172.21.15.52:0/452960561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa130003970 msgr2=0x7fa130003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:52.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.108+0000 7fa14b200640 1 --2- 172.21.15.52:0/452960561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa130003970 0x7fa130003d90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00a790 tx=0x7fa1300057f0 comp rx=0 tx=0).stop 2024-03-12T17:06:52.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.108+0000 7fa14a800640 1 -- 172.21.15.52:0/452960561 reap_dead start 2024-03-12T17:06:52.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.108+0000 7fa14b200640 1 -- 172.21.15.52:0/452960561 shutdown_connections 2024-03-12T17:06:52.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.108+0000 7fa14b200640 1 -- 172.21.15.52:0/452960561 >> 172.21.15.52:0/452960561 conn(0x7fa14c0d0520 msgr2=0x7fa14c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:52.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.108+0000 7fa14b200640 1 -- 172.21.15.52:0/452960561 shutdown_connections 2024-03-12T17:06:52.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.108+0000 7fa14b200640 1 -- 172.21.15.52:0/452960561 wait complete. 2024-03-12T17:06:52.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.110+0000 7fa14b200640 1 Processor -- start 2024-03-12T17:06:52.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.110+0000 7fa14b200640 1 -- start start 2024-03-12T17:06:52.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.111+0000 7fa14b200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa130003970 0x7fa14c15aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:52.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.111+0000 7fa14b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa14c15f6d0 0x7fa14c0b0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:52.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.111+0000 7fa14b200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa14c0b0670 0x7fa14c15ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:52.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.111+0000 7fa14b200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c0b3f60 con 0x7fa130003970 2024-03-12T17:06:52.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.111+0000 7fa14b200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c0b0af0 con 0x7fa14c15f6d0 2024-03-12T17:06:52.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.111+0000 7fa14b200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c0b32e0 con 0x7fa14c0b0670 2024-03-12T17:06:52.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.111+0000 7fa149e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa130003970 0x7fa14c15aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:52.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.111+0000 7fa149e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa130003970 0x7fa14c15aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53380/0 (socket says 172.21.15.52:53380) 2024-03-12T17:06:52.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.111+0000 7fa149e00640 1 -- 172.21.15.52:0/4038219919 learned_addr learned my addr 172.21.15.52:0/4038219919 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:52.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.112+0000 7fa14a800640 1 --2- 172.21.15.52:0/4038219919 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa14c0b0670 0x7fa14c15ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:52.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.112+0000 7fa149400640 1 --2- 172.21.15.52:0/4038219919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa14c15f6d0 0x7fa14c0b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:52.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.112+0000 7fa149e00640 1 -- 172.21.15.52:0/4038219919 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa14c0b0670 msgr2=0x7fa14c15ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:52.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.112+0000 7fa149e00640 1 --2- 172.21.15.52:0/4038219919 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa14c0b0670 0x7fa14c15ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:52.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.112+0000 7fa149e00640 1 -- 172.21.15.52:0/4038219919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa14c15f6d0 msgr2=0x7fa14c0b0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:52.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.112+0000 7fa149e00640 1 --2- 172.21.15.52:0/4038219919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa14c15f6d0 0x7fa14c0b0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:52.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.112+0000 7fa149e00640 1 -- 172.21.15.52:0/4038219919 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14c0b35c0 con 0x7fa130003970 2024-03-12T17:06:52.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.112+0000 7fa149e00640 1 --2- 172.21.15.52:0/4038219919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa130003970 0x7fa14c15aa60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa13c02cad0 tx=0x7fa13c001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:52.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.112+0000 7fa142000640 1 -- 172.21.15.52:0/4038219919 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c044030 con 0x7fa130003970 2024-03-12T17:06:52.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.112+0000 7fa142000640 1 -- 172.21.15.52:0/4038219919 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13c004e30 con 0x7fa130003970 2024-03-12T17:06:52.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.112+0000 7fa14b200640 1 -- 172.21.15.52:0/4038219919 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa14c0b38a0 con 0x7fa130003970 2024-03-12T17:06:52.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.112+0000 7fa142000640 1 -- 172.21.15.52:0/4038219919 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c034630 con 0x7fa130003970 2024-03-12T17:06:52.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.112+0000 7fa14b200640 1 -- 172.21.15.52:0/4038219919 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa14c15eb30 con 0x7fa130003970 2024-03-12T17:06:52.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.113+0000 7fa14b200640 1 -- 172.21.15.52:0/4038219919 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa14c0b41d0 con 0x7fa130003970 2024-03-12T17:06:52.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.114+0000 7fa142000640 1 -- 172.21.15.52:0/4038219919 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa13c047070 con 0x7fa130003970 2024-03-12T17:06:52.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.116+0000 7fa142000640 1 --2- 172.21.15.52:0/4038219919 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa12806ac00 0x7fa12806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:52.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.116+0000 7fa142000640 1 -- 172.21.15.52:0/4038219919 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa13c045070 con 0x7fa130003970 2024-03-12T17:06:52.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.117+0000 7fa149400640 1 --2- 172.21.15.52:0/4038219919 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa12806ac00 0x7fa12806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:52.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.118+0000 7fa149400640 1 --2- 172.21.15.52:0/4038219919 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa12806ac00 0x7fa12806d0c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa14c0d0a70 tx=0x7fa134008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:52.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.121+0000 7fa142000640 1 -- 172.21.15.52:0/4038219919 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fa13c076a60 con 0x7fa130003970 2024-03-12T17:06:52.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:52.364+0000 7fa14b200640 1 -- 172.21.15.52:0/4038219919 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa14c0b41d0 con 0x7fa130003970 2024-03-12T17:06:53.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.078+0000 7fa142000640 1 -- 172.21.15.52:0/4038219919 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa13c07a080 con 0x7fa130003970 2024-03-12T17:06:53.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.081+0000 7fa14b200640 1 -- 172.21.15.52:0/4038219919 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa12806ac00 msgr2=0x7fa12806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:53.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.081+0000 7fa14b200640 1 --2- 172.21.15.52:0/4038219919 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa12806ac00 0x7fa12806d0c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa14c0d0a70 tx=0x7fa134008040 comp rx=0 tx=0).stop 2024-03-12T17:06:53.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.082+0000 7fa14b200640 1 -- 172.21.15.52:0/4038219919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa130003970 msgr2=0x7fa14c15aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:53.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.082+0000 7fa14b200640 1 --2- 172.21.15.52:0/4038219919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa130003970 0x7fa14c15aa60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa13c02cad0 tx=0x7fa13c001cb0 comp rx=0 tx=0).stop 2024-03-12T17:06:53.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.082+0000 7fa14b200640 1 -- 172.21.15.52:0/4038219919 shutdown_connections 2024-03-12T17:06:53.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.082+0000 7fa14b200640 1 --2- 172.21.15.52:0/4038219919 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa12806ac00 0x7fa12806d0c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:53.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.082+0000 7fa14b200640 1 --2- 172.21.15.52:0/4038219919 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa14c0b0670 0x7fa14c15ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:53.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.082+0000 7fa14b200640 1 --2- 172.21.15.52:0/4038219919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa14c15f6d0 0x7fa14c0b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:53.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.082+0000 7fa14b200640 1 --2- 172.21.15.52:0/4038219919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa130003970 0x7fa14c15aa60 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:53.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.082+0000 7fa14b200640 1 -- 172.21.15.52:0/4038219919 >> 172.21.15.52:0/4038219919 conn(0x7fa14c0d0520 msgr2=0x7fa14c0b1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:53.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.082+0000 7fa14b200640 1 -- 172.21.15.52:0/4038219919 shutdown_connections 2024-03-12T17:06:53.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.082+0000 7fa14b200640 1 -- 172.21.15.52:0/4038219919 wait complete. 2024-03-12T17:06:53.100 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-03-12T17:06:53.101 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-12T17:06:53.170 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-12T17:06:53.177 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-12T17:06:53.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.321+0000 7f6b16a00640 1 Processor -- start 2024-03-12T17:06:53.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.321+0000 7f6b16a00640 1 -- start start 2024-03-12T17:06:53.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.323+0000 7f6b16a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6b1006a120 0x7f6b100b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:53.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.323+0000 7f6b16a00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b1006fc80 con 0x7f6b1015ca00 2024-03-12T17:06:53.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.323+0000 7f6b16a00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b100cf310 con 0x7f6b10069c90 2024-03-12T17:06:53.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.323+0000 7f6b16a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b100cf5f0 con 0x7f6b1006a120 2024-03-12T17:06:53.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.323+0000 7f6b0fe00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f6b1015ca00 0x7f6b1006f570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:57914/0 (socket says 172.21.15.52:57914) 2024-03-12T17:06:53.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.323+0000 7f6b0fe00640 1 -- 172.21.15.52:0/1504467430 learned_addr learned my addr 172.21.15.52:0/1504467430 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:06:53.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.324+0000 7f6b14a00640 1 --2- 172.21.15.52:0/1504467430 >> v2:172.21.15.102:3300/0 conn(0x7f6b1006a120 0x7f6b100b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:53.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.324+0000 7f6b0ea00640 1 -- 172.21.15.52:0/1504467430 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3991946034 0 0) 0x7f6b1006fc80 con 0x7f6b1015ca00 2024-03-12T17:06:53.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.324+0000 7f6b0ea00640 1 -- 172.21.15.52:0/1504467430 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6af4003520 con 0x7f6b1015ca00 2024-03-12T17:06:53.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.324+0000 7f6b0ea00640 1 -- 172.21.15.52:0/1504467430 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3928107529 0 0) 0x7f6af4003520 con 0x7f6b1015ca00 2024-03-12T17:06:53.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.324+0000 7f6b0ea00640 1 -- 172.21.15.52:0/1504467430 >> v2:172.21.15.102:3300/0 conn(0x7f6b1006a120 msgr2=0x7f6b100b2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:53.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.324+0000 7f6b0ea00640 1 --2- 172.21.15.52:0/1504467430 >> v2:172.21.15.102:3300/0 conn(0x7f6b1006a120 0x7f6b100b2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:53.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.324+0000 7f6b0ea00640 1 -- 172.21.15.52:0/1504467430 >> v1:172.21.15.102:6789/0 conn(0x7f6b10069c90 legacy=0x7f6b100b26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:53.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.324+0000 7f6b0ea00640 1 -- 172.21.15.52:0/1504467430 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b100cf8d0 con 0x7f6b1015ca00 2024-03-12T17:06:53.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.324+0000 7f6b0ea00640 1 -- 172.21.15.52:0/1504467430 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f6b00001670 con 0x7f6b1015ca00 2024-03-12T17:06:53.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.325+0000 7f6b0ea00640 1 -- 172.21.15.52:0/1504467430 >> v1:172.21.15.52:6789/0 conn(0x7f6b1015ca00 legacy=0x7f6b1006f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:53.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.325+0000 7f6b0ea00640 1 --2- 172.21.15.52:0/1504467430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6af4003970 0x7f6af4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:53.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.325+0000 7f6b0ea00640 1 --2- 172.21.15.52:0/1504467430 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6af40047c0 0x7f6af4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:53.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.325+0000 7f6b0ea00640 1 --2- 172.21.15.52:0/1504467430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6af4005180 0x7f6af400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:53.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.325+0000 7f6b0ea00640 1 -- 172.21.15.52:0/1504467430 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b100cf5f0 con 0x7f6af4005180 2024-03-12T17:06:53.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.325+0000 7f6b0ea00640 1 -- 172.21.15.52:0/1504467430 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af400c020 con 0x7f6af4003970 2024-03-12T17:06:53.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.325+0000 7f6b0ea00640 1 -- 172.21.15.52:0/1504467430 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af400c1e0 con 0x7f6af40047c0 2024-03-12T17:06:53.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.325+0000 7f6b14a00640 1 --2- 172.21.15.52:0/1504467430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6af4005180 0x7f6af400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:53.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.325+0000 7f6b0fe00640 1 --2- 172.21.15.52:0/1504467430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6af4003970 0x7f6af4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:53.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.325+0000 7f6b0f400640 1 --2- 172.21.15.52:0/1504467430 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6af40047c0 0x7f6af4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:53.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.326+0000 7f6b14a00640 1 -- 172.21.15.52:0/1504467430 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6af40047c0 msgr2=0x7f6af4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:53.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.326+0000 7f6b14a00640 1 --2- 172.21.15.52:0/1504467430 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6af40047c0 0x7f6af4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:53.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.326+0000 7f6b14a00640 1 -- 172.21.15.52:0/1504467430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6af4003970 msgr2=0x7f6af4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:53.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.326+0000 7f6b14a00640 1 --2- 172.21.15.52:0/1504467430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6af4003970 0x7f6af4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:53.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.326+0000 7f6b14a00640 1 -- 172.21.15.52:0/1504467430 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af4003520 con 0x7f6af4005180 2024-03-12T17:06:53.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.326+0000 7f6b0fe00640 1 --2- 172.21.15.52:0/1504467430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6af4003970 0x7f6af4003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:06:53.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.327+0000 7f6b14a00640 1 -- 172.21.15.52:0/1504467430 reap_dead start 2024-03-12T17:06:53.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.327+0000 7f6b14a00640 1 --2- 172.21.15.52:0/1504467430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6af4005180 0x7f6af400b980 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6b08009fc0 tx=0x7f6af40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4c17329de1404 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:53.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.327+0000 7f6b0ea00640 1 -- 172.21.15.52:0/1504467430 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0802cc60 con 0x7f6af4005180 2024-03-12T17:06:53.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.327+0000 7f6b0ea00640 1 -- 172.21.15.52:0/1504467430 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0800f040 con 0x7f6af4005180 2024-03-12T17:06:53.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.327+0000 7f6b0ea00640 1 -- 172.21.15.52:0/1504467430 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b08003990 con 0x7f6af4005180 2024-03-12T17:06:53.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.328+0000 7f6b16a00640 1 -- 172.21.15.52:0/1504467430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6af4005180 msgr2=0x7f6af400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:53.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.328+0000 7f6b16a00640 1 --2- 172.21.15.52:0/1504467430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6af4005180 0x7f6af400b980 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6b08009fc0 tx=0x7f6af40057f0 comp rx=0 tx=0).stop 2024-03-12T17:06:53.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.328+0000 7f6b16a00640 1 -- 172.21.15.52:0/1504467430 shutdown_connections 2024-03-12T17:06:53.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.328+0000 7f6b16a00640 1 --2- 172.21.15.52:0/1504467430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6af4005180 0x7f6af400b980 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:53.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.328+0000 7f6b16a00640 1 -- 172.21.15.52:0/1504467430 >> 172.21.15.52:0/1504467430 conn(0x7f6b1006d440 msgr2=0x7f6b1006d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:53.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.328+0000 7f6b16a00640 1 -- 172.21.15.52:0/1504467430 shutdown_connections 2024-03-12T17:06:53.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.328+0000 7f6b16a00640 1 -- 172.21.15.52:0/1504467430 wait complete. 2024-03-12T17:06:53.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.330+0000 7f6b16a00640 1 Processor -- start 2024-03-12T17:06:53.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.330+0000 7f6b16a00640 1 -- start start 2024-03-12T17:06:53.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.331+0000 7f6b16a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6af4005180 0x7f6b10148220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:53.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.331+0000 7f6b16a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6b1015f190 0x7f6b1015f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:53.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.331+0000 7f6b16a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b10148d30 0x7f6b101491b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:53.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.331+0000 7f6b16a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1015fb50 con 0x7f6b10148d30 2024-03-12T17:06:53.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.331+0000 7f6b16a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b101458b0 con 0x7f6af4005180 2024-03-12T17:06:53.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.331+0000 7f6b16a00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1014a5e0 con 0x7f6b1015f190 2024-03-12T17:06:53.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.331+0000 7f6b14a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b10148d30 0x7f6b101491b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:53.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.331+0000 7f6b14a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b10148d30 0x7f6b101491b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53400/0 (socket says 172.21.15.52:53400) 2024-03-12T17:06:53.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.332+0000 7f6b14a00640 1 -- 172.21.15.52:0/1954784868 learned_addr learned my addr 172.21.15.52:0/1954784868 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:53.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.332+0000 7f6b0f400640 1 --2- 172.21.15.52:0/1954784868 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6b1015f190 0x7f6b1015f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:53.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.332+0000 7f6b0fe00640 1 --2- 172.21.15.52:0/1954784868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6af4005180 0x7f6b10148220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:53.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.332+0000 7f6b14a00640 1 -- 172.21.15.52:0/1954784868 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6b1015f190 msgr2=0x7f6b1015f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:53.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.332+0000 7f6b14a00640 1 --2- 172.21.15.52:0/1954784868 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6b1015f190 0x7f6b1015f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:53.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.332+0000 7f6b14a00640 1 -- 172.21.15.52:0/1954784868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6af4005180 msgr2=0x7f6b10148220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:53.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.332+0000 7f6b14a00640 1 --2- 172.21.15.52:0/1954784868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6af4005180 0x7f6b10148220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:53.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.332+0000 7f6b14a00640 1 -- 172.21.15.52:0/1954784868 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b101499d0 con 0x7f6b10148d30 2024-03-12T17:06:53.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.332+0000 7f6b14a00640 1 --2- 172.21.15.52:0/1954784868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b10148d30 0x7f6b101491b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800e200 tx=0x7f6b08009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:53.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.333+0000 7f6b07e00640 1 -- 172.21.15.52:0/1954784868 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b08035e70 con 0x7f6b10148d30 2024-03-12T17:06:53.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.333+0000 7f6b07e00640 1 -- 172.21.15.52:0/1954784868 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b08003990 con 0x7f6b10148d30 2024-03-12T17:06:53.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.333+0000 7f6b07e00640 1 -- 172.21.15.52:0/1954784868 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b08003d40 con 0x7f6b10148d30 2024-03-12T17:06:53.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.333+0000 7f6b16a00640 1 -- 172.21.15.52:0/1954784868 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b10149cb0 con 0x7f6b10148d30 2024-03-12T17:06:53.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.333+0000 7f6b16a00640 1 -- 172.21.15.52:0/1954784868 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b1014a270 con 0x7f6b10148d30 2024-03-12T17:06:53.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.334+0000 7f6b16a00640 1 -- 172.21.15.52:0/1954784868 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b10066bb0 con 0x7f6b10148d30 2024-03-12T17:06:53.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.335+0000 7f6b07e00640 1 -- 172.21.15.52:0/1954784868 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6b08004030 con 0x7f6b10148d30 2024-03-12T17:06:53.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.336+0000 7f6b07e00640 1 --2- 172.21.15.52:0/1954784868 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f6af006ac00 0x7f6af006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:53.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.336+0000 7f6b07e00640 1 -- 172.21.15.52:0/1954784868 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6b080ac400 con 0x7f6b10148d30 2024-03-12T17:06:53.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.336+0000 7f6b0fe00640 1 --2- 172.21.15.52:0/1954784868 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f6af006ac00 0x7f6af006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:53.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.338+0000 7f6b0fe00640 1 --2- 172.21.15.52:0/1954784868 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f6af006ac00 0x7f6af006d0c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6b00000ff0 tx=0x7f6b00009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:53.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.340+0000 7f6b07e00640 1 -- 172.21.15.52:0/1954784868 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f6b08077250 con 0x7f6b10148d30 2024-03-12T17:06:53.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:53.580+0000 7f6b16a00640 1 -- 172.21.15.52:0/1954784868 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6b100aea80 con 0x7f6b10148d30 2024-03-12T17:06:54.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.081+0000 7f6b07e00640 1 -- 172.21.15.52:0/1954784868 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6b0807a870 con 0x7f6b10148d30 2024-03-12T17:06:54.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.084+0000 7f6b16a00640 1 -- 172.21.15.52:0/1954784868 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f6af006ac00 msgr2=0x7f6af006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:54.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.084+0000 7f6b16a00640 1 --2- 172.21.15.52:0/1954784868 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f6af006ac00 0x7f6af006d0c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6b00000ff0 tx=0x7f6b00009040 comp rx=0 tx=0).stop 2024-03-12T17:06:54.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.084+0000 7f6b16a00640 1 -- 172.21.15.52:0/1954784868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b10148d30 msgr2=0x7f6b101491b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:54.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.084+0000 7f6b16a00640 1 --2- 172.21.15.52:0/1954784868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b10148d30 0x7f6b101491b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800e200 tx=0x7f6b08009750 comp rx=0 tx=0).stop 2024-03-12T17:06:54.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.085+0000 7f6b16a00640 1 -- 172.21.15.52:0/1954784868 shutdown_connections 2024-03-12T17:06:54.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.085+0000 7f6b16a00640 1 --2- 172.21.15.52:0/1954784868 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f6af006ac00 0x7f6af006d0c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:54.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.085+0000 7f6b16a00640 1 --2- 172.21.15.52:0/1954784868 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6b1015f190 0x7f6b1015f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:54.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.085+0000 7f6b16a00640 1 --2- 172.21.15.52:0/1954784868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b10148d30 0x7f6b101491b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:54.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.085+0000 7f6b16a00640 1 --2- 172.21.15.52:0/1954784868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6af4005180 0x7f6b10148220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:54.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.085+0000 7f6b16a00640 1 -- 172.21.15.52:0/1954784868 >> 172.21.15.52:0/1954784868 conn(0x7f6b1006d440 msgr2=0x7f6b101570d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:54.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.085+0000 7f6b16a00640 1 -- 172.21.15.52:0/1954784868 shutdown_connections 2024-03-12T17:06:54.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.085+0000 7f6b16a00640 1 -- 172.21.15.52:0/1954784868 wait complete. 2024-03-12T17:06:54.097 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-12T17:06:54.097 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T17:06:54.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.347+0000 7fc1e6200640 1 Processor -- start 2024-03-12T17:06:54.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.347+0000 7fc1e6200640 1 -- start start 2024-03-12T17:06:54.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.348+0000 7fc1e6200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc1e00b5a30 0x7fc1e00b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:54.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.348+0000 7fc1e6200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc1e00b4350 0x7fc1e00b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:54.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.348+0000 7fc1e6200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1e00d1830 con 0x7fc1e015cda0 2024-03-12T17:06:54.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.349+0000 7fc1e6200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1e00b4d00 con 0x7fc1e00b5a30 2024-03-12T17:06:54.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.349+0000 7fc1e6200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1e00b1f20 con 0x7fc1e00b4350 2024-03-12T17:06:54.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.349+0000 7fc1dea00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc1e00b5a30 0x7fc1e00b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:54.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.349+0000 7fc1dea00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc1e00b5a30 0x7fc1e00b5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53404/0 (socket says 172.21.15.52:53404) 2024-03-12T17:06:54.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.349+0000 7fc1dea00640 1 -- 172.21.15.52:0/440531989 learned_addr learned my addr 172.21.15.52:0/440531989 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:54.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.349+0000 7fc1dfe00640 1 --2- 172.21.15.52:0/440531989 >> v2:172.21.15.102:3300/0 conn(0x7fc1e00b4350 0x7fc1e00b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:54.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.349+0000 7fc1dea00640 1 -- 172.21.15.52:0/440531989 >> v2:172.21.15.102:3300/0 conn(0x7fc1e00b4350 msgr2=0x7fc1e00b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:54.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.349+0000 7fc1dea00640 1 --2- 172.21.15.52:0/440531989 >> v2:172.21.15.102:3300/0 conn(0x7fc1e00b4350 0x7fc1e00b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:54.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.349+0000 7fc1dea00640 1 -- 172.21.15.52:0/440531989 >> v1:172.21.15.102:6789/0 conn(0x7fc1e015cda0 legacy=0x7fc1e00cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:54.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.349+0000 7fc1dea00640 1 -- 172.21.15.52:0/440531989 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e00d0c20 con 0x7fc1e00b5a30 2024-03-12T17:06:54.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.349+0000 7fc1dfe00640 1 --2- 172.21.15.52:0/440531989 >> v2:172.21.15.102:3300/0 conn(0x7fc1e00b4350 0x7fc1e00b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:06:54.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.350+0000 7fc1dea00640 1 --2- 172.21.15.52:0/440531989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1e00b5a30 0x7fc1e00b5e90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc1e00b8460 tx=0x7fc1c802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c240242af90fcd2c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:54.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.351+0000 7fc1de000640 1 -- 172.21.15.52:0/440531989 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c8030860 con 0x7fc1e00b5a30 2024-03-12T17:06:54.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.351+0000 7fc1de000640 1 -- 172.21.15.52:0/440531989 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1c80309c0 con 0x7fc1e00b5a30 2024-03-12T17:06:54.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.351+0000 7fc1de000640 1 -- 172.21.15.52:0/440531989 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c8030d50 con 0x7fc1e00b5a30 2024-03-12T17:06:54.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.351+0000 7fc1e6200640 1 -- 172.21.15.52:0/440531989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1e00b5a30 msgr2=0x7fc1e00b5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:54.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.351+0000 7fc1e6200640 1 --2- 172.21.15.52:0/440531989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1e00b5a30 0x7fc1e00b5e90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc1e00b8460 tx=0x7fc1c802b9a0 comp rx=0 tx=0).stop 2024-03-12T17:06:54.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.351+0000 7fc1e6200640 1 -- 172.21.15.52:0/440531989 shutdown_connections 2024-03-12T17:06:54.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.351+0000 7fc1e6200640 1 --2- 172.21.15.52:0/440531989 >> v2:172.21.15.102:3300/0 conn(0x7fc1e00b4350 0x7fc1e00b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:54.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.351+0000 7fc1e6200640 1 --2- 172.21.15.52:0/440531989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1e00b5a30 0x7fc1e00b5e90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:54.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.351+0000 7fc1e6200640 1 -- 172.21.15.52:0/440531989 >> 172.21.15.52:0/440531989 conn(0x7fc1e00cf350 msgr2=0x7fc1e00cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:54.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.352+0000 7fc1e6200640 1 -- 172.21.15.52:0/440531989 shutdown_connections 2024-03-12T17:06:54.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.352+0000 7fc1e6200640 1 -- 172.21.15.52:0/440531989 wait complete. 2024-03-12T17:06:54.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.354+0000 7fc1e6200640 1 Processor -- start 2024-03-12T17:06:54.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.354+0000 7fc1e6200640 1 -- start start 2024-03-12T17:06:54.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.356+0000 7fc1e6200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1e00b4350 0x7fc1e01570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:54.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.356+0000 7fc1e6200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1e00b5a30 0x7fc1e015e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:54.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.356+0000 7fc1e6200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1e015cda0 0x7fc1e015e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:54.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.356+0000 7fc1e6200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e015dc50 con 0x7fc1e00b4350 2024-03-12T17:06:54.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.356+0000 7fc1e6200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e015f010 con 0x7fc1e00b5a30 2024-03-12T17:06:54.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.356+0000 7fc1e6200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e015d320 con 0x7fc1e015cda0 2024-03-12T17:06:54.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.356+0000 7fc1df400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1e00b4350 0x7fc1e01570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:54.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.356+0000 7fc1df400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1e00b4350 0x7fc1e01570d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53412/0 (socket says 172.21.15.52:53412) 2024-03-12T17:06:54.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.356+0000 7fc1df400640 1 -- 172.21.15.52:0/2520942414 learned_addr learned my addr 172.21.15.52:0/2520942414 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:54.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.356+0000 7fc1dea00640 1 --2- 172.21.15.52:0/2520942414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1e00b5a30 0x7fc1e015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:54.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.356+0000 7fc1dfe00640 1 --2- 172.21.15.52:0/2520942414 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1e015cda0 0x7fc1e015e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:54.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.356+0000 7fc1df400640 1 -- 172.21.15.52:0/2520942414 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1e015cda0 msgr2=0x7fc1e015e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:54.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.356+0000 7fc1df400640 1 --2- 172.21.15.52:0/2520942414 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1e015cda0 0x7fc1e015e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:54.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.357+0000 7fc1df400640 1 -- 172.21.15.52:0/2520942414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1e00b5a30 msgr2=0x7fc1e015e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:54.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.357+0000 7fc1df400640 1 --2- 172.21.15.52:0/2520942414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1e00b5a30 0x7fc1e015e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:54.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.357+0000 7fc1df400640 1 -- 172.21.15.52:0/2520942414 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c80094d0 con 0x7fc1e00b4350 2024-03-12T17:06:54.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.357+0000 7fc1dea00640 1 --2- 172.21.15.52:0/2520942414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1e00b5a30 0x7fc1e015e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:54.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.357+0000 7fc1df400640 1 --2- 172.21.15.52:0/2520942414 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1e00b4350 0x7fc1e01570d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc1d000c400 tx=0x7fc1d000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:54.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.357+0000 7fc1d7400640 1 -- 172.21.15.52:0/2520942414 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d00052d0 con 0x7fc1e00b4350 2024-03-12T17:06:54.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.357+0000 7fc1d7400640 1 -- 172.21.15.52:0/2520942414 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d0005430 con 0x7fc1e00b4350 2024-03-12T17:06:54.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.357+0000 7fc1e6200640 1 -- 172.21.15.52:0/2520942414 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1e015d600 con 0x7fc1e00b4350 2024-03-12T17:06:54.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.357+0000 7fc1d7400640 1 -- 172.21.15.52:0/2520942414 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0003830 con 0x7fc1e00b4350 2024-03-12T17:06:54.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.357+0000 7fc1e6200640 1 -- 172.21.15.52:0/2520942414 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1e01f3230 con 0x7fc1e00b4350 2024-03-12T17:06:54.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.358+0000 7fc1e6200640 1 -- 172.21.15.52:0/2520942414 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1e0073390 con 0x7fc1e00b4350 2024-03-12T17:06:54.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.359+0000 7fc1d7400640 1 -- 172.21.15.52:0/2520942414 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc1d0003990 con 0x7fc1e00b4350 2024-03-12T17:06:54.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.360+0000 7fc1d7400640 1 --2- 172.21.15.52:0/2520942414 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc1bc06ac00 0x7fc1bc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:54.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.360+0000 7fc1d7400640 1 -- 172.21.15.52:0/2520942414 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc1d0088bc0 con 0x7fc1e00b4350 2024-03-12T17:06:54.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.361+0000 7fc1dea00640 1 --2- 172.21.15.52:0/2520942414 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc1bc06ac00 0x7fc1bc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:54.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.362+0000 7fc1dea00640 1 --2- 172.21.15.52:0/2520942414 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc1bc06ac00 0x7fc1bc06d0c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8036000 tx=0x7fc1c803b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:54.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.364+0000 7fc1d7400640 1 -- 172.21.15.52:0/2520942414 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fc1d0053ab0 con 0x7fc1e00b4350 2024-03-12T17:06:54.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.601+0000 7fc1e6200640 1 -- 172.21.15.52:0/2520942414 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1e00b7da0 con 0x7fc1e00b4350 2024-03-12T17:06:54.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.602+0000 7fc1d7400640 1 -- 172.21.15.52:0/2520942414 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4733 (secure 0 0 0) 0x7fc1d0017090 con 0x7fc1e00b4350 2024-03-12T17:06:54.595 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:06:54.596 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2950099092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2950099092},{"type":"v1","addr":"172.21.15.52:6834","nonce":2950099092}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/846108993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":846108993},{"type":"v1","addr":"172.21.15.52:6835","nonce":846108993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6836/3002500679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3002500679},{"type":"v1","addr":"172.21.15.102:6836","nonce":3002500679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-12T17:06:25.905186+0000","modified":"2024-03-12T17:06:54.075752+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6837/3333451504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6835","nonce":3333451504},{"type":"v1","addr":"172.21.15.102:6837","nonce":3333451504}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T17:06:54.596 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 9 2024-03-12T17:06:54.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.604+0000 7fc1e6200640 1 -- 172.21.15.52:0/2520942414 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc1bc06ac00 msgr2=0x7fc1bc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:54.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.604+0000 7fc1e6200640 1 --2- 172.21.15.52:0/2520942414 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc1bc06ac00 0x7fc1bc06d0c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8036000 tx=0x7fc1c803b040 comp rx=0 tx=0).stop 2024-03-12T17:06:54.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.604+0000 7fc1e6200640 1 -- 172.21.15.52:0/2520942414 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1e00b4350 msgr2=0x7fc1e01570d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:54.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.604+0000 7fc1e6200640 1 --2- 172.21.15.52:0/2520942414 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1e00b4350 0x7fc1e01570d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc1d000c400 tx=0x7fc1d000c8c0 comp rx=0 tx=0).stop 2024-03-12T17:06:54.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.604+0000 7fc1e6200640 1 -- 172.21.15.52:0/2520942414 shutdown_connections 2024-03-12T17:06:54.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.604+0000 7fc1e6200640 1 --2- 172.21.15.52:0/2520942414 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc1bc06ac00 0x7fc1bc06d0c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:54.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.604+0000 7fc1e6200640 1 --2- 172.21.15.52:0/2520942414 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1e015cda0 0x7fc1e015e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:54.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.604+0000 7fc1e6200640 1 --2- 172.21.15.52:0/2520942414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1e00b5a30 0x7fc1e015e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:54.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.604+0000 7fc1e6200640 1 --2- 172.21.15.52:0/2520942414 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1e00b4350 0x7fc1e01570d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:54.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.604+0000 7fc1e6200640 1 -- 172.21.15.52:0/2520942414 >> 172.21.15.52:0/2520942414 conn(0x7fc1e00cf350 msgr2=0x7fc1e00cb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:54.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.604+0000 7fc1e6200640 1 -- 172.21.15.52:0/2520942414 shutdown_connections 2024-03-12T17:06:54.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.604+0000 7fc1e6200640 1 -- 172.21.15.52:0/2520942414 wait complete. 2024-03-12T17:06:54.616 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:06:54.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.829+0000 7f1bce200640 1 Processor -- start 2024-03-12T17:06:54.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.829+0000 7f1bce200640 1 -- start start 2024-03-12T17:06:54.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.830+0000 7f1bce200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f1bc815b560 0x7f1bc815b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:54.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.830+0000 7f1bce200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1bc815d410 0x7f1bc80b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:54.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.830+0000 7f1bce200640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bc806c6f0 con 0x7f1bc80b39b0 2024-03-12T17:06:54.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.831+0000 7f1bce200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bc80b4370 con 0x7f1bc815b560 2024-03-12T17:06:54.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.831+0000 7f1bce200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bc806c9d0 con 0x7f1bc815d410 2024-03-12T17:06:54.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.831+0000 7f1bc7400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f1bc815b560 0x7f1bc815b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:54.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.831+0000 7f1bc7400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f1bc815b560 0x7f1bc815b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53428/0 (socket says 172.21.15.52:53428) 2024-03-12T17:06:54.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.831+0000 7f1bc7400640 1 -- 172.21.15.52:0/3414960261 learned_addr learned my addr 172.21.15.52:0/3414960261 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:54.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.831+0000 7f1bbfe00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f1bc80b39b0 0x7f1bc815e500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:57924/0 (socket says 172.21.15.52:57924) 2024-03-12T17:06:54.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.831+0000 7f1bc7e00640 1 --2- 172.21.15.52:0/3414960261 >> v2:172.21.15.102:3300/0 conn(0x7f1bc815d410 0x7f1bc80b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:54.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.831+0000 7f1bc7400640 1 -- 172.21.15.52:0/3414960261 >> v2:172.21.15.102:3300/0 conn(0x7f1bc815d410 msgr2=0x7f1bc80b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:54.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.831+0000 7f1bc7400640 1 --2- 172.21.15.52:0/3414960261 >> v2:172.21.15.102:3300/0 conn(0x7f1bc815d410 0x7f1bc80b3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:54.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.831+0000 7f1bc7400640 1 -- 172.21.15.52:0/3414960261 >> v1:172.21.15.52:6789/0 conn(0x7f1bc80b39b0 legacy=0x7f1bc815e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:54.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.831+0000 7f1bc7400640 1 -- 172.21.15.52:0/3414960261 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc806ccb0 con 0x7f1bc815b560 2024-03-12T17:06:54.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.831+0000 7f1bc6a00640 1 -- 172.21.15.52:0/3414960261 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3141054989 0 0) 0x7f1bc806c6f0 con 0x7f1bc80b39b0 2024-03-12T17:06:54.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.833+0000 7f1bc7400640 1 --2- 172.21.15.52:0/3414960261 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bc815b560 0x7f1bc815b940 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8070100 tx=0x7f1bb802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=374089fcff45161b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:54.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.833+0000 7f1bc6a00640 1 -- 172.21.15.52:0/3414960261 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8031870 con 0x7f1bc815b560 2024-03-12T17:06:54.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.833+0000 7f1bc6a00640 1 -- 172.21.15.52:0/3414960261 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb80319d0 con 0x7f1bc815b560 2024-03-12T17:06:54.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.833+0000 7f1bc6a00640 1 -- 172.21.15.52:0/3414960261 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8031d60 con 0x7f1bc815b560 2024-03-12T17:06:54.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.834+0000 7f1bce200640 1 -- 172.21.15.52:0/3414960261 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bc815b560 msgr2=0x7f1bc815b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:54.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.834+0000 7f1bce200640 1 --2- 172.21.15.52:0/3414960261 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bc815b560 0x7f1bc815b940 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8070100 tx=0x7f1bb802b910 comp rx=0 tx=0).stop 2024-03-12T17:06:54.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.834+0000 7f1bce200640 1 -- 172.21.15.52:0/3414960261 shutdown_connections 2024-03-12T17:06:54.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.834+0000 7f1bce200640 1 --2- 172.21.15.52:0/3414960261 >> v2:172.21.15.102:3300/0 conn(0x7f1bc815d410 0x7f1bc80b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:54.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.834+0000 7f1bce200640 1 --2- 172.21.15.52:0/3414960261 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bc815b560 0x7f1bc815b940 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:54.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.834+0000 7f1bce200640 1 -- 172.21.15.52:0/3414960261 >> 172.21.15.52:0/3414960261 conn(0x7f1bc806a3d0 msgr2=0x7f1bc80b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:54.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.834+0000 7f1bce200640 1 -- 172.21.15.52:0/3414960261 shutdown_connections 2024-03-12T17:06:54.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.834+0000 7f1bce200640 1 -- 172.21.15.52:0/3414960261 wait complete. 2024-03-12T17:06:54.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.836+0000 7f1bce200640 1 Processor -- start 2024-03-12T17:06:54.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.836+0000 7f1bce200640 1 -- start start 2024-03-12T17:06:54.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.837+0000 7f1bce200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bc80b39b0 0x7f1bc81f1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:54.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.837+0000 7f1bce200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1bc815b560 0x7f1bc80af530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:54.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.837+0000 7f1bce200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bc815d410 0x7f1bc8139190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:54.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.837+0000 7f1bce200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc81583e0 con 0x7f1bc815d410 2024-03-12T17:06:54.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.837+0000 7f1bce200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc80ae490 con 0x7f1bc80b39b0 2024-03-12T17:06:54.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.837+0000 7f1bce200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc80ae770 con 0x7f1bc815b560 2024-03-12T17:06:54.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.837+0000 7f1bc7e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bc815d410 0x7f1bc8139190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:54.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.837+0000 7f1bc7e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bc815d410 0x7f1bc8139190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53434/0 (socket says 172.21.15.52:53434) 2024-03-12T17:06:54.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.837+0000 7f1bc7e00640 1 -- 172.21.15.52:0/4261640972 learned_addr learned my addr 172.21.15.52:0/4261640972 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:54.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.837+0000 7f1bc7400640 1 --2- 172.21.15.52:0/4261640972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bc80b39b0 0x7f1bc81f1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:54.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.838+0000 7f1bbfe00640 1 --2- 172.21.15.52:0/4261640972 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1bc815b560 0x7f1bc80af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:54.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.838+0000 7f1bc7e00640 1 -- 172.21.15.52:0/4261640972 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1bc815b560 msgr2=0x7f1bc80af530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:54.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.838+0000 7f1bc7e00640 1 --2- 172.21.15.52:0/4261640972 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1bc815b560 0x7f1bc80af530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:54.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.838+0000 7f1bc7e00640 1 -- 172.21.15.52:0/4261640972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bc80b39b0 msgr2=0x7f1bc81f1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:54.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.838+0000 7f1bc7e00640 1 --2- 172.21.15.52:0/4261640972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bc80b39b0 0x7f1bc81f1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:54.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.838+0000 7f1bc7e00640 1 -- 172.21.15.52:0/4261640972 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb80094d0 con 0x7f1bc815d410 2024-03-12T17:06:54.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.838+0000 7f1bc7400640 1 --2- 172.21.15.52:0/4261640972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bc80b39b0 0x7f1bc81f1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:54.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.838+0000 7f1bc7e00640 1 --2- 172.21.15.52:0/4261640972 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bc815d410 0x7f1bc8139190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0007e60 tx=0x7f1bc000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:54.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.838+0000 7f1bbf400640 1 -- 172.21.15.52:0/4261640972 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0014420 con 0x7f1bc815d410 2024-03-12T17:06:54.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.838+0000 7f1bbf400640 1 -- 172.21.15.52:0/4261640972 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc00145a0 con 0x7f1bc815d410 2024-03-12T17:06:54.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.838+0000 7f1bce200640 1 -- 172.21.15.52:0/4261640972 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bc80aea50 con 0x7f1bc815d410 2024-03-12T17:06:54.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.838+0000 7f1bbf400640 1 -- 172.21.15.52:0/4261640972 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0014950 con 0x7f1bc815d410 2024-03-12T17:06:54.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.838+0000 7f1bce200640 1 -- 172.21.15.52:0/4261640972 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc80af010 con 0x7f1bc815d410 2024-03-12T17:06:54.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.839+0000 7f1bce200640 1 -- 172.21.15.52:0/4261640972 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bc806ac20 con 0x7f1bc815d410 2024-03-12T17:06:54.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.840+0000 7f1bbf400640 1 -- 172.21.15.52:0/4261640972 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1bc0014ab0 con 0x7f1bc815d410 2024-03-12T17:06:54.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.841+0000 7f1bbf400640 1 --2- 172.21.15.52:0/4261640972 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1ba406ac00 0x7f1ba406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:54.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.841+0000 7f1bbf400640 1 -- 172.21.15.52:0/4261640972 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1bc0089ff0 con 0x7f1bc815d410 2024-03-12T17:06:54.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.842+0000 7f1bc7400640 1 --2- 172.21.15.52:0/4261640972 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1ba406ac00 0x7f1ba406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:54.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.843+0000 7f1bc7400640 1 --2- 172.21.15.52:0/4261640972 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1ba406ac00 0x7f1ba406d0c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8036000 tx=0x7f1bb803c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:54.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:54.845+0000 7f1bbf400640 1 -- 172.21.15.52:0/4261640972 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f1bc0054e40 con 0x7f1bc815d410 2024-03-12T17:06:55.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.043+0000 7f1bce200640 1 -- 172.21.15.52:0/4261640972 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1bc80af200 con 0x7f1bc815d410 2024-03-12T17:06:55.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.046+0000 7f1bbf400640 1 -- 172.21.15.52:0/4261640972 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7f1bc0058460 con 0x7f1bc815d410 2024-03-12T17:06:55.040 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:06:55.040 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":25,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:06:30.468476+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T17:06:19.218543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T17:06:23.525474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T17:06:24.911574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-12T17:06:28.017172+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.733031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.775178+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.591522+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.820838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.829252+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.582481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:06:55.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.048+0000 7f1bce200640 1 -- 172.21.15.52:0/4261640972 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1ba406ac00 msgr2=0x7f1ba406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:55.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.048+0000 7f1bce200640 1 --2- 172.21.15.52:0/4261640972 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1ba406ac00 0x7f1ba406d0c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8036000 tx=0x7f1bb803c040 comp rx=0 tx=0).stop 2024-03-12T17:06:55.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.048+0000 7f1bce200640 1 -- 172.21.15.52:0/4261640972 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bc815d410 msgr2=0x7f1bc8139190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:55.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.048+0000 7f1bce200640 1 --2- 172.21.15.52:0/4261640972 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bc815d410 0x7f1bc8139190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0007e60 tx=0x7f1bc000e4b0 comp rx=0 tx=0).stop 2024-03-12T17:06:55.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.048+0000 7f1bce200640 1 -- 172.21.15.52:0/4261640972 shutdown_connections 2024-03-12T17:06:55.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.048+0000 7f1bce200640 1 --2- 172.21.15.52:0/4261640972 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1ba406ac00 0x7f1ba406d0c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:55.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.048+0000 7f1bce200640 1 --2- 172.21.15.52:0/4261640972 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1bc815b560 0x7f1bc80af530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:55.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.048+0000 7f1bce200640 1 --2- 172.21.15.52:0/4261640972 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bc815d410 0x7f1bc8139190 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:55.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.048+0000 7f1bce200640 1 --2- 172.21.15.52:0/4261640972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bc80b39b0 0x7f1bc81f1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:55.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.048+0000 7f1bce200640 1 -- 172.21.15.52:0/4261640972 >> 172.21.15.52:0/4261640972 conn(0x7f1bc806a3d0 msgr2=0x7f1bc80b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:55.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.048+0000 7f1bce200640 1 -- 172.21.15.52:0/4261640972 shutdown_connections 2024-03-12T17:06:55.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.049+0000 7f1bce200640 1 -- 172.21.15.52:0/4261640972 wait complete. 2024-03-12T17:06:55.060 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-12T17:06:55.060 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-12T17:06:55.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.276+0000 7fcc45800640 1 Processor -- start 2024-03-12T17:06:55.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.277+0000 7fcc45800640 1 -- start start 2024-03-12T17:06:55.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.278+0000 7fcc45800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fcc40098a40 0x7fcc4006be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:55.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.278+0000 7fcc45800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc4015ccd0 0x7fcc40098e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:55.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.278+0000 7fcc45800640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fcc400cf3c0 0x7fcc400cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:55.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.278+0000 7fcc45800640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc4006c380 con 0x7fcc400cf3c0 2024-03-12T17:06:55.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.278+0000 7fcc45800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc40099360 con 0x7fcc40098a40 2024-03-12T17:06:55.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.278+0000 7fcc45800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4006ada0 con 0x7fcc4015ccd0 2024-03-12T17:06:55.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.278+0000 7fcc3f400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fcc400cf3c0 0x7fcc400cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:55.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.279+0000 7fcc3f400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fcc400cf3c0 0x7fcc400cf850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60684/0 (socket says 172.21.15.52:60684) 2024-03-12T17:06:55.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.279+0000 7fcc3f400640 1 -- 172.21.15.52:0/1172840356 learned_addr learned my addr 172.21.15.52:0/1172840356 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:55.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.279+0000 7fcc3e000640 1 --2- 172.21.15.52:0/1172840356 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc4015ccd0 0x7fcc40098e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:55.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.279+0000 7fcc3ea00640 1 --2- 172.21.15.52:0/1172840356 >> v2:172.21.15.102:3300/0 conn(0x7fcc40098a40 0x7fcc4006be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:55.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.279+0000 7fcc3f400640 1 -- 172.21.15.52:0/1172840356 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc4015ccd0 msgr2=0x7fcc40098e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:55.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.279+0000 7fcc3f400640 1 --2- 172.21.15.52:0/1172840356 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc4015ccd0 0x7fcc40098e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:55.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.279+0000 7fcc3f400640 1 -- 172.21.15.52:0/1172840356 >> v2:172.21.15.102:3300/0 conn(0x7fcc40098a40 msgr2=0x7fcc4006be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:55.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.279+0000 7fcc3f400640 1 --2- 172.21.15.52:0/1172840356 >> v2:172.21.15.102:3300/0 conn(0x7fcc40098a40 0x7fcc4006be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:55.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.279+0000 7fcc3f400640 1 -- 172.21.15.52:0/1172840356 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc4006b080 con 0x7fcc400cf3c0 2024-03-12T17:06:55.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.280+0000 7fcc3f400640 1 --2- 172.21.15.52:0/1172840356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcc400cf3c0 0x7fcc400cf850 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcc400b31e0 tx=0x7fcc3802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed86c5aea04b441 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:55.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.281+0000 7fcc3d600640 1 -- 172.21.15.52:0/1172840356 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3802c9f0 con 0x7fcc400cf3c0 2024-03-12T17:06:55.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.281+0000 7fcc3d600640 1 -- 172.21.15.52:0/1172840356 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc3802cb50 con 0x7fcc400cf3c0 2024-03-12T17:06:55.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.281+0000 7fcc3d600640 1 -- 172.21.15.52:0/1172840356 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc380343f0 con 0x7fcc400cf3c0 2024-03-12T17:06:55.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.281+0000 7fcc45800640 1 -- 172.21.15.52:0/1172840356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcc400cf3c0 msgr2=0x7fcc400cf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:55.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.281+0000 7fcc45800640 1 --2- 172.21.15.52:0/1172840356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcc400cf3c0 0x7fcc400cf850 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcc400b31e0 tx=0x7fcc3802be50 comp rx=0 tx=0).stop 2024-03-12T17:06:55.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.281+0000 7fcc45800640 1 -- 172.21.15.52:0/1172840356 shutdown_connections 2024-03-12T17:06:55.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.281+0000 7fcc45800640 1 --2- 172.21.15.52:0/1172840356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcc400cf3c0 0x7fcc400cf850 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:55.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.281+0000 7fcc45800640 1 --2- 172.21.15.52:0/1172840356 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc4015ccd0 0x7fcc40098e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:55.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.281+0000 7fcc45800640 1 --2- 172.21.15.52:0/1172840356 >> v2:172.21.15.102:3300/0 conn(0x7fcc40098a40 0x7fcc4006be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:55.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.281+0000 7fcc45800640 1 -- 172.21.15.52:0/1172840356 >> 172.21.15.52:0/1172840356 conn(0x7fcc400b2160 msgr2=0x7fcc400b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:55.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.282+0000 7fcc45800640 1 -- 172.21.15.52:0/1172840356 shutdown_connections 2024-03-12T17:06:55.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.282+0000 7fcc45800640 1 -- 172.21.15.52:0/1172840356 wait complete. 2024-03-12T17:06:55.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.283+0000 7fcc45800640 1 Processor -- start 2024-03-12T17:06:55.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.283+0000 7fcc45800640 1 -- start start 2024-03-12T17:06:55.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.284+0000 7fcc45800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc40098a40 0x7fcc400b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:55.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.284+0000 7fcc45800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc400cf3c0 0x7fcc401f1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:55.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.284+0000 7fcc45800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcc4015ccd0 0x7fcc4015f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:55.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.284+0000 7fcc45800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc40154620 con 0x7fcc4015ccd0 2024-03-12T17:06:55.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.284+0000 7fcc45800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc401f21b0 con 0x7fcc40098a40 2024-03-12T17:06:55.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.284+0000 7fcc45800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc400c41a0 con 0x7fcc400cf3c0 2024-03-12T17:06:55.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.285+0000 7fcc3f400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcc4015ccd0 0x7fcc4015f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:55.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.285+0000 7fcc3f400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcc4015ccd0 0x7fcc4015f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60690/0 (socket says 172.21.15.52:60690) 2024-03-12T17:06:55.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.285+0000 7fcc3f400640 1 -- 172.21.15.52:0/3269896580 learned_addr learned my addr 172.21.15.52:0/3269896580 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:55.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.285+0000 7fcc3e000640 1 --2- 172.21.15.52:0/3269896580 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc400cf3c0 0x7fcc401f1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:55.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.285+0000 7fcc3ea00640 1 --2- 172.21.15.52:0/3269896580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc40098a40 0x7fcc400b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:55.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.285+0000 7fcc3f400640 1 -- 172.21.15.52:0/3269896580 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc400cf3c0 msgr2=0x7fcc401f1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:55.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.285+0000 7fcc3f400640 1 --2- 172.21.15.52:0/3269896580 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc400cf3c0 0x7fcc401f1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:55.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.285+0000 7fcc3f400640 1 -- 172.21.15.52:0/3269896580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc40098a40 msgr2=0x7fcc400b36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:55.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.285+0000 7fcc3f400640 1 --2- 172.21.15.52:0/3269896580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc40098a40 0x7fcc400b36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:55.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.285+0000 7fcc3f400640 1 -- 172.21.15.52:0/3269896580 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc38009b30 con 0x7fcc4015ccd0 2024-03-12T17:06:55.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.285+0000 7fcc3f400640 1 --2- 172.21.15.52:0/3269896580 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcc4015ccd0 0x7fcc4015f020 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcc38036040 tx=0x7fcc38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:55.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.286+0000 7fcc36a00640 1 -- 172.21.15.52:0/3269896580 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc38009510 con 0x7fcc4015ccd0 2024-03-12T17:06:55.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.286+0000 7fcc36a00640 1 -- 172.21.15.52:0/3269896580 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc3803e450 con 0x7fcc4015ccd0 2024-03-12T17:06:55.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.286+0000 7fcc36a00640 1 -- 172.21.15.52:0/3269896580 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3803e800 con 0x7fcc4015ccd0 2024-03-12T17:06:55.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.286+0000 7fcc45800640 1 -- 172.21.15.52:0/3269896580 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc400c43f0 con 0x7fcc4015ccd0 2024-03-12T17:06:55.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.286+0000 7fcc45800640 1 -- 172.21.15.52:0/3269896580 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc400c49b0 con 0x7fcc4015ccd0 2024-03-12T17:06:55.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.287+0000 7fcc45800640 1 -- 172.21.15.52:0/3269896580 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc40073390 con 0x7fcc4015ccd0 2024-03-12T17:06:55.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.288+0000 7fcc36a00640 1 -- 172.21.15.52:0/3269896580 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fcc3803e960 con 0x7fcc4015ccd0 2024-03-12T17:06:55.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.289+0000 7fcc36a00640 1 --2- 172.21.15.52:0/3269896580 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fcc2006ac00 0x7fcc2006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:55.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.289+0000 7fcc36a00640 1 -- 172.21.15.52:0/3269896580 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fcc380ac480 con 0x7fcc4015ccd0 2024-03-12T17:06:55.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.289+0000 7fcc3ea00640 1 --2- 172.21.15.52:0/3269896580 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fcc2006ac00 0x7fcc2006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:55.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.290+0000 7fcc3ea00640 1 --2- 172.21.15.52:0/3269896580 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fcc2006ac00 0x7fcc2006d0c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcc30006fd0 tx=0x7fcc30008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:55.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.293+0000 7fcc36a00640 1 -- 172.21.15.52:0/3269896580 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fcc380772d0 con 0x7fcc4015ccd0 2024-03-12T17:06:55.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.491+0000 7fcc45800640 1 -- 172.21.15.52:0/3269896580 --> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcc4006b920 con 0x7fcc2006ac00 2024-03-12T17:06:55.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.523+0000 7fcc36a00640 1 -- 172.21.15.52:0/3269896580 <== mgr.4109 v2:172.21.15.102:6832/36622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345051 (secure 0 0 0) 0x7fcc3000f830 con 0x7fcc2006ac00 2024-03-12T17:06:55.522 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:06:55.524 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-03-12T17:06:55.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.530+0000 7fcc45800640 1 -- 172.21.15.52:0/3269896580 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fcc2006ac00 msgr2=0x7fcc2006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:55.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.530+0000 7fcc45800640 1 --2- 172.21.15.52:0/3269896580 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fcc2006ac00 0x7fcc2006d0c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcc30006fd0 tx=0x7fcc30008040 comp rx=0 tx=0).stop 2024-03-12T17:06:55.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.530+0000 7fcc45800640 1 -- 172.21.15.52:0/3269896580 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcc4015ccd0 msgr2=0x7fcc4015f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:55.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.530+0000 7fcc45800640 1 --2- 172.21.15.52:0/3269896580 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcc4015ccd0 0x7fcc4015f020 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcc38036040 tx=0x7fcc38004a40 comp rx=0 tx=0).stop 2024-03-12T17:06:55.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.530+0000 7fcc45800640 1 -- 172.21.15.52:0/3269896580 shutdown_connections 2024-03-12T17:06:55.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.530+0000 7fcc45800640 1 --2- 172.21.15.52:0/3269896580 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fcc2006ac00 0x7fcc2006d0c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:55.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.530+0000 7fcc45800640 1 --2- 172.21.15.52:0/3269896580 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcc400cf3c0 0x7fcc401f1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:55.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.530+0000 7fcc45800640 1 --2- 172.21.15.52:0/3269896580 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcc4015ccd0 0x7fcc4015f020 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:55.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.530+0000 7fcc45800640 1 --2- 172.21.15.52:0/3269896580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc40098a40 0x7fcc400b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:55.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.530+0000 7fcc45800640 1 -- 172.21.15.52:0/3269896580 >> 172.21.15.52:0/3269896580 conn(0x7fcc400b2160 msgr2=0x7fcc400726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:55.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.530+0000 7fcc45800640 1 -- 172.21.15.52:0/3269896580 shutdown_connections 2024-03-12T17:06:55.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.530+0000 7fcc45800640 1 -- 172.21.15.52:0/3269896580 wait complete. 2024-03-12T17:06:55.545 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-03-12T17:06:55.157508+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594961,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231808,"kb_used_data":3304,"kb_used_omap":13,"kb_used_meta":228338,"kb_avail":754742912,"statfs":{"total":773094113280,"available":772856741888,"internally_reserved":0,"allocated":3383296,"data_stored":2025530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":233818200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005387"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.999570+0000","last_change":"2024-03-12T17:06:25.442740+0000","last_active":"2024-03-12T17:06:29.999570+0000","last_peered":"2024-03-12T17:06:29.999570+0000","last_clean":"2024-03-12T17:06:29.999570+0000","last_became_active":"2024-03-12T17:06:25.438608+0000","last_became_peered":"2024-03-12T17:06:25.438608+0000","last_unstale":"2024-03-12T17:06:29.999570+0000","last_undegraded":"2024-03-12T17:06:29.999570+0000","last_fullsized":"2024-03-12T17:06:29.999570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:59:41.676210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.004064+0000","last_change":"2024-03-12T17:06:25.939770+0000","last_active":"2024-03-12T17:06:30.004064+0000","last_peered":"2024-03-12T17:06:30.004064+0000","last_clean":"2024-03-12T17:06:30.004064+0000","last_became_active":"2024-03-12T17:06:25.938969+0000","last_became_peered":"2024-03-12T17:06:25.938969+0000","last_unstale":"2024-03-12T17:06:30.004064+0000","last_undegraded":"2024-03-12T17:06:30.004064+0000","last_fullsized":"2024-03-12T17:06:30.004064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:23:25.984714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.476841+0000","last_change":"2024-03-12T17:06:30.023660+0000","last_active":"2024-03-12T17:06:30.476841+0000","last_peered":"2024-03-12T17:06:30.476841+0000","last_clean":"2024-03-12T17:06:30.476841+0000","last_became_active":"2024-03-12T17:06:30.022948+0000","last_became_peered":"2024-03-12T17:06:30.022948+0000","last_unstale":"2024-03-12T17:06:30.476841+0000","last_undegraded":"2024-03-12T17:06:30.476841+0000","last_fullsized":"2024-03-12T17:06:30.476841+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:04:13.473422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.003821+0000","last_change":"2024-03-12T17:06:25.446629+0000","last_active":"2024-03-12T17:06:30.003821+0000","last_peered":"2024-03-12T17:06:30.003821+0000","last_clean":"2024-03-12T17:06:30.003821+0000","last_became_active":"2024-03-12T17:06:25.442784+0000","last_became_peered":"2024-03-12T17:06:25.442784+0000","last_unstale":"2024-03-12T17:06:30.003821+0000","last_undegraded":"2024-03-12T17:06:30.003821+0000","last_fullsized":"2024-03-12T17:06:30.003821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:59:41.676210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.999425+0000","last_change":"2024-03-12T17:06:25.953462+0000","last_active":"2024-03-12T17:06:29.999425+0000","last_peered":"2024-03-12T17:06:29.999425+0000","last_clean":"2024-03-12T17:06:29.999425+0000","last_became_active":"2024-03-12T17:06:25.953198+0000","last_became_peered":"2024-03-12T17:06:25.953198+0000","last_unstale":"2024-03-12T17:06:29.999425+0000","last_undegraded":"2024-03-12T17:06:29.999425+0000","last_fullsized":"2024-03-12T17:06:29.999425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:52:14.405796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.480606+0000","last_change":"2024-03-12T17:06:30.017287+0000","last_active":"2024-03-12T17:06:30.480606+0000","last_peered":"2024-03-12T17:06:30.480606+0000","last_clean":"2024-03-12T17:06:30.480606+0000","last_became_active":"2024-03-12T17:06:30.016834+0000","last_became_peered":"2024-03-12T17:06:30.016834+0000","last_unstale":"2024-03-12T17:06:30.480606+0000","last_undegraded":"2024-03-12T17:06:30.480606+0000","last_fullsized":"2024-03-12T17:06:30.480606+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:21:26.853333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012212+0000","last_change":"2024-03-12T17:06:25.454124+0000","last_active":"2024-03-12T17:06:30.012212+0000","last_peered":"2024-03-12T17:06:30.012212+0000","last_clean":"2024-03-12T17:06:30.012212+0000","last_became_active":"2024-03-12T17:06:25.453877+0000","last_became_peered":"2024-03-12T17:06:25.453877+0000","last_unstale":"2024-03-12T17:06:30.012212+0000","last_undegraded":"2024-03-12T17:06:30.012212+0000","last_fullsized":"2024-03-12T17:06:30.012212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:45:01.926495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.991068+0000","last_change":"2024-03-12T17:06:25.926772+0000","last_active":"2024-03-12T17:06:29.991068+0000","last_peered":"2024-03-12T17:06:29.991068+0000","last_clean":"2024-03-12T17:06:29.991068+0000","last_became_active":"2024-03-12T17:06:25.926486+0000","last_became_peered":"2024-03-12T17:06:25.926486+0000","last_unstale":"2024-03-12T17:06:29.991068+0000","last_undegraded":"2024-03-12T17:06:29.991068+0000","last_fullsized":"2024-03-12T17:06:29.991068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:28:58.739417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.482311+0000","last_change":"2024-03-12T17:06:30.028104+0000","last_active":"2024-03-12T17:06:30.482311+0000","last_peered":"2024-03-12T17:06:30.482311+0000","last_clean":"2024-03-12T17:06:30.482311+0000","last_became_active":"2024-03-12T17:06:30.027645+0000","last_became_peered":"2024-03-12T17:06:30.027645+0000","last_unstale":"2024-03-12T17:06:30.482311+0000","last_undegraded":"2024-03-12T17:06:30.482311+0000","last_fullsized":"2024-03-12T17:06:30.482311+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:13:55.311599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.011875+0000","last_change":"2024-03-12T17:06:25.455911+0000","last_active":"2024-03-12T17:06:30.011875+0000","last_peered":"2024-03-12T17:06:30.011875+0000","last_clean":"2024-03-12T17:06:30.011875+0000","last_became_active":"2024-03-12T17:06:25.455251+0000","last_became_peered":"2024-03-12T17:06:25.455251+0000","last_unstale":"2024-03-12T17:06:30.011875+0000","last_undegraded":"2024-03-12T17:06:30.011875+0000","last_fullsized":"2024-03-12T17:06:30.011875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:36:25.468644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990029+0000","last_change":"2024-03-12T17:06:25.940304+0000","last_active":"2024-03-12T17:06:29.990029+0000","last_peered":"2024-03-12T17:06:29.990029+0000","last_clean":"2024-03-12T17:06:29.990029+0000","last_became_active":"2024-03-12T17:06:25.940019+0000","last_became_peered":"2024-03-12T17:06:25.940019+0000","last_unstale":"2024-03-12T17:06:29.990029+0000","last_undegraded":"2024-03-12T17:06:29.990029+0000","last_fullsized":"2024-03-12T17:06:29.990029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:44:34.582031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.481007+0000","last_change":"2024-03-12T17:06:30.030105+0000","last_active":"2024-03-12T17:06:30.481007+0000","last_peered":"2024-03-12T17:06:30.481007+0000","last_clean":"2024-03-12T17:06:30.481007+0000","last_became_active":"2024-03-12T17:06:30.029702+0000","last_became_peered":"2024-03-12T17:06:30.029702+0000","last_unstale":"2024-03-12T17:06:30.481007+0000","last_undegraded":"2024-03-12T17:06:30.481007+0000","last_fullsized":"2024-03-12T17:06:30.481007+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:33:44.182800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.986543+0000","last_change":"2024-03-12T17:06:25.942985+0000","last_active":"2024-03-12T17:06:29.986543+0000","last_peered":"2024-03-12T17:06:29.986543+0000","last_clean":"2024-03-12T17:06:29.986543+0000","last_became_active":"2024-03-12T17:06:25.942526+0000","last_became_peered":"2024-03-12T17:06:25.942526+0000","last_unstale":"2024-03-12T17:06:29.986543+0000","last_undegraded":"2024-03-12T17:06:29.986543+0000","last_fullsized":"2024-03-12T17:06:29.986543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:44:25.750833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012322+0000","last_change":"2024-03-12T17:06:25.453557+0000","last_active":"2024-03-12T17:06:30.012322+0000","last_peered":"2024-03-12T17:06:30.012322+0000","last_clean":"2024-03-12T17:06:30.012322+0000","last_became_active":"2024-03-12T17:06:25.451663+0000","last_became_peered":"2024-03-12T17:06:25.451663+0000","last_unstale":"2024-03-12T17:06:30.012322+0000","last_undegraded":"2024-03-12T17:06:30.012322+0000","last_fullsized":"2024-03-12T17:06:30.012322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:36:47.407275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.482348+0000","last_change":"2024-03-12T17:06:30.018487+0000","last_active":"2024-03-12T17:06:30.482348+0000","last_peered":"2024-03-12T17:06:30.482348+0000","last_clean":"2024-03-12T17:06:30.482348+0000","last_became_active":"2024-03-12T17:06:30.017933+0000","last_became_peered":"2024-03-12T17:06:30.017933+0000","last_unstale":"2024-03-12T17:06:30.482348+0000","last_undegraded":"2024-03-12T17:06:30.482348+0000","last_fullsized":"2024-03-12T17:06:30.482348+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:42:49.327905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.995828+0000","last_change":"2024-03-12T17:06:25.952715+0000","last_active":"2024-03-12T17:06:29.995828+0000","last_peered":"2024-03-12T17:06:29.995828+0000","last_clean":"2024-03-12T17:06:29.995828+0000","last_became_active":"2024-03-12T17:06:25.952288+0000","last_became_peered":"2024-03-12T17:06:25.952288+0000","last_unstale":"2024-03-12T17:06:29.995828+0000","last_undegraded":"2024-03-12T17:06:29.995828+0000","last_fullsized":"2024-03-12T17:06:29.995828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:29:49.461651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.015251+0000","last_change":"2024-03-12T17:06:25.448043+0000","last_active":"2024-03-12T17:06:30.015251+0000","last_peered":"2024-03-12T17:06:30.015251+0000","last_clean":"2024-03-12T17:06:30.015251+0000","last_became_active":"2024-03-12T17:06:25.447665+0000","last_became_peered":"2024-03-12T17:06:25.447665+0000","last_unstale":"2024-03-12T17:06:30.015251+0000","last_undegraded":"2024-03-12T17:06:30.015251+0000","last_fullsized":"2024-03-12T17:06:30.015251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:28:25.809578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.481001+0000","last_change":"2024-03-12T17:06:30.020157+0000","last_active":"2024-03-12T17:06:30.481001+0000","last_peered":"2024-03-12T17:06:30.481001+0000","last_clean":"2024-03-12T17:06:30.481001+0000","last_became_active":"2024-03-12T17:06:30.019631+0000","last_became_peered":"2024-03-12T17:06:30.019631+0000","last_unstale":"2024-03-12T17:06:30.481001+0000","last_undegraded":"2024-03-12T17:06:30.481001+0000","last_fullsized":"2024-03-12T17:06:30.481001+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:46:40.007538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.987316+0000","last_change":"2024-03-12T17:06:25.925596+0000","last_active":"2024-03-12T17:06:29.987316+0000","last_peered":"2024-03-12T17:06:29.987316+0000","last_clean":"2024-03-12T17:06:29.987316+0000","last_became_active":"2024-03-12T17:06:25.925225+0000","last_became_peered":"2024-03-12T17:06:25.925225+0000","last_unstale":"2024-03-12T17:06:29.987316+0000","last_undegraded":"2024-03-12T17:06:29.987316+0000","last_fullsized":"2024-03-12T17:06:29.987316+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:44:34.582031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.014268+0000","last_change":"2024-03-12T17:06:25.450336+0000","last_active":"2024-03-12T17:06:30.014268+0000","last_peered":"2024-03-12T17:06:30.014268+0000","last_clean":"2024-03-12T17:06:30.014268+0000","last_became_active":"2024-03-12T17:06:25.450049+0000","last_became_peered":"2024-03-12T17:06:25.450049+0000","last_unstale":"2024-03-12T17:06:30.014268+0000","last_undegraded":"2024-03-12T17:06:30.014268+0000","last_fullsized":"2024-03-12T17:06:30.014268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:00:06.114876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.511116+0000","last_change":"2024-03-12T17:06:30.024652+0000","last_active":"2024-03-12T17:06:30.511116+0000","last_peered":"2024-03-12T17:06:30.511116+0000","last_clean":"2024-03-12T17:06:30.511116+0000","last_became_active":"2024-03-12T17:06:30.024156+0000","last_became_peered":"2024-03-12T17:06:30.024156+0000","last_unstale":"2024-03-12T17:06:30.511116+0000","last_undegraded":"2024-03-12T17:06:30.511116+0000","last_fullsized":"2024-03-12T17:06:30.511116+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:21:26.853333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012195+0000","last_change":"2024-03-12T17:06:25.935633+0000","last_active":"2024-03-12T17:06:30.012195+0000","last_peered":"2024-03-12T17:06:30.012195+0000","last_clean":"2024-03-12T17:06:30.012195+0000","last_became_active":"2024-03-12T17:06:25.935322+0000","last_became_peered":"2024-03-12T17:06:25.935322+0000","last_unstale":"2024-03-12T17:06:30.012195+0000","last_undegraded":"2024-03-12T17:06:30.012195+0000","last_fullsized":"2024-03-12T17:06:30.012195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:34:27.281126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.014503+0000","last_change":"2024-03-12T17:06:25.456881+0000","last_active":"2024-03-12T17:06:30.014503+0000","last_peered":"2024-03-12T17:06:30.014503+0000","last_clean":"2024-03-12T17:06:30.014503+0000","last_became_active":"2024-03-12T17:06:25.456425+0000","last_became_peered":"2024-03-12T17:06:25.456425+0000","last_unstale":"2024-03-12T17:06:30.014503+0000","last_undegraded":"2024-03-12T17:06:30.014503+0000","last_fullsized":"2024-03-12T17:06:30.014503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:33:42.671579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.482388+0000","last_change":"2024-03-12T17:06:30.021976+0000","last_active":"2024-03-12T17:06:30.482388+0000","last_peered":"2024-03-12T17:06:30.482388+0000","last_clean":"2024-03-12T17:06:30.482388+0000","last_became_active":"2024-03-12T17:06:30.021512+0000","last_became_peered":"2024-03-12T17:06:30.021512+0000","last_unstale":"2024-03-12T17:06:30.482388+0000","last_undegraded":"2024-03-12T17:06:30.482388+0000","last_fullsized":"2024-03-12T17:06:30.482388+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:31:47.699683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.999227+0000","last_change":"2024-03-12T17:06:25.937491+0000","last_active":"2024-03-12T17:06:29.999227+0000","last_peered":"2024-03-12T17:06:29.999227+0000","last_clean":"2024-03-12T17:06:29.999227+0000","last_became_active":"2024-03-12T17:06:25.937173+0000","last_became_peered":"2024-03-12T17:06:25.937173+0000","last_unstale":"2024-03-12T17:06:29.999227+0000","last_undegraded":"2024-03-12T17:06:29.999227+0000","last_fullsized":"2024-03-12T17:06:29.999227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:29:49.461651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990464+0000","last_change":"2024-03-12T17:06:25.450121+0000","last_active":"2024-03-12T17:06:29.990464+0000","last_peered":"2024-03-12T17:06:29.990464+0000","last_clean":"2024-03-12T17:06:29.990464+0000","last_became_active":"2024-03-12T17:06:25.448549+0000","last_became_peered":"2024-03-12T17:06:25.448549+0000","last_unstale":"2024-03-12T17:06:29.990464+0000","last_undegraded":"2024-03-12T17:06:29.990464+0000","last_fullsized":"2024-03-12T17:06:29.990464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:36:47.407275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.481700+0000","last_change":"2024-03-12T17:06:30.020438+0000","last_active":"2024-03-12T17:06:30.481700+0000","last_peered":"2024-03-12T17:06:30.481700+0000","last_clean":"2024-03-12T17:06:30.481700+0000","last_became_active":"2024-03-12T17:06:30.020055+0000","last_became_peered":"2024-03-12T17:06:30.020055+0000","last_unstale":"2024-03-12T17:06:30.481700+0000","last_undegraded":"2024-03-12T17:06:30.481700+0000","last_fullsized":"2024-03-12T17:06:30.481700+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:17:09.055890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.004123+0000","last_change":"2024-03-12T17:06:25.947199+0000","last_active":"2024-03-12T17:06:30.004123+0000","last_peered":"2024-03-12T17:06:30.004123+0000","last_clean":"2024-03-12T17:06:30.004123+0000","last_became_active":"2024-03-12T17:06:25.941750+0000","last_became_peered":"2024-03-12T17:06:25.941750+0000","last_unstale":"2024-03-12T17:06:30.004123+0000","last_undegraded":"2024-03-12T17:06:30.004123+0000","last_fullsized":"2024-03-12T17:06:30.004123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:04:04.198678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.021973+0000","last_change":"2024-03-12T17:06:25.452843+0000","last_active":"2024-03-12T17:06:30.021973+0000","last_peered":"2024-03-12T17:06:30.021973+0000","last_clean":"2024-03-12T17:06:30.021973+0000","last_became_active":"2024-03-12T17:06:25.452410+0000","last_became_peered":"2024-03-12T17:06:25.452410+0000","last_unstale":"2024-03-12T17:06:30.021973+0000","last_undegraded":"2024-03-12T17:06:30.021973+0000","last_fullsized":"2024-03-12T17:06:30.021973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:36:25.468644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.476561+0000","last_change":"2024-03-12T17:06:30.028820+0000","last_active":"2024-03-12T17:06:30.476561+0000","last_peered":"2024-03-12T17:06:30.476561+0000","last_clean":"2024-03-12T17:06:30.476561+0000","last_became_active":"2024-03-12T17:06:30.028425+0000","last_became_peered":"2024-03-12T17:06:30.028425+0000","last_unstale":"2024-03-12T17:06:30.476561+0000","last_undegraded":"2024-03-12T17:06:30.476561+0000","last_fullsized":"2024-03-12T17:06:30.476561+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:46:40.007538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.019530+0000","last_change":"2024-03-12T17:06:25.929758+0000","last_active":"2024-03-12T17:06:30.019530+0000","last_peered":"2024-03-12T17:06:30.019530+0000","last_clean":"2024-03-12T17:06:30.019530+0000","last_became_active":"2024-03-12T17:06:25.929481+0000","last_became_peered":"2024-03-12T17:06:25.929481+0000","last_unstale":"2024-03-12T17:06:30.019530+0000","last_undegraded":"2024-03-12T17:06:30.019530+0000","last_fullsized":"2024-03-12T17:06:30.019530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:44:25.750833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012765+0000","last_change":"2024-03-12T17:06:25.456024+0000","last_active":"2024-03-12T17:06:30.012765+0000","last_peered":"2024-03-12T17:06:30.012765+0000","last_clean":"2024-03-12T17:06:30.012765+0000","last_became_active":"2024-03-12T17:06:25.455718+0000","last_became_peered":"2024-03-12T17:06:25.455718+0000","last_unstale":"2024-03-12T17:06:30.012765+0000","last_undegraded":"2024-03-12T17:06:30.012765+0000","last_fullsized":"2024-03-12T17:06:30.012765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:00:06.114876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.508448+0000","last_change":"2024-03-12T17:06:30.020641+0000","last_active":"2024-03-12T17:06:30.508448+0000","last_peered":"2024-03-12T17:06:30.508448+0000","last_clean":"2024-03-12T17:06:30.508448+0000","last_became_active":"2024-03-12T17:06:30.020373+0000","last_became_peered":"2024-03-12T17:06:30.020373+0000","last_unstale":"2024-03-12T17:06:30.508448+0000","last_undegraded":"2024-03-12T17:06:30.508448+0000","last_fullsized":"2024-03-12T17:06:30.508448+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:03:49.781309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990659+0000","last_change":"2024-03-12T17:06:25.943799+0000","last_active":"2024-03-12T17:06:29.990659+0000","last_peered":"2024-03-12T17:06:29.990659+0000","last_clean":"2024-03-12T17:06:29.990659+0000","last_became_active":"2024-03-12T17:06:25.943543+0000","last_became_peered":"2024-03-12T17:06:25.943543+0000","last_unstale":"2024-03-12T17:06:29.990659+0000","last_undegraded":"2024-03-12T17:06:29.990659+0000","last_fullsized":"2024-03-12T17:06:29.990659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:38:51.036295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.019581+0000","last_change":"2024-03-12T17:06:25.456899+0000","last_active":"2024-03-12T17:06:30.019581+0000","last_peered":"2024-03-12T17:06:30.019581+0000","last_clean":"2024-03-12T17:06:30.019581+0000","last_became_active":"2024-03-12T17:06:25.456513+0000","last_became_peered":"2024-03-12T17:06:25.456513+0000","last_unstale":"2024-03-12T17:06:30.019581+0000","last_undegraded":"2024-03-12T17:06:30.019581+0000","last_fullsized":"2024-03-12T17:06:30.019581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:58:32.509940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.508585+0000","last_change":"2024-03-12T17:06:30.018815+0000","last_active":"2024-03-12T17:06:30.508585+0000","last_peered":"2024-03-12T17:06:30.508585+0000","last_clean":"2024-03-12T17:06:30.508585+0000","last_became_active":"2024-03-12T17:06:30.014225+0000","last_became_peered":"2024-03-12T17:06:30.014225+0000","last_unstale":"2024-03-12T17:06:30.508585+0000","last_undegraded":"2024-03-12T17:06:30.508585+0000","last_fullsized":"2024-03-12T17:06:30.508585+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:49:38.782632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012440+0000","last_change":"2024-03-12T17:06:25.954609+0000","last_active":"2024-03-12T17:06:30.012440+0000","last_peered":"2024-03-12T17:06:30.012440+0000","last_clean":"2024-03-12T17:06:30.012440+0000","last_became_active":"2024-03-12T17:06:25.954337+0000","last_became_peered":"2024-03-12T17:06:25.954337+0000","last_unstale":"2024-03-12T17:06:30.012440+0000","last_undegraded":"2024-03-12T17:06:30.012440+0000","last_fullsized":"2024-03-12T17:06:30.012440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:33:43.674985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.992408+0000","last_change":"2024-03-12T17:06:25.442368+0000","last_active":"2024-03-12T17:06:29.992408+0000","last_peered":"2024-03-12T17:06:29.992408+0000","last_clean":"2024-03-12T17:06:29.992408+0000","last_became_active":"2024-03-12T17:06:25.442084+0000","last_became_peered":"2024-03-12T17:06:25.442084+0000","last_unstale":"2024-03-12T17:06:29.992408+0000","last_undegraded":"2024-03-12T17:06:29.992408+0000","last_fullsized":"2024-03-12T17:06:29.992408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:58:32.509940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.509666+0000","last_change":"2024-03-12T17:06:30.031785+0000","last_active":"2024-03-12T17:06:30.509666+0000","last_peered":"2024-03-12T17:06:30.509666+0000","last_clean":"2024-03-12T17:06:30.509666+0000","last_became_active":"2024-03-12T17:06:30.031328+0000","last_became_peered":"2024-03-12T17:06:30.031328+0000","last_unstale":"2024-03-12T17:06:30.509666+0000","last_undegraded":"2024-03-12T17:06:30.509666+0000","last_fullsized":"2024-03-12T17:06:30.509666+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:35:06.129757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012185+0000","last_change":"2024-03-12T17:06:25.931255+0000","last_active":"2024-03-12T17:06:30.012185+0000","last_peered":"2024-03-12T17:06:30.012185+0000","last_clean":"2024-03-12T17:06:30.012185+0000","last_became_active":"2024-03-12T17:06:25.929387+0000","last_became_peered":"2024-03-12T17:06:25.929387+0000","last_unstale":"2024-03-12T17:06:30.012185+0000","last_undegraded":"2024-03-12T17:06:30.012185+0000","last_fullsized":"2024-03-12T17:06:30.012185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:04:04.198678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.980151+0000","last_change":"2024-03-12T17:06:25.443901+0000","last_active":"2024-03-12T17:06:29.980151+0000","last_peered":"2024-03-12T17:06:29.980151+0000","last_clean":"2024-03-12T17:06:29.980151+0000","last_became_active":"2024-03-12T17:06:25.443602+0000","last_became_peered":"2024-03-12T17:06:25.443602+0000","last_unstale":"2024-03-12T17:06:29.980151+0000","last_undegraded":"2024-03-12T17:06:29.980151+0000","last_fullsized":"2024-03-12T17:06:29.980151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:00:32.347614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.511172+0000","last_change":"2024-03-12T17:06:30.025943+0000","last_active":"2024-03-12T17:06:30.511172+0000","last_peered":"2024-03-12T17:06:30.511172+0000","last_clean":"2024-03-12T17:06:30.511172+0000","last_became_active":"2024-03-12T17:06:30.025481+0000","last_became_peered":"2024-03-12T17:06:30.025481+0000","last_unstale":"2024-03-12T17:06:30.511172+0000","last_undegraded":"2024-03-12T17:06:30.511172+0000","last_fullsized":"2024-03-12T17:06:30.511172+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:17:09.055890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.996440+0000","last_change":"2024-03-12T17:06:25.940996+0000","last_active":"2024-03-12T17:06:29.996440+0000","last_peered":"2024-03-12T17:06:29.996440+0000","last_clean":"2024-03-12T17:06:29.996440+0000","last_became_active":"2024-03-12T17:06:25.940740+0000","last_became_peered":"2024-03-12T17:06:25.940740+0000","last_unstale":"2024-03-12T17:06:29.996440+0000","last_undegraded":"2024-03-12T17:06:29.996440+0000","last_fullsized":"2024-03-12T17:06:29.996440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:36:26.472050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.014590+0000","last_change":"2024-03-12T17:06:25.450168+0000","last_active":"2024-03-12T17:06:30.014590+0000","last_peered":"2024-03-12T17:06:30.014590+0000","last_clean":"2024-03-12T17:06:30.014590+0000","last_became_active":"2024-03-12T17:06:25.449620+0000","last_became_peered":"2024-03-12T17:06:25.449620+0000","last_unstale":"2024-03-12T17:06:30.014590+0000","last_undegraded":"2024-03-12T17:06:30.014590+0000","last_fullsized":"2024-03-12T17:06:30.014590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:58:32.509940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.509124+0000","last_change":"2024-03-12T17:06:30.020545+0000","last_active":"2024-03-12T17:06:30.509124+0000","last_peered":"2024-03-12T17:06:30.509124+0000","last_clean":"2024-03-12T17:06:30.509124+0000","last_became_active":"2024-03-12T17:06:30.020246+0000","last_became_peered":"2024-03-12T17:06:30.020246+0000","last_unstale":"2024-03-12T17:06:30.509124+0000","last_undegraded":"2024-03-12T17:06:30.509124+0000","last_fullsized":"2024-03-12T17:06:30.509124+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:08:39.661179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990996+0000","last_change":"2024-03-12T17:06:25.931831+0000","last_active":"2024-03-12T17:06:29.990996+0000","last_peered":"2024-03-12T17:06:29.990996+0000","last_clean":"2024-03-12T17:06:29.990996+0000","last_became_active":"2024-03-12T17:06:25.931525+0000","last_became_peered":"2024-03-12T17:06:25.931525+0000","last_unstale":"2024-03-12T17:06:29.990996+0000","last_undegraded":"2024-03-12T17:06:29.990996+0000","last_fullsized":"2024-03-12T17:06:29.990996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:06:19.464242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.989265+0000","last_change":"2024-03-12T17:06:25.435909+0000","last_active":"2024-03-12T17:06:29.989265+0000","last_peered":"2024-03-12T17:06:29.989265+0000","last_clean":"2024-03-12T17:06:29.989265+0000","last_became_active":"2024-03-12T17:06:25.434838+0000","last_became_peered":"2024-03-12T17:06:25.434838+0000","last_unstale":"2024-03-12T17:06:29.989265+0000","last_undegraded":"2024-03-12T17:06:29.989265+0000","last_fullsized":"2024-03-12T17:06:29.989265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:36:47.407275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.014690+0000","last_change":"2024-03-12T17:06:21.714813+0000","last_active":"2024-03-12T17:06:30.014690+0000","last_peered":"2024-03-12T17:06:30.014690+0000","last_clean":"2024-03-12T17:06:30.014690+0000","last_became_active":"2024-03-12T17:06:21.712488+0000","last_became_peered":"2024-03-12T17:06:21.712488+0000","last_unstale":"2024-03-12T17:06:30.014690+0000","last_undegraded":"2024-03-12T17:06:30.014690+0000","last_fullsized":"2024-03-12T17:06:30.014690+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:19.815688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:19.815688+0000","last_clean_scrub_stamp":"2024-03-12T17:06:19.815688+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:38:54.300793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.510890+0000","last_change":"2024-03-12T17:06:30.017415+0000","last_active":"2024-03-12T17:06:30.510890+0000","last_peered":"2024-03-12T17:06:30.510890+0000","last_clean":"2024-03-12T17:06:30.510890+0000","last_became_active":"2024-03-12T17:06:30.015976+0000","last_became_peered":"2024-03-12T17:06:30.015976+0000","last_unstale":"2024-03-12T17:06:30.510890+0000","last_undegraded":"2024-03-12T17:06:30.510890+0000","last_fullsized":"2024-03-12T17:06:30.510890+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:33:44.182800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012124+0000","last_change":"2024-03-12T17:06:25.959282+0000","last_active":"2024-03-12T17:06:30.012124+0000","last_peered":"2024-03-12T17:06:30.012124+0000","last_clean":"2024-03-12T17:06:30.012124+0000","last_became_active":"2024-03-12T17:06:25.959028+0000","last_became_peered":"2024-03-12T17:06:25.959028+0000","last_unstale":"2024-03-12T17:06:30.012124+0000","last_undegraded":"2024-03-12T17:06:30.012124+0000","last_fullsized":"2024-03-12T17:06:30.012124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:38:01.478195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.980482+0000","last_change":"2024-03-12T17:06:25.442608+0000","last_active":"2024-03-12T17:06:29.980482+0000","last_peered":"2024-03-12T17:06:29.980482+0000","last_clean":"2024-03-12T17:06:29.980482+0000","last_became_active":"2024-03-12T17:06:25.441652+0000","last_became_peered":"2024-03-12T17:06:25.441652+0000","last_unstale":"2024-03-12T17:06:29.980482+0000","last_undegraded":"2024-03-12T17:06:29.980482+0000","last_fullsized":"2024-03-12T17:06:29.980482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:34:26.277720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.510468+0000","last_change":"2024-03-12T17:06:30.020225+0000","last_active":"2024-03-12T17:06:30.510468+0000","last_peered":"2024-03-12T17:06:30.510468+0000","last_clean":"2024-03-12T17:06:30.510468+0000","last_became_active":"2024-03-12T17:06:30.017277+0000","last_became_peered":"2024-03-12T17:06:30.017277+0000","last_unstale":"2024-03-12T17:06:30.510468+0000","last_undegraded":"2024-03-12T17:06:30.510468+0000","last_fullsized":"2024-03-12T17:06:30.510468+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:35:06.129757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.991439+0000","last_change":"2024-03-12T17:06:25.942345+0000","last_active":"2024-03-12T17:06:29.991439+0000","last_peered":"2024-03-12T17:06:29.991439+0000","last_clean":"2024-03-12T17:06:29.991439+0000","last_became_active":"2024-03-12T17:06:25.941771+0000","last_became_peered":"2024-03-12T17:06:25.941771+0000","last_unstale":"2024-03-12T17:06:29.991439+0000","last_undegraded":"2024-03-12T17:06:29.991439+0000","last_fullsized":"2024-03-12T17:06:29.991439+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:32:08.311478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.022019+0000","last_change":"2024-03-12T17:06:25.448435+0000","last_active":"2024-03-12T17:06:30.022019+0000","last_peered":"2024-03-12T17:06:30.022019+0000","last_clean":"2024-03-12T17:06:30.022019+0000","last_became_active":"2024-03-12T17:06:25.448141+0000","last_became_peered":"2024-03-12T17:06:25.448141+0000","last_unstale":"2024-03-12T17:06:30.022019+0000","last_undegraded":"2024-03-12T17:06:30.022019+0000","last_fullsized":"2024-03-12T17:06:30.022019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:36:47.407275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.508634+0000","last_change":"2024-03-12T17:06:30.017805+0000","last_active":"2024-03-12T17:06:30.508634+0000","last_peered":"2024-03-12T17:06:30.508634+0000","last_clean":"2024-03-12T17:06:30.508634+0000","last_became_active":"2024-03-12T17:06:30.017383+0000","last_became_peered":"2024-03-12T17:06:30.017383+0000","last_unstale":"2024-03-12T17:06:30.508634+0000","last_undegraded":"2024-03-12T17:06:30.508634+0000","last_fullsized":"2024-03-12T17:06:30.508634+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:17:09.055890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.013754+0000","last_change":"2024-03-12T17:06:25.958864+0000","last_active":"2024-03-12T17:06:30.013754+0000","last_peered":"2024-03-12T17:06:30.013754+0000","last_clean":"2024-03-12T17:06:30.013754+0000","last_became_active":"2024-03-12T17:06:25.958588+0000","last_became_peered":"2024-03-12T17:06:25.958588+0000","last_unstale":"2024-03-12T17:06:30.013754+0000","last_undegraded":"2024-03-12T17:06:30.013754+0000","last_fullsized":"2024-03-12T17:06:30.013754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:21:02.182215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.980396+0000","last_change":"2024-03-12T17:06:25.453348+0000","last_active":"2024-03-12T17:06:29.980396+0000","last_peered":"2024-03-12T17:06:29.980396+0000","last_clean":"2024-03-12T17:06:29.980396+0000","last_became_active":"2024-03-12T17:06:25.449918+0000","last_became_peered":"2024-03-12T17:06:25.449918+0000","last_unstale":"2024-03-12T17:06:29.980396+0000","last_undegraded":"2024-03-12T17:06:29.980396+0000","last_fullsized":"2024-03-12T17:06:29.980396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:49:52.284732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.479802+0000","last_change":"2024-03-12T17:06:30.023418+0000","last_active":"2024-03-12T17:06:30.479802+0000","last_peered":"2024-03-12T17:06:30.479802+0000","last_clean":"2024-03-12T17:06:30.479802+0000","last_became_active":"2024-03-12T17:06:30.022919+0000","last_became_peered":"2024-03-12T17:06:30.022919+0000","last_unstale":"2024-03-12T17:06:30.479802+0000","last_undegraded":"2024-03-12T17:06:30.479802+0000","last_fullsized":"2024-03-12T17:06:30.479802+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:13:55.311599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.015933+0000","last_change":"2024-03-12T17:06:25.961193+0000","last_active":"2024-03-12T17:06:30.015933+0000","last_peered":"2024-03-12T17:06:30.015933+0000","last_clean":"2024-03-12T17:06:30.015933+0000","last_became_active":"2024-03-12T17:06:25.960906+0000","last_became_peered":"2024-03-12T17:06:25.960906+0000","last_unstale":"2024-03-12T17:06:30.015933+0000","last_undegraded":"2024-03-12T17:06:30.015933+0000","last_fullsized":"2024-03-12T17:06:30.015933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:17:09.067136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.013327+0000","last_change":"2024-03-12T17:06:25.442363+0000","last_active":"2024-03-12T17:06:30.013327+0000","last_peered":"2024-03-12T17:06:30.013327+0000","last_clean":"2024-03-12T17:06:30.013327+0000","last_became_active":"2024-03-12T17:06:25.442053+0000","last_became_peered":"2024-03-12T17:06:25.442053+0000","last_unstale":"2024-03-12T17:06:30.013327+0000","last_undegraded":"2024-03-12T17:06:30.013327+0000","last_fullsized":"2024-03-12T17:06:30.013327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:06:05.702956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.481199+0000","last_change":"2024-03-12T17:06:30.020741+0000","last_active":"2024-03-12T17:06:30.481199+0000","last_peered":"2024-03-12T17:06:30.481199+0000","last_clean":"2024-03-12T17:06:30.481199+0000","last_became_active":"2024-03-12T17:06:30.018043+0000","last_became_peered":"2024-03-12T17:06:30.018043+0000","last_unstale":"2024-03-12T17:06:30.481199+0000","last_undegraded":"2024-03-12T17:06:30.481199+0000","last_fullsized":"2024-03-12T17:06:30.481199+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:11:34.105824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.016104+0000","last_change":"2024-03-12T17:06:25.958260+0000","last_active":"2024-03-12T17:06:30.016104+0000","last_peered":"2024-03-12T17:06:30.016104+0000","last_clean":"2024-03-12T17:06:30.016104+0000","last_became_active":"2024-03-12T17:06:25.955966+0000","last_became_peered":"2024-03-12T17:06:25.955966+0000","last_unstale":"2024-03-12T17:06:30.016104+0000","last_undegraded":"2024-03-12T17:06:30.016104+0000","last_fullsized":"2024-03-12T17:06:30.016104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:33:16.553805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.979851+0000","last_change":"2024-03-12T17:06:25.452222+0000","last_active":"2024-03-12T17:06:29.979851+0000","last_peered":"2024-03-12T17:06:29.979851+0000","last_clean":"2024-03-12T17:06:29.979851+0000","last_became_active":"2024-03-12T17:06:25.451696+0000","last_became_peered":"2024-03-12T17:06:25.451696+0000","last_unstale":"2024-03-12T17:06:29.979851+0000","last_undegraded":"2024-03-12T17:06:29.979851+0000","last_fullsized":"2024-03-12T17:06:29.979851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:28:25.809578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.508879+0000","last_change":"2024-03-12T17:06:30.009551+0000","last_active":"2024-03-12T17:06:30.508879+0000","last_peered":"2024-03-12T17:06:30.508879+0000","last_clean":"2024-03-12T17:06:30.508879+0000","last_became_active":"2024-03-12T17:06:30.009269+0000","last_became_peered":"2024-03-12T17:06:30.009269+0000","last_unstale":"2024-03-12T17:06:30.508879+0000","last_undegraded":"2024-03-12T17:06:30.508879+0000","last_fullsized":"2024-03-12T17:06:30.508879+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:42:49.327905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.984652+0000","last_change":"2024-03-12T17:06:25.949275+0000","last_active":"2024-03-12T17:06:29.984652+0000","last_peered":"2024-03-12T17:06:29.984652+0000","last_clean":"2024-03-12T17:06:29.984652+0000","last_became_active":"2024-03-12T17:06:25.947884+0000","last_became_peered":"2024-03-12T17:06:25.947884+0000","last_unstale":"2024-03-12T17:06:29.984652+0000","last_undegraded":"2024-03-12T17:06:29.984652+0000","last_fullsized":"2024-03-12T17:06:29.984652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:28:59.465721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.984927+0000","last_change":"2024-03-12T17:06:25.450408+0000","last_active":"2024-03-12T17:06:29.984927+0000","last_peered":"2024-03-12T17:06:29.984927+0000","last_clean":"2024-03-12T17:06:29.984927+0000","last_became_active":"2024-03-12T17:06:25.448867+0000","last_became_peered":"2024-03-12T17:06:25.448867+0000","last_unstale":"2024-03-12T17:06:29.984927+0000","last_undegraded":"2024-03-12T17:06:29.984927+0000","last_fullsized":"2024-03-12T17:06:29.984927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:49:52.284732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.477643+0000","last_change":"2024-03-12T17:06:30.022424+0000","last_active":"2024-03-12T17:06:30.477643+0000","last_peered":"2024-03-12T17:06:30.477643+0000","last_clean":"2024-03-12T17:06:30.477643+0000","last_became_active":"2024-03-12T17:06:30.021994+0000","last_became_peered":"2024-03-12T17:06:30.021994+0000","last_unstale":"2024-03-12T17:06:30.477643+0000","last_undegraded":"2024-03-12T17:06:30.477643+0000","last_fullsized":"2024-03-12T17:06:30.477643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:32:37.399700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012128+0000","last_change":"2024-03-12T17:06:25.958282+0000","last_active":"2024-03-12T17:06:30.012128+0000","last_peered":"2024-03-12T17:06:30.012128+0000","last_clean":"2024-03-12T17:06:30.012128+0000","last_became_active":"2024-03-12T17:06:25.955357+0000","last_became_peered":"2024-03-12T17:06:25.955357+0000","last_unstale":"2024-03-12T17:06:30.012128+0000","last_undegraded":"2024-03-12T17:06:30.012128+0000","last_fullsized":"2024-03-12T17:06:30.012128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:15:36.195143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.992283+0000","last_change":"2024-03-12T17:06:25.448887+0000","last_active":"2024-03-12T17:06:29.992283+0000","last_peered":"2024-03-12T17:06:29.992283+0000","last_clean":"2024-03-12T17:06:29.992283+0000","last_became_active":"2024-03-12T17:06:25.448299+0000","last_became_peered":"2024-03-12T17:06:25.448299+0000","last_unstale":"2024-03-12T17:06:29.992283+0000","last_undegraded":"2024-03-12T17:06:29.992283+0000","last_fullsized":"2024-03-12T17:06:29.992283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:00:06.114876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.477924+0000","last_change":"2024-03-12T17:06:30.022221+0000","last_active":"2024-03-12T17:06:30.477924+0000","last_peered":"2024-03-12T17:06:30.477924+0000","last_clean":"2024-03-12T17:06:30.477924+0000","last_became_active":"2024-03-12T17:06:30.021765+0000","last_became_peered":"2024-03-12T17:06:30.021765+0000","last_unstale":"2024-03-12T17:06:30.477924+0000","last_undegraded":"2024-03-12T17:06:30.477924+0000","last_fullsized":"2024-03-12T17:06:30.477924+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:28:34.686064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.996652+0000","last_change":"2024-03-12T17:06:25.944466+0000","last_active":"2024-03-12T17:06:29.996652+0000","last_peered":"2024-03-12T17:06:29.996652+0000","last_clean":"2024-03-12T17:06:29.996652+0000","last_became_active":"2024-03-12T17:06:25.944075+0000","last_became_peered":"2024-03-12T17:06:25.944075+0000","last_unstale":"2024-03-12T17:06:29.996652+0000","last_undegraded":"2024-03-12T17:06:29.996652+0000","last_fullsized":"2024-03-12T17:06:29.996652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T20:48:48.332409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.996480+0000","last_change":"2024-03-12T17:06:25.444436+0000","last_active":"2024-03-12T17:06:29.996480+0000","last_peered":"2024-03-12T17:06:29.996480+0000","last_clean":"2024-03-12T17:06:29.996480+0000","last_became_active":"2024-03-12T17:06:25.443700+0000","last_became_peered":"2024-03-12T17:06:25.443700+0000","last_unstale":"2024-03-12T17:06:29.996480+0000","last_undegraded":"2024-03-12T17:06:29.996480+0000","last_fullsized":"2024-03-12T17:06:29.996480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:55:17.158520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.483163+0000","last_change":"2024-03-12T17:06:30.022439+0000","last_active":"2024-03-12T17:06:30.483163+0000","last_peered":"2024-03-12T17:06:30.483163+0000","last_clean":"2024-03-12T17:06:30.483163+0000","last_became_active":"2024-03-12T17:06:30.017040+0000","last_became_peered":"2024-03-12T17:06:30.017040+0000","last_unstale":"2024-03-12T17:06:30.483163+0000","last_undegraded":"2024-03-12T17:06:30.483163+0000","last_fullsized":"2024-03-12T17:06:30.483163+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:08:39.661179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012020+0000","last_change":"2024-03-12T17:06:25.958251+0000","last_active":"2024-03-12T17:06:30.012020+0000","last_peered":"2024-03-12T17:06:30.012020+0000","last_clean":"2024-03-12T17:06:30.012020+0000","last_became_active":"2024-03-12T17:06:25.956505+0000","last_became_peered":"2024-03-12T17:06:25.956505+0000","last_unstale":"2024-03-12T17:06:30.012020+0000","last_undegraded":"2024-03-12T17:06:30.012020+0000","last_fullsized":"2024-03-12T17:06:30.012020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:32:08.311478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:44.758048+0000","last_change":"2024-03-12T17:06:25.448251+0000","last_active":"2024-03-12T17:06:44.758048+0000","last_peered":"2024-03-12T17:06:44.758048+0000","last_clean":"2024-03-12T17:06:44.758048+0000","last_became_active":"2024-03-12T17:06:25.447768+0000","last_became_peered":"2024-03-12T17:06:25.447768+0000","last_unstale":"2024-03-12T17:06:44.758048+0000","last_undegraded":"2024-03-12T17:06:44.758048+0000","last_fullsized":"2024-03-12T17:06:44.758048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:34:26.277720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3795,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.508684+0000","last_change":"2024-03-12T17:06:30.019328+0000","last_active":"2024-03-12T17:06:30.508684+0000","last_peered":"2024-03-12T17:06:30.508684+0000","last_clean":"2024-03-12T17:06:30.508684+0000","last_became_active":"2024-03-12T17:06:30.014815+0000","last_became_peered":"2024-03-12T17:06:30.014815+0000","last_unstale":"2024-03-12T17:06:30.508684+0000","last_undegraded":"2024-03-12T17:06:30.508684+0000","last_fullsized":"2024-03-12T17:06:30.508684+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:08:39.661179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012873+0000","last_change":"2024-03-12T17:06:25.945208+0000","last_active":"2024-03-12T17:06:30.012873+0000","last_peered":"2024-03-12T17:06:30.012873+0000","last_clean":"2024-03-12T17:06:30.012873+0000","last_became_active":"2024-03-12T17:06:25.944043+0000","last_became_peered":"2024-03-12T17:06:25.944043+0000","last_unstale":"2024-03-12T17:06:30.012873+0000","last_undegraded":"2024-03-12T17:06:30.012873+0000","last_fullsized":"2024-03-12T17:06:30.012873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:36:48.410681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.021900+0000","last_change":"2024-03-12T17:06:25.455080+0000","last_active":"2024-03-12T17:06:30.021900+0000","last_peered":"2024-03-12T17:06:30.021900+0000","last_clean":"2024-03-12T17:06:30.021900+0000","last_became_active":"2024-03-12T17:06:25.454788+0000","last_became_peered":"2024-03-12T17:06:25.454788+0000","last_unstale":"2024-03-12T17:06:30.021900+0000","last_undegraded":"2024-03-12T17:06:30.021900+0000","last_fullsized":"2024-03-12T17:06:30.021900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:33:42.671579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.482470+0000","last_change":"2024-03-12T17:06:30.018328+0000","last_active":"2024-03-12T17:06:30.482470+0000","last_peered":"2024-03-12T17:06:30.482470+0000","last_clean":"2024-03-12T17:06:30.482470+0000","last_became_active":"2024-03-12T17:06:30.017852+0000","last_became_peered":"2024-03-12T17:06:30.017852+0000","last_unstale":"2024-03-12T17:06:30.482470+0000","last_undegraded":"2024-03-12T17:06:30.482470+0000","last_fullsized":"2024-03-12T17:06:30.482470+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T02:49:07.614077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.999490+0000","last_change":"2024-03-12T17:06:25.938178+0000","last_active":"2024-03-12T17:06:29.999490+0000","last_peered":"2024-03-12T17:06:29.999490+0000","last_clean":"2024-03-12T17:06:29.999490+0000","last_became_active":"2024-03-12T17:06:25.937926+0000","last_became_peered":"2024-03-12T17:06:25.937926+0000","last_unstale":"2024-03-12T17:06:29.999490+0000","last_undegraded":"2024-03-12T17:06:29.999490+0000","last_fullsized":"2024-03-12T17:06:29.999490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:57:56.009742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990778+0000","last_change":"2024-03-12T17:06:25.451993+0000","last_active":"2024-03-12T17:06:29.990778+0000","last_peered":"2024-03-12T17:06:29.990778+0000","last_clean":"2024-03-12T17:06:29.990778+0000","last_became_active":"2024-03-12T17:06:25.451794+0000","last_became_peered":"2024-03-12T17:06:25.451794+0000","last_unstale":"2024-03-12T17:06:29.990778+0000","last_undegraded":"2024-03-12T17:06:29.990778+0000","last_fullsized":"2024-03-12T17:06:29.990778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:33:42.671579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.476846+0000","last_change":"2024-03-12T17:06:30.016232+0000","last_active":"2024-03-12T17:06:30.476846+0000","last_peered":"2024-03-12T17:06:30.476846+0000","last_clean":"2024-03-12T17:06:30.476846+0000","last_became_active":"2024-03-12T17:06:30.015825+0000","last_became_peered":"2024-03-12T17:06:30.015825+0000","last_unstale":"2024-03-12T17:06:30.476846+0000","last_undegraded":"2024-03-12T17:06:30.476846+0000","last_fullsized":"2024-03-12T17:06:30.476846+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:28:28.745853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990537+0000","last_change":"2024-03-12T17:06:25.938249+0000","last_active":"2024-03-12T17:06:29.990537+0000","last_peered":"2024-03-12T17:06:29.990537+0000","last_clean":"2024-03-12T17:06:29.990537+0000","last_became_active":"2024-03-12T17:06:25.937744+0000","last_became_peered":"2024-03-12T17:06:25.937744+0000","last_unstale":"2024-03-12T17:06:29.990537+0000","last_undegraded":"2024-03-12T17:06:29.990537+0000","last_fullsized":"2024-03-12T17:06:29.990537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:06:19.464242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990879+0000","last_change":"2024-03-12T17:06:25.452356+0000","last_active":"2024-03-12T17:06:29.990879+0000","last_peered":"2024-03-12T17:06:29.990879+0000","last_clean":"2024-03-12T17:06:29.990879+0000","last_became_active":"2024-03-12T17:06:25.452033+0000","last_became_peered":"2024-03-12T17:06:25.452033+0000","last_unstale":"2024-03-12T17:06:29.990879+0000","last_undegraded":"2024-03-12T17:06:29.990879+0000","last_fullsized":"2024-03-12T17:06:29.990879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:58:32.509940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.477019+0000","last_change":"2024-03-12T17:06:30.023373+0000","last_active":"2024-03-12T17:06:30.477019+0000","last_peered":"2024-03-12T17:06:30.477019+0000","last_clean":"2024-03-12T17:06:30.477019+0000","last_became_active":"2024-03-12T17:06:30.022331+0000","last_became_peered":"2024-03-12T17:06:30.022331+0000","last_unstale":"2024-03-12T17:06:30.477019+0000","last_undegraded":"2024-03-12T17:06:30.477019+0000","last_fullsized":"2024-03-12T17:06:30.477019+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:50:22.021457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.508638+0000","last_change":"2024-03-12T17:06:30.020503+0000","last_active":"2024-03-12T17:06:30.508638+0000","last_peered":"2024-03-12T17:06:30.508638+0000","last_clean":"2024-03-12T17:06:30.508638+0000","last_became_active":"2024-03-12T17:06:30.019897+0000","last_became_peered":"2024-03-12T17:06:30.019897+0000","last_unstale":"2024-03-12T17:06:30.508638+0000","last_undegraded":"2024-03-12T17:06:30.508638+0000","last_fullsized":"2024-03-12T17:06:30.508638+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:13:55.311599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.013985+0000","last_change":"2024-03-12T17:06:25.927847+0000","last_active":"2024-03-12T17:06:30.013985+0000","last_peered":"2024-03-12T17:06:30.013985+0000","last_clean":"2024-03-12T17:06:30.013985+0000","last_became_active":"2024-03-12T17:06:25.927572+0000","last_became_peered":"2024-03-12T17:06:25.927572+0000","last_unstale":"2024-03-12T17:06:30.013985+0000","last_undegraded":"2024-03-12T17:06:30.013985+0000","last_fullsized":"2024-03-12T17:06:30.013985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:29:49.461651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.989420+0000","last_change":"2024-03-12T17:06:25.443473+0000","last_active":"2024-03-12T17:06:29.989420+0000","last_peered":"2024-03-12T17:06:29.989420+0000","last_clean":"2024-03-12T17:06:29.989420+0000","last_became_active":"2024-03-12T17:06:25.443046+0000","last_became_peered":"2024-03-12T17:06:25.443046+0000","last_unstale":"2024-03-12T17:06:29.989420+0000","last_undegraded":"2024-03-12T17:06:29.989420+0000","last_fullsized":"2024-03-12T17:06:29.989420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:49:52.284732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.513045+0000","last_change":"2024-03-12T17:06:30.017610+0000","last_active":"2024-03-12T17:06:30.513045+0000","last_peered":"2024-03-12T17:06:30.513045+0000","last_clean":"2024-03-12T17:06:30.513045+0000","last_became_active":"2024-03-12T17:06:30.017119+0000","last_became_peered":"2024-03-12T17:06:30.017119+0000","last_unstale":"2024-03-12T17:06:30.513045+0000","last_undegraded":"2024-03-12T17:06:30.513045+0000","last_fullsized":"2024-03-12T17:06:30.513045+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:42:49.327905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.986700+0000","last_change":"2024-03-12T17:06:25.935139+0000","last_active":"2024-03-12T17:06:29.986700+0000","last_peered":"2024-03-12T17:06:29.986700+0000","last_clean":"2024-03-12T17:06:29.986700+0000","last_became_active":"2024-03-12T17:06:25.934842+0000","last_became_peered":"2024-03-12T17:06:25.934842+0000","last_unstale":"2024-03-12T17:06:29.986700+0000","last_undegraded":"2024-03-12T17:06:29.986700+0000","last_fullsized":"2024-03-12T17:06:29.986700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:28:59.465721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.989691+0000","last_change":"2024-03-12T17:06:25.447398+0000","last_active":"2024-03-12T17:06:29.989691+0000","last_peered":"2024-03-12T17:06:29.989691+0000","last_clean":"2024-03-12T17:06:29.989691+0000","last_became_active":"2024-03-12T17:06:25.447080+0000","last_became_peered":"2024-03-12T17:06:25.447080+0000","last_unstale":"2024-03-12T17:06:29.989691+0000","last_undegraded":"2024-03-12T17:06:29.989691+0000","last_fullsized":"2024-03-12T17:06:29.989691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:00:32.347614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.986717+0000","last_change":"2024-03-12T17:06:25.447858+0000","last_active":"2024-03-12T17:06:29.986717+0000","last_peered":"2024-03-12T17:06:29.986717+0000","last_clean":"2024-03-12T17:06:29.986717+0000","last_became_active":"2024-03-12T17:06:25.447551+0000","last_became_peered":"2024-03-12T17:06:25.447551+0000","last_unstale":"2024-03-12T17:06:29.986717+0000","last_undegraded":"2024-03-12T17:06:29.986717+0000","last_fullsized":"2024-03-12T17:06:29.986717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:28:25.809578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.996427+0000","last_change":"2024-03-12T17:06:25.943160+0000","last_active":"2024-03-12T17:06:29.996427+0000","last_peered":"2024-03-12T17:06:29.996427+0000","last_clean":"2024-03-12T17:06:29.996427+0000","last_became_active":"2024-03-12T17:06:25.942883+0000","last_became_peered":"2024-03-12T17:06:25.942883+0000","last_unstale":"2024-03-12T17:06:29.996427+0000","last_undegraded":"2024-03-12T17:06:29.996427+0000","last_fullsized":"2024-03-12T17:06:29.996427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:04:04.198678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.509831+0000","last_change":"2024-03-12T17:06:30.018594+0000","last_active":"2024-03-12T17:06:30.509831+0000","last_peered":"2024-03-12T17:06:30.509831+0000","last_clean":"2024-03-12T17:06:30.509831+0000","last_became_active":"2024-03-12T17:06:30.016503+0000","last_became_peered":"2024-03-12T17:06:30.016503+0000","last_unstale":"2024-03-12T17:06:30.509831+0000","last_undegraded":"2024-03-12T17:06:30.509831+0000","last_fullsized":"2024-03-12T17:06:30.509831+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:29:03.020890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.995919+0000","last_change":"2024-03-12T17:06:25.443901+0000","last_active":"2024-03-12T17:06:29.995919+0000","last_peered":"2024-03-12T17:06:29.995919+0000","last_clean":"2024-03-12T17:06:29.995919+0000","last_became_active":"2024-03-12T17:06:25.443607+0000","last_became_peered":"2024-03-12T17:06:25.443607+0000","last_unstale":"2024-03-12T17:06:29.995919+0000","last_undegraded":"2024-03-12T17:06:29.995919+0000","last_fullsized":"2024-03-12T17:06:29.995919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T05:00:49.177317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.011582+0000","last_change":"2024-03-12T17:06:25.931011+0000","last_active":"2024-03-12T17:06:30.011582+0000","last_peered":"2024-03-12T17:06:30.011582+0000","last_clean":"2024-03-12T17:06:30.011582+0000","last_became_active":"2024-03-12T17:06:25.930306+0000","last_became_peered":"2024-03-12T17:06:25.930306+0000","last_unstale":"2024-03-12T17:06:30.011582+0000","last_undegraded":"2024-03-12T17:06:30.011582+0000","last_fullsized":"2024-03-12T17:06:30.011582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:58:33.513346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.478204+0000","last_change":"2024-03-12T17:06:30.018640+0000","last_active":"2024-03-12T17:06:30.478204+0000","last_peered":"2024-03-12T17:06:30.478204+0000","last_clean":"2024-03-12T17:06:30.478204+0000","last_became_active":"2024-03-12T17:06:30.018278+0000","last_became_peered":"2024-03-12T17:06:30.018278+0000","last_unstale":"2024-03-12T17:06:30.478204+0000","last_undegraded":"2024-03-12T17:06:30.478204+0000","last_fullsized":"2024-03-12T17:06:30.478204+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:01:05.684140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.984555+0000","last_change":"2024-03-12T17:06:25.947304+0000","last_active":"2024-03-12T17:06:29.984555+0000","last_peered":"2024-03-12T17:06:29.984555+0000","last_clean":"2024-03-12T17:06:29.984555+0000","last_became_active":"2024-03-12T17:06:25.947030+0000","last_became_peered":"2024-03-12T17:06:25.947030+0000","last_unstale":"2024-03-12T17:06:29.984555+0000","last_undegraded":"2024-03-12T17:06:29.984555+0000","last_fullsized":"2024-03-12T17:06:29.984555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:29:49.461651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.979886+0000","last_change":"2024-03-12T17:06:25.929080+0000","last_active":"2024-03-12T17:06:29.979886+0000","last_peered":"2024-03-12T17:06:29.979886+0000","last_clean":"2024-03-12T17:06:29.979886+0000","last_became_active":"2024-03-12T17:06:25.928820+0000","last_became_peered":"2024-03-12T17:06:25.928820+0000","last_unstale":"2024-03-12T17:06:29.979886+0000","last_undegraded":"2024-03-12T17:06:29.979886+0000","last_fullsized":"2024-03-12T17:06:29.979886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:36:26.472050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.992041+0000","last_change":"2024-03-12T17:06:25.932309+0000","last_active":"2024-03-12T17:06:29.992041+0000","last_peered":"2024-03-12T17:06:29.992041+0000","last_clean":"2024-03-12T17:06:29.992041+0000","last_became_active":"2024-03-12T17:06:25.932014+0000","last_became_peered":"2024-03-12T17:06:25.932014+0000","last_unstale":"2024-03-12T17:06:29.992041+0000","last_undegraded":"2024-03-12T17:06:29.992041+0000","last_fullsized":"2024-03-12T17:06:29.992041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:11:15.145295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.996376+0000","last_change":"2024-03-12T17:06:25.937407+0000","last_active":"2024-03-12T17:06:29.996376+0000","last_peered":"2024-03-12T17:06:29.996376+0000","last_clean":"2024-03-12T17:06:29.996376+0000","last_became_active":"2024-03-12T17:06:25.937065+0000","last_became_peered":"2024-03-12T17:06:25.937065+0000","last_unstale":"2024-03-12T17:06:29.996376+0000","last_undegraded":"2024-03-12T17:06:29.996376+0000","last_fullsized":"2024-03-12T17:06:29.996376+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:04:28.921316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.994102+0000","last_change":"2024-03-12T17:06:25.930358+0000","last_active":"2024-03-12T17:06:29.994102+0000","last_peered":"2024-03-12T17:06:29.994102+0000","last_clean":"2024-03-12T17:06:29.994102+0000","last_became_active":"2024-03-12T17:06:25.929870+0000","last_became_peered":"2024-03-12T17:06:25.929870+0000","last_unstale":"2024-03-12T17:06:29.994102+0000","last_undegraded":"2024-03-12T17:06:29.994102+0000","last_fullsized":"2024-03-12T17:06:29.994102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:57:56.009742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.021765+0000","last_change":"2024-03-12T17:06:25.928987+0000","last_active":"2024-03-12T17:06:30.021765+0000","last_peered":"2024-03-12T17:06:30.021765+0000","last_clean":"2024-03-12T17:06:30.021765+0000","last_became_active":"2024-03-12T17:06:25.928717+0000","last_became_peered":"2024-03-12T17:06:25.928717+0000","last_unstale":"2024-03-12T17:06:30.021765+0000","last_undegraded":"2024-03-12T17:06:30.021765+0000","last_fullsized":"2024-03-12T17:06:30.021765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:23:25.984714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990238+0000","last_change":"2024-03-12T17:06:25.943871+0000","last_active":"2024-03-12T17:06:29.990238+0000","last_peered":"2024-03-12T17:06:29.990238+0000","last_clean":"2024-03-12T17:06:29.990238+0000","last_became_active":"2024-03-12T17:06:25.938802+0000","last_became_peered":"2024-03-12T17:06:25.938802+0000","last_unstale":"2024-03-12T17:06:29.990238+0000","last_undegraded":"2024-03-12T17:06:29.990238+0000","last_fullsized":"2024-03-12T17:06:29.990238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:23:25.984714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.004028+0000","last_change":"2024-03-12T17:06:25.935272+0000","last_active":"2024-03-12T17:06:30.004028+0000","last_peered":"2024-03-12T17:06:30.004028+0000","last_clean":"2024-03-12T17:06:30.004028+0000","last_became_active":"2024-03-12T17:06:25.928645+0000","last_became_peered":"2024-03-12T17:06:25.928645+0000","last_unstale":"2024-03-12T17:06:30.004028+0000","last_undegraded":"2024-03-12T17:06:30.004028+0000","last_fullsized":"2024-03-12T17:06:30.004028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:44:25.750833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.989166+0000","last_change":"2024-03-12T17:06:25.937035+0000","last_active":"2024-03-12T17:06:29.989166+0000","last_peered":"2024-03-12T17:06:29.989166+0000","last_clean":"2024-03-12T17:06:29.989166+0000","last_became_active":"2024-03-12T17:06:25.936598+0000","last_became_peered":"2024-03-12T17:06:25.936598+0000","last_unstale":"2024-03-12T17:06:29.989166+0000","last_undegraded":"2024-03-12T17:06:29.989166+0000","last_fullsized":"2024-03-12T17:06:29.989166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:21:02.182215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.989614+0000","last_change":"2024-03-12T17:06:25.931310+0000","last_active":"2024-03-12T17:06:29.989614+0000","last_peered":"2024-03-12T17:06:29.989614+0000","last_clean":"2024-03-12T17:06:29.989614+0000","last_became_active":"2024-03-12T17:06:25.930910+0000","last_became_peered":"2024-03-12T17:06:25.930910+0000","last_unstale":"2024-03-12T17:06:29.989614+0000","last_undegraded":"2024-03-12T17:06:29.989614+0000","last_fullsized":"2024-03-12T17:06:29.989614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:48:33.016370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.991821+0000","last_change":"2024-03-12T17:06:25.937111+0000","last_active":"2024-03-12T17:06:29.991821+0000","last_peered":"2024-03-12T17:06:29.991821+0000","last_clean":"2024-03-12T17:06:29.991821+0000","last_became_active":"2024-03-12T17:06:25.936895+0000","last_became_peered":"2024-03-12T17:06:25.936895+0000","last_unstale":"2024-03-12T17:06:29.991821+0000","last_undegraded":"2024-03-12T17:06:29.991821+0000","last_fullsized":"2024-03-12T17:06:29.991821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:38:01.478195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.996309+0000","last_change":"2024-03-12T17:06:25.947115+0000","last_active":"2024-03-12T17:06:29.996309+0000","last_peered":"2024-03-12T17:06:29.996309+0000","last_clean":"2024-03-12T17:06:29.996309+0000","last_became_active":"2024-03-12T17:06:25.946820+0000","last_became_peered":"2024-03-12T17:06:25.946820+0000","last_unstale":"2024-03-12T17:06:29.996309+0000","last_undegraded":"2024-03-12T17:06:29.996309+0000","last_fullsized":"2024-03-12T17:06:29.996309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:48:33.016370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.984992+0000","last_change":"2024-03-12T17:06:25.950973+0000","last_active":"2024-03-12T17:06:29.984992+0000","last_peered":"2024-03-12T17:06:29.984992+0000","last_clean":"2024-03-12T17:06:29.984992+0000","last_became_active":"2024-03-12T17:06:25.948336+0000","last_became_peered":"2024-03-12T17:06:25.948336+0000","last_unstale":"2024-03-12T17:06:29.984992+0000","last_undegraded":"2024-03-12T17:06:29.984992+0000","last_fullsized":"2024-03-12T17:06:29.984992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:04:04.198678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012237+0000","last_change":"2024-03-12T17:06:25.961536+0000","last_active":"2024-03-12T17:06:30.012237+0000","last_peered":"2024-03-12T17:06:30.012237+0000","last_clean":"2024-03-12T17:06:30.012237+0000","last_became_active":"2024-03-12T17:06:25.961147+0000","last_became_peered":"2024-03-12T17:06:25.961147+0000","last_unstale":"2024-03-12T17:06:30.012237+0000","last_undegraded":"2024-03-12T17:06:30.012237+0000","last_fullsized":"2024-03-12T17:06:30.012237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:06:19.464242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.979994+0000","last_change":"2024-03-12T17:06:25.939977+0000","last_active":"2024-03-12T17:06:29.979994+0000","last_peered":"2024-03-12T17:06:29.979994+0000","last_clean":"2024-03-12T17:06:29.979994+0000","last_became_active":"2024-03-12T17:06:25.939725+0000","last_became_peered":"2024-03-12T17:06:25.939725+0000","last_unstale":"2024-03-12T17:06:29.979994+0000","last_undegraded":"2024-03-12T17:06:29.979994+0000","last_fullsized":"2024-03-12T17:06:29.979994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:45:02.929901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.991517+0000","last_change":"2024-03-12T17:06:25.939649+0000","last_active":"2024-03-12T17:06:29.991517+0000","last_peered":"2024-03-12T17:06:29.991517+0000","last_clean":"2024-03-12T17:06:29.991517+0000","last_became_active":"2024-03-12T17:06:25.939379+0000","last_became_peered":"2024-03-12T17:06:25.939379+0000","last_unstale":"2024-03-12T17:06:29.991517+0000","last_undegraded":"2024-03-12T17:06:29.991517+0000","last_fullsized":"2024-03-12T17:06:29.991517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:17:09.067136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.988936+0000","last_change":"2024-03-12T17:06:25.948556+0000","last_active":"2024-03-12T17:06:29.988936+0000","last_peered":"2024-03-12T17:06:29.988936+0000","last_clean":"2024-03-12T17:06:29.988936+0000","last_became_active":"2024-03-12T17:06:25.948217+0000","last_became_peered":"2024-03-12T17:06:25.948217+0000","last_unstale":"2024-03-12T17:06:29.988936+0000","last_undegraded":"2024-03-12T17:06:29.988936+0000","last_fullsized":"2024-03-12T17:06:29.988936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:26:54.235648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.013984+0000","last_change":"2024-03-12T17:06:25.931098+0000","last_active":"2024-03-12T17:06:30.013984+0000","last_peered":"2024-03-12T17:06:30.013984+0000","last_clean":"2024-03-12T17:06:30.013984+0000","last_became_active":"2024-03-12T17:06:25.930273+0000","last_became_peered":"2024-03-12T17:06:25.930273+0000","last_unstale":"2024-03-12T17:06:30.013984+0000","last_undegraded":"2024-03-12T17:06:30.013984+0000","last_fullsized":"2024-03-12T17:06:30.013984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T20:48:48.332409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.996609+0000","last_change":"2024-03-12T17:06:25.933421+0000","last_active":"2024-03-12T17:06:29.996609+0000","last_peered":"2024-03-12T17:06:29.996609+0000","last_clean":"2024-03-12T17:06:29.996609+0000","last_became_active":"2024-03-12T17:06:25.933133+0000","last_became_peered":"2024-03-12T17:06:25.933133+0000","last_unstale":"2024-03-12T17:06:29.996609+0000","last_undegraded":"2024-03-12T17:06:29.996609+0000","last_fullsized":"2024-03-12T17:06:29.996609+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:28:59.465721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.991543+0000","last_change":"2024-03-12T17:06:25.948622+0000","last_active":"2024-03-12T17:06:29.991543+0000","last_peered":"2024-03-12T17:06:29.991543+0000","last_clean":"2024-03-12T17:06:29.991543+0000","last_became_active":"2024-03-12T17:06:25.948293+0000","last_became_peered":"2024-03-12T17:06:25.948293+0000","last_unstale":"2024-03-12T17:06:29.991543+0000","last_undegraded":"2024-03-12T17:06:29.991543+0000","last_fullsized":"2024-03-12T17:06:29.991543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:52:14.405796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012611+0000","last_change":"2024-03-12T17:06:25.944171+0000","last_active":"2024-03-12T17:06:30.012611+0000","last_peered":"2024-03-12T17:06:30.012611+0000","last_clean":"2024-03-12T17:06:30.012611+0000","last_became_active":"2024-03-12T17:06:25.940309+0000","last_became_peered":"2024-03-12T17:06:25.940309+0000","last_unstale":"2024-03-12T17:06:30.012611+0000","last_undegraded":"2024-03-12T17:06:30.012611+0000","last_fullsized":"2024-03-12T17:06:30.012611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:00:07.118282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.987793+0000","last_change":"2024-03-12T17:06:25.946194+0000","last_active":"2024-03-12T17:06:29.987793+0000","last_peered":"2024-03-12T17:06:29.987793+0000","last_clean":"2024-03-12T17:06:29.987793+0000","last_became_active":"2024-03-12T17:06:25.944383+0000","last_became_peered":"2024-03-12T17:06:25.944383+0000","last_unstale":"2024-03-12T17:06:29.987793+0000","last_undegraded":"2024-03-12T17:06:29.987793+0000","last_fullsized":"2024-03-12T17:06:29.987793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:48:33.016370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.989993+0000","last_change":"2024-03-12T17:06:25.936660+0000","last_active":"2024-03-12T17:06:29.989993+0000","last_peered":"2024-03-12T17:06:29.989993+0000","last_clean":"2024-03-12T17:06:29.989993+0000","last_became_active":"2024-03-12T17:06:25.936366+0000","last_became_peered":"2024-03-12T17:06:25.936366+0000","last_unstale":"2024-03-12T17:06:29.989993+0000","last_undegraded":"2024-03-12T17:06:29.989993+0000","last_fullsized":"2024-03-12T17:06:29.989993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:44:34.582031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.013702+0000","last_change":"2024-03-12T17:06:25.960447+0000","last_active":"2024-03-12T17:06:30.013702+0000","last_peered":"2024-03-12T17:06:30.013702+0000","last_clean":"2024-03-12T17:06:30.013702+0000","last_became_active":"2024-03-12T17:06:25.958764+0000","last_became_peered":"2024-03-12T17:06:25.958764+0000","last_unstale":"2024-03-12T17:06:30.013702+0000","last_undegraded":"2024-03-12T17:06:30.013702+0000","last_fullsized":"2024-03-12T17:06:30.013702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:16:50.369045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.987173+0000","last_change":"2024-03-12T17:06:25.928162+0000","last_active":"2024-03-12T17:06:29.987173+0000","last_peered":"2024-03-12T17:06:29.987173+0000","last_clean":"2024-03-12T17:06:29.987173+0000","last_became_active":"2024-03-12T17:06:25.927864+0000","last_became_peered":"2024-03-12T17:06:25.927864+0000","last_unstale":"2024-03-12T17:06:29.987173+0000","last_undegraded":"2024-03-12T17:06:29.987173+0000","last_fullsized":"2024-03-12T17:06:29.987173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:28:58.739417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.018239+0000","last_change":"2024-03-12T17:06:25.956401+0000","last_active":"2024-03-12T17:06:30.018239+0000","last_peered":"2024-03-12T17:06:30.018239+0000","last_clean":"2024-03-12T17:06:30.018239+0000","last_became_active":"2024-03-12T17:06:25.955835+0000","last_became_peered":"2024-03-12T17:06:25.955835+0000","last_unstale":"2024-03-12T17:06:30.018239+0000","last_undegraded":"2024-03-12T17:06:30.018239+0000","last_fullsized":"2024-03-12T17:06:30.018239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:11:15.145295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.985932+0000","last_change":"2024-03-12T17:06:25.952812+0000","last_active":"2024-03-12T17:06:29.985932+0000","last_peered":"2024-03-12T17:06:29.985932+0000","last_clean":"2024-03-12T17:06:29.985932+0000","last_became_active":"2024-03-12T17:06:25.946931+0000","last_became_peered":"2024-03-12T17:06:25.946931+0000","last_unstale":"2024-03-12T17:06:29.985932+0000","last_undegraded":"2024-03-12T17:06:29.985932+0000","last_fullsized":"2024-03-12T17:06:29.985932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:16:50.369045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012000+0000","last_change":"2024-03-12T17:06:25.950752+0000","last_active":"2024-03-12T17:06:30.012000+0000","last_peered":"2024-03-12T17:06:30.012000+0000","last_clean":"2024-03-12T17:06:30.012000+0000","last_became_active":"2024-03-12T17:06:25.950440+0000","last_became_peered":"2024-03-12T17:06:25.950440+0000","last_unstale":"2024-03-12T17:06:30.012000+0000","last_undegraded":"2024-03-12T17:06:30.012000+0000","last_fullsized":"2024-03-12T17:06:30.012000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:38:51.036295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990839+0000","last_change":"2024-03-12T17:06:25.944733+0000","last_active":"2024-03-12T17:06:29.990839+0000","last_peered":"2024-03-12T17:06:29.990839+0000","last_clean":"2024-03-12T17:06:29.990839+0000","last_became_active":"2024-03-12T17:06:25.942878+0000","last_became_peered":"2024-03-12T17:06:25.942878+0000","last_unstale":"2024-03-12T17:06:29.990839+0000","last_undegraded":"2024-03-12T17:06:29.990839+0000","last_fullsized":"2024-03-12T17:06:29.990839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:15:36.195143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.996580+0000","last_change":"2024-03-12T17:06:25.943005+0000","last_active":"2024-03-12T17:06:29.996580+0000","last_peered":"2024-03-12T17:06:29.996580+0000","last_clean":"2024-03-12T17:06:29.996580+0000","last_became_active":"2024-03-12T17:06:25.933118+0000","last_became_peered":"2024-03-12T17:06:25.933118+0000","last_unstale":"2024-03-12T17:06:29.996580+0000","last_undegraded":"2024-03-12T17:06:29.996580+0000","last_fullsized":"2024-03-12T17:06:29.996580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:44:25.750833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.022750+0000","last_change":"2024-03-12T17:06:25.947390+0000","last_active":"2024-03-12T17:06:30.022750+0000","last_peered":"2024-03-12T17:06:30.022750+0000","last_clean":"2024-03-12T17:06:30.022750+0000","last_became_active":"2024-03-12T17:06:25.947027+0000","last_became_peered":"2024-03-12T17:06:25.947027+0000","last_unstale":"2024-03-12T17:06:30.022750+0000","last_undegraded":"2024-03-12T17:06:30.022750+0000","last_fullsized":"2024-03-12T17:06:30.022750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:44:34.582031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.013617+0000","last_change":"2024-03-12T17:06:25.958459+0000","last_active":"2024-03-12T17:06:30.013617+0000","last_peered":"2024-03-12T17:06:30.013617+0000","last_clean":"2024-03-12T17:06:30.013617+0000","last_became_active":"2024-03-12T17:06:25.956687+0000","last_became_peered":"2024-03-12T17:06:25.956687+0000","last_unstale":"2024-03-12T17:06:30.013617+0000","last_undegraded":"2024-03-12T17:06:30.013617+0000","last_fullsized":"2024-03-12T17:06:30.013617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:26:54.235648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4593,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":864256,"data_stored":695401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4690000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.284}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.371}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.675}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4370000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":270336,"data_stored":104541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.274}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4350000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.409}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5629999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":104577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4419999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.401}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.546}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.663}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":278528,"data_stored":108246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.331}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":104451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":274432,"data_stored":104611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.125}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.268}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3140000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":872448,"data_stored":695457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.286}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.377}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.617}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3440000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-12T17:06:55.551 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-12T17:06:55.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.812+0000 7f905a600640 1 Processor -- start 2024-03-12T17:06:55.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.813+0000 7f905a600640 1 -- start start 2024-03-12T17:06:55.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.814+0000 7f905a600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f905415cda0 0x7f90540cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:55.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.814+0000 7f905a600640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90540d1830 con 0x7f90540b5a30 2024-03-12T17:06:55.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.814+0000 7f905a600640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90540b1f20 con 0x7f90540b4a60 2024-03-12T17:06:55.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.814+0000 7f905a600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90540d0c20 con 0x7f905415cda0 2024-03-12T17:06:55.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.814+0000 7f9052a00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f90540b5a30 0x7f90540b4350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:54868/0 (socket says 172.21.15.52:54868) 2024-03-12T17:06:55.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.815+0000 7f9052a00640 1 -- 172.21.15.52:0/1090566266 learned_addr learned my addr 172.21.15.52:0/1090566266 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:06:55.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.815+0000 7f9053400640 1 --2- 172.21.15.52:0/1090566266 >> v2:172.21.15.102:3300/0 conn(0x7f905415cda0 0x7f90540cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:55.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.815+0000 7f9052000640 1 -- 172.21.15.52:0/1090566266 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1453374420 0 0) 0x7f90540d1830 con 0x7f90540b5a30 2024-03-12T17:06:55.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.815+0000 7f9052000640 1 -- 172.21.15.52:0/1090566266 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9038003520 con 0x7f90540b5a30 2024-03-12T17:06:55.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.815+0000 7f9052000640 1 -- 172.21.15.52:0/1090566266 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2786133117 0 0) 0x7f9038003520 con 0x7f90540b5a30 2024-03-12T17:06:55.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.815+0000 7f9052000640 1 -- 172.21.15.52:0/1090566266 >> v2:172.21.15.102:3300/0 conn(0x7f905415cda0 msgr2=0x7f90540cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:55.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.815+0000 7f9052000640 1 --2- 172.21.15.52:0/1090566266 >> v2:172.21.15.102:3300/0 conn(0x7f905415cda0 0x7f90540cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:55.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.815+0000 7f9052000640 1 -- 172.21.15.52:0/1090566266 >> v1:172.21.15.102:6789/0 conn(0x7f90540b4a60 legacy=0x7f90540d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:55.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.816+0000 7f9052000640 1 -- 172.21.15.52:0/1090566266 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90540d0f00 con 0x7f90540b5a30 2024-03-12T17:06:55.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.816+0000 7f9052000640 1 -- 172.21.15.52:0/1090566266 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f9040001670 con 0x7f90540b5a30 2024-03-12T17:06:55.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.816+0000 7f9052000640 1 -- 172.21.15.52:0/1090566266 >> v1:172.21.15.52:6789/0 conn(0x7f90540b5a30 legacy=0x7f90540b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:55.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.816+0000 7f9052000640 1 --2- 172.21.15.52:0/1090566266 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9038003970 0x7f9038003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:55.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.816+0000 7f9052000640 1 --2- 172.21.15.52:0/1090566266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90380047c0 0x7f9038004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:55.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.816+0000 7f9052000640 1 --2- 172.21.15.52:0/1090566266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9038005180 0x7f903800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:55.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.816+0000 7f9052000640 1 -- 172.21.15.52:0/1090566266 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90540d0c20 con 0x7f90380047c0 2024-03-12T17:06:55.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.816+0000 7f9052000640 1 -- 172.21.15.52:0/1090566266 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903800c020 con 0x7f9038005180 2024-03-12T17:06:55.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.816+0000 7f9052000640 1 -- 172.21.15.52:0/1090566266 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f903800c1e0 con 0x7f9038003970 2024-03-12T17:06:55.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.816+0000 7f9052a00640 1 --2- 172.21.15.52:0/1090566266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90380047c0 0x7f9038004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:55.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.816+0000 7f9052a00640 1 -- 172.21.15.52:0/1090566266 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9038003970 msgr2=0x7f9038003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:55.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.816+0000 7f9052a00640 1 --2- 172.21.15.52:0/1090566266 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9038003970 0x7f9038003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:55.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.816+0000 7f9052a00640 1 -- 172.21.15.52:0/1090566266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9038005180 msgr2=0x7f903800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:55.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.816+0000 7f9053e00640 1 --2- 172.21.15.52:0/1090566266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9038005180 0x7f903800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:55.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.816+0000 7f9052a00640 1 --2- 172.21.15.52:0/1090566266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9038005180 0x7f903800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:55.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.816+0000 7f9052a00640 1 -- 172.21.15.52:0/1090566266 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9038003520 con 0x7f90380047c0 2024-03-12T17:06:55.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.816+0000 7f9053e00640 1 --2- 172.21.15.52:0/1090566266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9038005180 0x7f903800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:55.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.816+0000 7f9053e00640 1 -- 172.21.15.52:0/1090566266 reap_dead start 2024-03-12T17:06:55.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.818+0000 7f9052a00640 1 --2- 172.21.15.52:0/1090566266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90380047c0 0x7f9038004c40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f904000c240 tx=0x7f90380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b29cb129d007e8b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:55.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.818+0000 7f9052000640 1 -- 172.21.15.52:0/1090566266 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9040005ad0 con 0x7f90380047c0 2024-03-12T17:06:55.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.818+0000 7f9052000640 1 -- 172.21.15.52:0/1090566266 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9040005c30 con 0x7f90380047c0 2024-03-12T17:06:55.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.818+0000 7f9052000640 1 -- 172.21.15.52:0/1090566266 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904002caf0 con 0x7f90380047c0 2024-03-12T17:06:55.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.818+0000 7f905a600640 1 -- 172.21.15.52:0/1090566266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90380047c0 msgr2=0x7f9038004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:55.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.818+0000 7f905a600640 1 --2- 172.21.15.52:0/1090566266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90380047c0 0x7f9038004c40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f904000c240 tx=0x7f90380057f0 comp rx=0 tx=0).stop 2024-03-12T17:06:55.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.818+0000 7f905a600640 1 -- 172.21.15.52:0/1090566266 shutdown_connections 2024-03-12T17:06:55.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.818+0000 7f905a600640 1 --2- 172.21.15.52:0/1090566266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90380047c0 0x7f9038004c40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:55.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.818+0000 7f905a600640 1 -- 172.21.15.52:0/1090566266 >> 172.21.15.52:0/1090566266 conn(0x7f90540cf350 msgr2=0x7f90540cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:55.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.819+0000 7f905a600640 1 -- 172.21.15.52:0/1090566266 shutdown_connections 2024-03-12T17:06:55.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.819+0000 7f905a600640 1 -- 172.21.15.52:0/1090566266 wait complete. 2024-03-12T17:06:55.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.820+0000 7f905a600640 1 Processor -- start 2024-03-12T17:06:55.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.821+0000 7f905a600640 1 -- start start 2024-03-12T17:06:55.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.821+0000 7f905a600640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f90380047c0 0x7f90540d2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:55.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.821+0000 7f905a600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90541f59e0 0x7f905414fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:55.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.821+0000 7f905a600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f905415f7b0 0x7f905415fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:55.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.821+0000 7f905a600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90541f5e60 con 0x7f905415f7b0 2024-03-12T17:06:55.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.821+0000 7f905a600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90541f68f0 con 0x7f90541f59e0 2024-03-12T17:06:55.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.821+0000 7f905a600640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f905415dc10 con 0x7f90380047c0 2024-03-12T17:06:55.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.822+0000 7f9053e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f905415f7b0 0x7f905415fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:55.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.822+0000 7f9053e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f905415f7b0 0x7f905415fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60700/0 (socket says 172.21.15.52:60700) 2024-03-12T17:06:55.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.822+0000 7f9053e00640 1 -- 172.21.15.52:0/3621066345 learned_addr learned my addr 172.21.15.52:0/3621066345 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:55.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.822+0000 7f9053400640 1 --2- 172.21.15.52:0/3621066345 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f90380047c0 0x7f90540d2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:55.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.822+0000 7f9052a00640 1 --2- 172.21.15.52:0/3621066345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90541f59e0 0x7f905414fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:55.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.822+0000 7f9053e00640 1 -- 172.21.15.52:0/3621066345 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f90380047c0 msgr2=0x7f90540d2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:55.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.822+0000 7f9053e00640 1 --2- 172.21.15.52:0/3621066345 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f90380047c0 0x7f90540d2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:55.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.822+0000 7f9053e00640 1 -- 172.21.15.52:0/3621066345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90541f59e0 msgr2=0x7f905414fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:55.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.822+0000 7f9053e00640 1 --2- 172.21.15.52:0/3621066345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90541f59e0 0x7f905414fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:55.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.822+0000 7f9053e00640 1 -- 172.21.15.52:0/3621066345 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90541f6050 con 0x7f905415f7b0 2024-03-12T17:06:55.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.823+0000 7f9053e00640 1 --2- 172.21.15.52:0/3621066345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f905415f7b0 0x7f905415fc30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f90540b1c00 tx=0x7f904c00d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:55.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.823+0000 7f904b400640 1 -- 172.21.15.52:0/3621066345 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c003d80 con 0x7f905415f7b0 2024-03-12T17:06:55.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.823+0000 7f904b400640 1 -- 172.21.15.52:0/3621066345 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904c005070 con 0x7f905415f7b0 2024-03-12T17:06:55.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.823+0000 7f904b400640 1 -- 172.21.15.52:0/3621066345 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c0053e0 con 0x7f905415f7b0 2024-03-12T17:06:55.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.823+0000 7f905a600640 1 -- 172.21.15.52:0/3621066345 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90541f6330 con 0x7f905415f7b0 2024-03-12T17:06:55.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.823+0000 7f905a600640 1 -- 172.21.15.52:0/3621066345 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90541f3650 con 0x7f905415f7b0 2024-03-12T17:06:55.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.824+0000 7f905a600640 1 -- 172.21.15.52:0/3621066345 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90541f3870 con 0x7f905415f7b0 2024-03-12T17:06:55.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.825+0000 7f904b400640 1 -- 172.21.15.52:0/3621066345 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f904c005540 con 0x7f905415f7b0 2024-03-12T17:06:55.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.826+0000 7f904b400640 1 --2- 172.21.15.52:0/3621066345 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f903406ac00 0x7f903406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:55.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.826+0000 7f904b400640 1 -- 172.21.15.52:0/3621066345 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f904c088dd0 con 0x7f905415f7b0 2024-03-12T17:06:55.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.827+0000 7f9053400640 1 --2- 172.21.15.52:0/3621066345 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f903406ac00 0x7f903406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:55.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.828+0000 7f9053400640 1 --2- 172.21.15.52:0/3621066345 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f903406ac00 0x7f903406d0c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f90440047b0 tx=0x7f9044008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:55.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:55.831+0000 7f904b400640 1 -- 172.21.15.52:0/3621066345 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f904c053cd0 con 0x7f905415f7b0 2024-03-12T17:06:56.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.026+0000 7f905a600640 1 -- 172.21.15.52:0/3621066345 --> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f90540d14c0 con 0x7f903406ac00 2024-03-12T17:06:56.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.053+0000 7f904b400640 1 -- 172.21.15.52:0/3621066345 <== mgr.4109 v2:172.21.15.102:6832/36622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345051 (secure 0 0 0) 0x7f9044011920 con 0x7f903406ac00 2024-03-12T17:06:56.051 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:06:56.053 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-03-12T17:06:56.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.059+0000 7f905a600640 1 -- 172.21.15.52:0/3621066345 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f903406ac00 msgr2=0x7f903406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:56.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.059+0000 7f905a600640 1 --2- 172.21.15.52:0/3621066345 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f903406ac00 0x7f903406d0c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f90440047b0 tx=0x7f9044008040 comp rx=0 tx=0).stop 2024-03-12T17:06:56.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.059+0000 7f905a600640 1 -- 172.21.15.52:0/3621066345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f905415f7b0 msgr2=0x7f905415fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:56.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.059+0000 7f905a600640 1 --2- 172.21.15.52:0/3621066345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f905415f7b0 0x7f905415fc30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f90540b1c00 tx=0x7f904c00d9d0 comp rx=0 tx=0).stop 2024-03-12T17:06:56.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.059+0000 7f905a600640 1 -- 172.21.15.52:0/3621066345 shutdown_connections 2024-03-12T17:06:56.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.059+0000 7f905a600640 1 --2- 172.21.15.52:0/3621066345 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f903406ac00 0x7f903406d0c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:56.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.059+0000 7f905a600640 1 --2- 172.21.15.52:0/3621066345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f905415f7b0 0x7f905415fc30 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:56.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.059+0000 7f905a600640 1 --2- 172.21.15.52:0/3621066345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90541f59e0 0x7f905414fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:56.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.059+0000 7f905a600640 1 --2- 172.21.15.52:0/3621066345 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f90380047c0 0x7f90540d2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:56.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.059+0000 7f905a600640 1 -- 172.21.15.52:0/3621066345 >> 172.21.15.52:0/3621066345 conn(0x7f90540cf350 msgr2=0x7f905415bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:56.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.059+0000 7f905a600640 1 -- 172.21.15.52:0/3621066345 shutdown_connections 2024-03-12T17:06:56.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.059+0000 7f905a600640 1 -- 172.21.15.52:0/3621066345 wait complete. 2024-03-12T17:06:56.074 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-03-12T17:06:55.157508+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594961,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231808,"kb_used_data":3304,"kb_used_omap":13,"kb_used_meta":228338,"kb_avail":754742912,"statfs":{"total":773094113280,"available":772856741888,"internally_reserved":0,"allocated":3383296,"data_stored":2025530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":233818200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005387"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.999570+0000","last_change":"2024-03-12T17:06:25.442740+0000","last_active":"2024-03-12T17:06:29.999570+0000","last_peered":"2024-03-12T17:06:29.999570+0000","last_clean":"2024-03-12T17:06:29.999570+0000","last_became_active":"2024-03-12T17:06:25.438608+0000","last_became_peered":"2024-03-12T17:06:25.438608+0000","last_unstale":"2024-03-12T17:06:29.999570+0000","last_undegraded":"2024-03-12T17:06:29.999570+0000","last_fullsized":"2024-03-12T17:06:29.999570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:59:41.676210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.004064+0000","last_change":"2024-03-12T17:06:25.939770+0000","last_active":"2024-03-12T17:06:30.004064+0000","last_peered":"2024-03-12T17:06:30.004064+0000","last_clean":"2024-03-12T17:06:30.004064+0000","last_became_active":"2024-03-12T17:06:25.938969+0000","last_became_peered":"2024-03-12T17:06:25.938969+0000","last_unstale":"2024-03-12T17:06:30.004064+0000","last_undegraded":"2024-03-12T17:06:30.004064+0000","last_fullsized":"2024-03-12T17:06:30.004064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:23:25.984714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.476841+0000","last_change":"2024-03-12T17:06:30.023660+0000","last_active":"2024-03-12T17:06:30.476841+0000","last_peered":"2024-03-12T17:06:30.476841+0000","last_clean":"2024-03-12T17:06:30.476841+0000","last_became_active":"2024-03-12T17:06:30.022948+0000","last_became_peered":"2024-03-12T17:06:30.022948+0000","last_unstale":"2024-03-12T17:06:30.476841+0000","last_undegraded":"2024-03-12T17:06:30.476841+0000","last_fullsized":"2024-03-12T17:06:30.476841+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:04:13.473422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.003821+0000","last_change":"2024-03-12T17:06:25.446629+0000","last_active":"2024-03-12T17:06:30.003821+0000","last_peered":"2024-03-12T17:06:30.003821+0000","last_clean":"2024-03-12T17:06:30.003821+0000","last_became_active":"2024-03-12T17:06:25.442784+0000","last_became_peered":"2024-03-12T17:06:25.442784+0000","last_unstale":"2024-03-12T17:06:30.003821+0000","last_undegraded":"2024-03-12T17:06:30.003821+0000","last_fullsized":"2024-03-12T17:06:30.003821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:59:41.676210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.999425+0000","last_change":"2024-03-12T17:06:25.953462+0000","last_active":"2024-03-12T17:06:29.999425+0000","last_peered":"2024-03-12T17:06:29.999425+0000","last_clean":"2024-03-12T17:06:29.999425+0000","last_became_active":"2024-03-12T17:06:25.953198+0000","last_became_peered":"2024-03-12T17:06:25.953198+0000","last_unstale":"2024-03-12T17:06:29.999425+0000","last_undegraded":"2024-03-12T17:06:29.999425+0000","last_fullsized":"2024-03-12T17:06:29.999425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:52:14.405796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.480606+0000","last_change":"2024-03-12T17:06:30.017287+0000","last_active":"2024-03-12T17:06:30.480606+0000","last_peered":"2024-03-12T17:06:30.480606+0000","last_clean":"2024-03-12T17:06:30.480606+0000","last_became_active":"2024-03-12T17:06:30.016834+0000","last_became_peered":"2024-03-12T17:06:30.016834+0000","last_unstale":"2024-03-12T17:06:30.480606+0000","last_undegraded":"2024-03-12T17:06:30.480606+0000","last_fullsized":"2024-03-12T17:06:30.480606+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:21:26.853333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012212+0000","last_change":"2024-03-12T17:06:25.454124+0000","last_active":"2024-03-12T17:06:30.012212+0000","last_peered":"2024-03-12T17:06:30.012212+0000","last_clean":"2024-03-12T17:06:30.012212+0000","last_became_active":"2024-03-12T17:06:25.453877+0000","last_became_peered":"2024-03-12T17:06:25.453877+0000","last_unstale":"2024-03-12T17:06:30.012212+0000","last_undegraded":"2024-03-12T17:06:30.012212+0000","last_fullsized":"2024-03-12T17:06:30.012212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:45:01.926495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.991068+0000","last_change":"2024-03-12T17:06:25.926772+0000","last_active":"2024-03-12T17:06:29.991068+0000","last_peered":"2024-03-12T17:06:29.991068+0000","last_clean":"2024-03-12T17:06:29.991068+0000","last_became_active":"2024-03-12T17:06:25.926486+0000","last_became_peered":"2024-03-12T17:06:25.926486+0000","last_unstale":"2024-03-12T17:06:29.991068+0000","last_undegraded":"2024-03-12T17:06:29.991068+0000","last_fullsized":"2024-03-12T17:06:29.991068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:28:58.739417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.482311+0000","last_change":"2024-03-12T17:06:30.028104+0000","last_active":"2024-03-12T17:06:30.482311+0000","last_peered":"2024-03-12T17:06:30.482311+0000","last_clean":"2024-03-12T17:06:30.482311+0000","last_became_active":"2024-03-12T17:06:30.027645+0000","last_became_peered":"2024-03-12T17:06:30.027645+0000","last_unstale":"2024-03-12T17:06:30.482311+0000","last_undegraded":"2024-03-12T17:06:30.482311+0000","last_fullsized":"2024-03-12T17:06:30.482311+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:13:55.311599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.011875+0000","last_change":"2024-03-12T17:06:25.455911+0000","last_active":"2024-03-12T17:06:30.011875+0000","last_peered":"2024-03-12T17:06:30.011875+0000","last_clean":"2024-03-12T17:06:30.011875+0000","last_became_active":"2024-03-12T17:06:25.455251+0000","last_became_peered":"2024-03-12T17:06:25.455251+0000","last_unstale":"2024-03-12T17:06:30.011875+0000","last_undegraded":"2024-03-12T17:06:30.011875+0000","last_fullsized":"2024-03-12T17:06:30.011875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:36:25.468644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990029+0000","last_change":"2024-03-12T17:06:25.940304+0000","last_active":"2024-03-12T17:06:29.990029+0000","last_peered":"2024-03-12T17:06:29.990029+0000","last_clean":"2024-03-12T17:06:29.990029+0000","last_became_active":"2024-03-12T17:06:25.940019+0000","last_became_peered":"2024-03-12T17:06:25.940019+0000","last_unstale":"2024-03-12T17:06:29.990029+0000","last_undegraded":"2024-03-12T17:06:29.990029+0000","last_fullsized":"2024-03-12T17:06:29.990029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:44:34.582031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.481007+0000","last_change":"2024-03-12T17:06:30.030105+0000","last_active":"2024-03-12T17:06:30.481007+0000","last_peered":"2024-03-12T17:06:30.481007+0000","last_clean":"2024-03-12T17:06:30.481007+0000","last_became_active":"2024-03-12T17:06:30.029702+0000","last_became_peered":"2024-03-12T17:06:30.029702+0000","last_unstale":"2024-03-12T17:06:30.481007+0000","last_undegraded":"2024-03-12T17:06:30.481007+0000","last_fullsized":"2024-03-12T17:06:30.481007+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:33:44.182800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.986543+0000","last_change":"2024-03-12T17:06:25.942985+0000","last_active":"2024-03-12T17:06:29.986543+0000","last_peered":"2024-03-12T17:06:29.986543+0000","last_clean":"2024-03-12T17:06:29.986543+0000","last_became_active":"2024-03-12T17:06:25.942526+0000","last_became_peered":"2024-03-12T17:06:25.942526+0000","last_unstale":"2024-03-12T17:06:29.986543+0000","last_undegraded":"2024-03-12T17:06:29.986543+0000","last_fullsized":"2024-03-12T17:06:29.986543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:44:25.750833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012322+0000","last_change":"2024-03-12T17:06:25.453557+0000","last_active":"2024-03-12T17:06:30.012322+0000","last_peered":"2024-03-12T17:06:30.012322+0000","last_clean":"2024-03-12T17:06:30.012322+0000","last_became_active":"2024-03-12T17:06:25.451663+0000","last_became_peered":"2024-03-12T17:06:25.451663+0000","last_unstale":"2024-03-12T17:06:30.012322+0000","last_undegraded":"2024-03-12T17:06:30.012322+0000","last_fullsized":"2024-03-12T17:06:30.012322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:36:47.407275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.482348+0000","last_change":"2024-03-12T17:06:30.018487+0000","last_active":"2024-03-12T17:06:30.482348+0000","last_peered":"2024-03-12T17:06:30.482348+0000","last_clean":"2024-03-12T17:06:30.482348+0000","last_became_active":"2024-03-12T17:06:30.017933+0000","last_became_peered":"2024-03-12T17:06:30.017933+0000","last_unstale":"2024-03-12T17:06:30.482348+0000","last_undegraded":"2024-03-12T17:06:30.482348+0000","last_fullsized":"2024-03-12T17:06:30.482348+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:42:49.327905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.995828+0000","last_change":"2024-03-12T17:06:25.952715+0000","last_active":"2024-03-12T17:06:29.995828+0000","last_peered":"2024-03-12T17:06:29.995828+0000","last_clean":"2024-03-12T17:06:29.995828+0000","last_became_active":"2024-03-12T17:06:25.952288+0000","last_became_peered":"2024-03-12T17:06:25.952288+0000","last_unstale":"2024-03-12T17:06:29.995828+0000","last_undegraded":"2024-03-12T17:06:29.995828+0000","last_fullsized":"2024-03-12T17:06:29.995828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:29:49.461651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.015251+0000","last_change":"2024-03-12T17:06:25.448043+0000","last_active":"2024-03-12T17:06:30.015251+0000","last_peered":"2024-03-12T17:06:30.015251+0000","last_clean":"2024-03-12T17:06:30.015251+0000","last_became_active":"2024-03-12T17:06:25.447665+0000","last_became_peered":"2024-03-12T17:06:25.447665+0000","last_unstale":"2024-03-12T17:06:30.015251+0000","last_undegraded":"2024-03-12T17:06:30.015251+0000","last_fullsized":"2024-03-12T17:06:30.015251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:28:25.809578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.481001+0000","last_change":"2024-03-12T17:06:30.020157+0000","last_active":"2024-03-12T17:06:30.481001+0000","last_peered":"2024-03-12T17:06:30.481001+0000","last_clean":"2024-03-12T17:06:30.481001+0000","last_became_active":"2024-03-12T17:06:30.019631+0000","last_became_peered":"2024-03-12T17:06:30.019631+0000","last_unstale":"2024-03-12T17:06:30.481001+0000","last_undegraded":"2024-03-12T17:06:30.481001+0000","last_fullsized":"2024-03-12T17:06:30.481001+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:46:40.007538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.987316+0000","last_change":"2024-03-12T17:06:25.925596+0000","last_active":"2024-03-12T17:06:29.987316+0000","last_peered":"2024-03-12T17:06:29.987316+0000","last_clean":"2024-03-12T17:06:29.987316+0000","last_became_active":"2024-03-12T17:06:25.925225+0000","last_became_peered":"2024-03-12T17:06:25.925225+0000","last_unstale":"2024-03-12T17:06:29.987316+0000","last_undegraded":"2024-03-12T17:06:29.987316+0000","last_fullsized":"2024-03-12T17:06:29.987316+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:44:34.582031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.014268+0000","last_change":"2024-03-12T17:06:25.450336+0000","last_active":"2024-03-12T17:06:30.014268+0000","last_peered":"2024-03-12T17:06:30.014268+0000","last_clean":"2024-03-12T17:06:30.014268+0000","last_became_active":"2024-03-12T17:06:25.450049+0000","last_became_peered":"2024-03-12T17:06:25.450049+0000","last_unstale":"2024-03-12T17:06:30.014268+0000","last_undegraded":"2024-03-12T17:06:30.014268+0000","last_fullsized":"2024-03-12T17:06:30.014268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:00:06.114876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.511116+0000","last_change":"2024-03-12T17:06:30.024652+0000","last_active":"2024-03-12T17:06:30.511116+0000","last_peered":"2024-03-12T17:06:30.511116+0000","last_clean":"2024-03-12T17:06:30.511116+0000","last_became_active":"2024-03-12T17:06:30.024156+0000","last_became_peered":"2024-03-12T17:06:30.024156+0000","last_unstale":"2024-03-12T17:06:30.511116+0000","last_undegraded":"2024-03-12T17:06:30.511116+0000","last_fullsized":"2024-03-12T17:06:30.511116+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:21:26.853333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012195+0000","last_change":"2024-03-12T17:06:25.935633+0000","last_active":"2024-03-12T17:06:30.012195+0000","last_peered":"2024-03-12T17:06:30.012195+0000","last_clean":"2024-03-12T17:06:30.012195+0000","last_became_active":"2024-03-12T17:06:25.935322+0000","last_became_peered":"2024-03-12T17:06:25.935322+0000","last_unstale":"2024-03-12T17:06:30.012195+0000","last_undegraded":"2024-03-12T17:06:30.012195+0000","last_fullsized":"2024-03-12T17:06:30.012195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:34:27.281126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.014503+0000","last_change":"2024-03-12T17:06:25.456881+0000","last_active":"2024-03-12T17:06:30.014503+0000","last_peered":"2024-03-12T17:06:30.014503+0000","last_clean":"2024-03-12T17:06:30.014503+0000","last_became_active":"2024-03-12T17:06:25.456425+0000","last_became_peered":"2024-03-12T17:06:25.456425+0000","last_unstale":"2024-03-12T17:06:30.014503+0000","last_undegraded":"2024-03-12T17:06:30.014503+0000","last_fullsized":"2024-03-12T17:06:30.014503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:33:42.671579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.482388+0000","last_change":"2024-03-12T17:06:30.021976+0000","last_active":"2024-03-12T17:06:30.482388+0000","last_peered":"2024-03-12T17:06:30.482388+0000","last_clean":"2024-03-12T17:06:30.482388+0000","last_became_active":"2024-03-12T17:06:30.021512+0000","last_became_peered":"2024-03-12T17:06:30.021512+0000","last_unstale":"2024-03-12T17:06:30.482388+0000","last_undegraded":"2024-03-12T17:06:30.482388+0000","last_fullsized":"2024-03-12T17:06:30.482388+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:31:47.699683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.999227+0000","last_change":"2024-03-12T17:06:25.937491+0000","last_active":"2024-03-12T17:06:29.999227+0000","last_peered":"2024-03-12T17:06:29.999227+0000","last_clean":"2024-03-12T17:06:29.999227+0000","last_became_active":"2024-03-12T17:06:25.937173+0000","last_became_peered":"2024-03-12T17:06:25.937173+0000","last_unstale":"2024-03-12T17:06:29.999227+0000","last_undegraded":"2024-03-12T17:06:29.999227+0000","last_fullsized":"2024-03-12T17:06:29.999227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:29:49.461651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990464+0000","last_change":"2024-03-12T17:06:25.450121+0000","last_active":"2024-03-12T17:06:29.990464+0000","last_peered":"2024-03-12T17:06:29.990464+0000","last_clean":"2024-03-12T17:06:29.990464+0000","last_became_active":"2024-03-12T17:06:25.448549+0000","last_became_peered":"2024-03-12T17:06:25.448549+0000","last_unstale":"2024-03-12T17:06:29.990464+0000","last_undegraded":"2024-03-12T17:06:29.990464+0000","last_fullsized":"2024-03-12T17:06:29.990464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:36:47.407275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.481700+0000","last_change":"2024-03-12T17:06:30.020438+0000","last_active":"2024-03-12T17:06:30.481700+0000","last_peered":"2024-03-12T17:06:30.481700+0000","last_clean":"2024-03-12T17:06:30.481700+0000","last_became_active":"2024-03-12T17:06:30.020055+0000","last_became_peered":"2024-03-12T17:06:30.020055+0000","last_unstale":"2024-03-12T17:06:30.481700+0000","last_undegraded":"2024-03-12T17:06:30.481700+0000","last_fullsized":"2024-03-12T17:06:30.481700+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:17:09.055890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.004123+0000","last_change":"2024-03-12T17:06:25.947199+0000","last_active":"2024-03-12T17:06:30.004123+0000","last_peered":"2024-03-12T17:06:30.004123+0000","last_clean":"2024-03-12T17:06:30.004123+0000","last_became_active":"2024-03-12T17:06:25.941750+0000","last_became_peered":"2024-03-12T17:06:25.941750+0000","last_unstale":"2024-03-12T17:06:30.004123+0000","last_undegraded":"2024-03-12T17:06:30.004123+0000","last_fullsized":"2024-03-12T17:06:30.004123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:04:04.198678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.021973+0000","last_change":"2024-03-12T17:06:25.452843+0000","last_active":"2024-03-12T17:06:30.021973+0000","last_peered":"2024-03-12T17:06:30.021973+0000","last_clean":"2024-03-12T17:06:30.021973+0000","last_became_active":"2024-03-12T17:06:25.452410+0000","last_became_peered":"2024-03-12T17:06:25.452410+0000","last_unstale":"2024-03-12T17:06:30.021973+0000","last_undegraded":"2024-03-12T17:06:30.021973+0000","last_fullsized":"2024-03-12T17:06:30.021973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:36:25.468644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.476561+0000","last_change":"2024-03-12T17:06:30.028820+0000","last_active":"2024-03-12T17:06:30.476561+0000","last_peered":"2024-03-12T17:06:30.476561+0000","last_clean":"2024-03-12T17:06:30.476561+0000","last_became_active":"2024-03-12T17:06:30.028425+0000","last_became_peered":"2024-03-12T17:06:30.028425+0000","last_unstale":"2024-03-12T17:06:30.476561+0000","last_undegraded":"2024-03-12T17:06:30.476561+0000","last_fullsized":"2024-03-12T17:06:30.476561+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:46:40.007538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.019530+0000","last_change":"2024-03-12T17:06:25.929758+0000","last_active":"2024-03-12T17:06:30.019530+0000","last_peered":"2024-03-12T17:06:30.019530+0000","last_clean":"2024-03-12T17:06:30.019530+0000","last_became_active":"2024-03-12T17:06:25.929481+0000","last_became_peered":"2024-03-12T17:06:25.929481+0000","last_unstale":"2024-03-12T17:06:30.019530+0000","last_undegraded":"2024-03-12T17:06:30.019530+0000","last_fullsized":"2024-03-12T17:06:30.019530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:44:25.750833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012765+0000","last_change":"2024-03-12T17:06:25.456024+0000","last_active":"2024-03-12T17:06:30.012765+0000","last_peered":"2024-03-12T17:06:30.012765+0000","last_clean":"2024-03-12T17:06:30.012765+0000","last_became_active":"2024-03-12T17:06:25.455718+0000","last_became_peered":"2024-03-12T17:06:25.455718+0000","last_unstale":"2024-03-12T17:06:30.012765+0000","last_undegraded":"2024-03-12T17:06:30.012765+0000","last_fullsized":"2024-03-12T17:06:30.012765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:00:06.114876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.508448+0000","last_change":"2024-03-12T17:06:30.020641+0000","last_active":"2024-03-12T17:06:30.508448+0000","last_peered":"2024-03-12T17:06:30.508448+0000","last_clean":"2024-03-12T17:06:30.508448+0000","last_became_active":"2024-03-12T17:06:30.020373+0000","last_became_peered":"2024-03-12T17:06:30.020373+0000","last_unstale":"2024-03-12T17:06:30.508448+0000","last_undegraded":"2024-03-12T17:06:30.508448+0000","last_fullsized":"2024-03-12T17:06:30.508448+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:03:49.781309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990659+0000","last_change":"2024-03-12T17:06:25.943799+0000","last_active":"2024-03-12T17:06:29.990659+0000","last_peered":"2024-03-12T17:06:29.990659+0000","last_clean":"2024-03-12T17:06:29.990659+0000","last_became_active":"2024-03-12T17:06:25.943543+0000","last_became_peered":"2024-03-12T17:06:25.943543+0000","last_unstale":"2024-03-12T17:06:29.990659+0000","last_undegraded":"2024-03-12T17:06:29.990659+0000","last_fullsized":"2024-03-12T17:06:29.990659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:38:51.036295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.019581+0000","last_change":"2024-03-12T17:06:25.456899+0000","last_active":"2024-03-12T17:06:30.019581+0000","last_peered":"2024-03-12T17:06:30.019581+0000","last_clean":"2024-03-12T17:06:30.019581+0000","last_became_active":"2024-03-12T17:06:25.456513+0000","last_became_peered":"2024-03-12T17:06:25.456513+0000","last_unstale":"2024-03-12T17:06:30.019581+0000","last_undegraded":"2024-03-12T17:06:30.019581+0000","last_fullsized":"2024-03-12T17:06:30.019581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:58:32.509940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.508585+0000","last_change":"2024-03-12T17:06:30.018815+0000","last_active":"2024-03-12T17:06:30.508585+0000","last_peered":"2024-03-12T17:06:30.508585+0000","last_clean":"2024-03-12T17:06:30.508585+0000","last_became_active":"2024-03-12T17:06:30.014225+0000","last_became_peered":"2024-03-12T17:06:30.014225+0000","last_unstale":"2024-03-12T17:06:30.508585+0000","last_undegraded":"2024-03-12T17:06:30.508585+0000","last_fullsized":"2024-03-12T17:06:30.508585+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:49:38.782632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012440+0000","last_change":"2024-03-12T17:06:25.954609+0000","last_active":"2024-03-12T17:06:30.012440+0000","last_peered":"2024-03-12T17:06:30.012440+0000","last_clean":"2024-03-12T17:06:30.012440+0000","last_became_active":"2024-03-12T17:06:25.954337+0000","last_became_peered":"2024-03-12T17:06:25.954337+0000","last_unstale":"2024-03-12T17:06:30.012440+0000","last_undegraded":"2024-03-12T17:06:30.012440+0000","last_fullsized":"2024-03-12T17:06:30.012440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:33:43.674985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.992408+0000","last_change":"2024-03-12T17:06:25.442368+0000","last_active":"2024-03-12T17:06:29.992408+0000","last_peered":"2024-03-12T17:06:29.992408+0000","last_clean":"2024-03-12T17:06:29.992408+0000","last_became_active":"2024-03-12T17:06:25.442084+0000","last_became_peered":"2024-03-12T17:06:25.442084+0000","last_unstale":"2024-03-12T17:06:29.992408+0000","last_undegraded":"2024-03-12T17:06:29.992408+0000","last_fullsized":"2024-03-12T17:06:29.992408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:58:32.509940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.509666+0000","last_change":"2024-03-12T17:06:30.031785+0000","last_active":"2024-03-12T17:06:30.509666+0000","last_peered":"2024-03-12T17:06:30.509666+0000","last_clean":"2024-03-12T17:06:30.509666+0000","last_became_active":"2024-03-12T17:06:30.031328+0000","last_became_peered":"2024-03-12T17:06:30.031328+0000","last_unstale":"2024-03-12T17:06:30.509666+0000","last_undegraded":"2024-03-12T17:06:30.509666+0000","last_fullsized":"2024-03-12T17:06:30.509666+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:35:06.129757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012185+0000","last_change":"2024-03-12T17:06:25.931255+0000","last_active":"2024-03-12T17:06:30.012185+0000","last_peered":"2024-03-12T17:06:30.012185+0000","last_clean":"2024-03-12T17:06:30.012185+0000","last_became_active":"2024-03-12T17:06:25.929387+0000","last_became_peered":"2024-03-12T17:06:25.929387+0000","last_unstale":"2024-03-12T17:06:30.012185+0000","last_undegraded":"2024-03-12T17:06:30.012185+0000","last_fullsized":"2024-03-12T17:06:30.012185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:04:04.198678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.980151+0000","last_change":"2024-03-12T17:06:25.443901+0000","last_active":"2024-03-12T17:06:29.980151+0000","last_peered":"2024-03-12T17:06:29.980151+0000","last_clean":"2024-03-12T17:06:29.980151+0000","last_became_active":"2024-03-12T17:06:25.443602+0000","last_became_peered":"2024-03-12T17:06:25.443602+0000","last_unstale":"2024-03-12T17:06:29.980151+0000","last_undegraded":"2024-03-12T17:06:29.980151+0000","last_fullsized":"2024-03-12T17:06:29.980151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:00:32.347614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.511172+0000","last_change":"2024-03-12T17:06:30.025943+0000","last_active":"2024-03-12T17:06:30.511172+0000","last_peered":"2024-03-12T17:06:30.511172+0000","last_clean":"2024-03-12T17:06:30.511172+0000","last_became_active":"2024-03-12T17:06:30.025481+0000","last_became_peered":"2024-03-12T17:06:30.025481+0000","last_unstale":"2024-03-12T17:06:30.511172+0000","last_undegraded":"2024-03-12T17:06:30.511172+0000","last_fullsized":"2024-03-12T17:06:30.511172+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:17:09.055890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.996440+0000","last_change":"2024-03-12T17:06:25.940996+0000","last_active":"2024-03-12T17:06:29.996440+0000","last_peered":"2024-03-12T17:06:29.996440+0000","last_clean":"2024-03-12T17:06:29.996440+0000","last_became_active":"2024-03-12T17:06:25.940740+0000","last_became_peered":"2024-03-12T17:06:25.940740+0000","last_unstale":"2024-03-12T17:06:29.996440+0000","last_undegraded":"2024-03-12T17:06:29.996440+0000","last_fullsized":"2024-03-12T17:06:29.996440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:36:26.472050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.014590+0000","last_change":"2024-03-12T17:06:25.450168+0000","last_active":"2024-03-12T17:06:30.014590+0000","last_peered":"2024-03-12T17:06:30.014590+0000","last_clean":"2024-03-12T17:06:30.014590+0000","last_became_active":"2024-03-12T17:06:25.449620+0000","last_became_peered":"2024-03-12T17:06:25.449620+0000","last_unstale":"2024-03-12T17:06:30.014590+0000","last_undegraded":"2024-03-12T17:06:30.014590+0000","last_fullsized":"2024-03-12T17:06:30.014590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:58:32.509940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.509124+0000","last_change":"2024-03-12T17:06:30.020545+0000","last_active":"2024-03-12T17:06:30.509124+0000","last_peered":"2024-03-12T17:06:30.509124+0000","last_clean":"2024-03-12T17:06:30.509124+0000","last_became_active":"2024-03-12T17:06:30.020246+0000","last_became_peered":"2024-03-12T17:06:30.020246+0000","last_unstale":"2024-03-12T17:06:30.509124+0000","last_undegraded":"2024-03-12T17:06:30.509124+0000","last_fullsized":"2024-03-12T17:06:30.509124+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:08:39.661179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990996+0000","last_change":"2024-03-12T17:06:25.931831+0000","last_active":"2024-03-12T17:06:29.990996+0000","last_peered":"2024-03-12T17:06:29.990996+0000","last_clean":"2024-03-12T17:06:29.990996+0000","last_became_active":"2024-03-12T17:06:25.931525+0000","last_became_peered":"2024-03-12T17:06:25.931525+0000","last_unstale":"2024-03-12T17:06:29.990996+0000","last_undegraded":"2024-03-12T17:06:29.990996+0000","last_fullsized":"2024-03-12T17:06:29.990996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:06:19.464242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.989265+0000","last_change":"2024-03-12T17:06:25.435909+0000","last_active":"2024-03-12T17:06:29.989265+0000","last_peered":"2024-03-12T17:06:29.989265+0000","last_clean":"2024-03-12T17:06:29.989265+0000","last_became_active":"2024-03-12T17:06:25.434838+0000","last_became_peered":"2024-03-12T17:06:25.434838+0000","last_unstale":"2024-03-12T17:06:29.989265+0000","last_undegraded":"2024-03-12T17:06:29.989265+0000","last_fullsized":"2024-03-12T17:06:29.989265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:36:47.407275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.014690+0000","last_change":"2024-03-12T17:06:21.714813+0000","last_active":"2024-03-12T17:06:30.014690+0000","last_peered":"2024-03-12T17:06:30.014690+0000","last_clean":"2024-03-12T17:06:30.014690+0000","last_became_active":"2024-03-12T17:06:21.712488+0000","last_became_peered":"2024-03-12T17:06:21.712488+0000","last_unstale":"2024-03-12T17:06:30.014690+0000","last_undegraded":"2024-03-12T17:06:30.014690+0000","last_fullsized":"2024-03-12T17:06:30.014690+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:19.815688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:19.815688+0000","last_clean_scrub_stamp":"2024-03-12T17:06:19.815688+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:38:54.300793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.510890+0000","last_change":"2024-03-12T17:06:30.017415+0000","last_active":"2024-03-12T17:06:30.510890+0000","last_peered":"2024-03-12T17:06:30.510890+0000","last_clean":"2024-03-12T17:06:30.510890+0000","last_became_active":"2024-03-12T17:06:30.015976+0000","last_became_peered":"2024-03-12T17:06:30.015976+0000","last_unstale":"2024-03-12T17:06:30.510890+0000","last_undegraded":"2024-03-12T17:06:30.510890+0000","last_fullsized":"2024-03-12T17:06:30.510890+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:33:44.182800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012124+0000","last_change":"2024-03-12T17:06:25.959282+0000","last_active":"2024-03-12T17:06:30.012124+0000","last_peered":"2024-03-12T17:06:30.012124+0000","last_clean":"2024-03-12T17:06:30.012124+0000","last_became_active":"2024-03-12T17:06:25.959028+0000","last_became_peered":"2024-03-12T17:06:25.959028+0000","last_unstale":"2024-03-12T17:06:30.012124+0000","last_undegraded":"2024-03-12T17:06:30.012124+0000","last_fullsized":"2024-03-12T17:06:30.012124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:38:01.478195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.980482+0000","last_change":"2024-03-12T17:06:25.442608+0000","last_active":"2024-03-12T17:06:29.980482+0000","last_peered":"2024-03-12T17:06:29.980482+0000","last_clean":"2024-03-12T17:06:29.980482+0000","last_became_active":"2024-03-12T17:06:25.441652+0000","last_became_peered":"2024-03-12T17:06:25.441652+0000","last_unstale":"2024-03-12T17:06:29.980482+0000","last_undegraded":"2024-03-12T17:06:29.980482+0000","last_fullsized":"2024-03-12T17:06:29.980482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:34:26.277720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.510468+0000","last_change":"2024-03-12T17:06:30.020225+0000","last_active":"2024-03-12T17:06:30.510468+0000","last_peered":"2024-03-12T17:06:30.510468+0000","last_clean":"2024-03-12T17:06:30.510468+0000","last_became_active":"2024-03-12T17:06:30.017277+0000","last_became_peered":"2024-03-12T17:06:30.017277+0000","last_unstale":"2024-03-12T17:06:30.510468+0000","last_undegraded":"2024-03-12T17:06:30.510468+0000","last_fullsized":"2024-03-12T17:06:30.510468+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:35:06.129757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.991439+0000","last_change":"2024-03-12T17:06:25.942345+0000","last_active":"2024-03-12T17:06:29.991439+0000","last_peered":"2024-03-12T17:06:29.991439+0000","last_clean":"2024-03-12T17:06:29.991439+0000","last_became_active":"2024-03-12T17:06:25.941771+0000","last_became_peered":"2024-03-12T17:06:25.941771+0000","last_unstale":"2024-03-12T17:06:29.991439+0000","last_undegraded":"2024-03-12T17:06:29.991439+0000","last_fullsized":"2024-03-12T17:06:29.991439+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:32:08.311478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.022019+0000","last_change":"2024-03-12T17:06:25.448435+0000","last_active":"2024-03-12T17:06:30.022019+0000","last_peered":"2024-03-12T17:06:30.022019+0000","last_clean":"2024-03-12T17:06:30.022019+0000","last_became_active":"2024-03-12T17:06:25.448141+0000","last_became_peered":"2024-03-12T17:06:25.448141+0000","last_unstale":"2024-03-12T17:06:30.022019+0000","last_undegraded":"2024-03-12T17:06:30.022019+0000","last_fullsized":"2024-03-12T17:06:30.022019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:36:47.407275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.508634+0000","last_change":"2024-03-12T17:06:30.017805+0000","last_active":"2024-03-12T17:06:30.508634+0000","last_peered":"2024-03-12T17:06:30.508634+0000","last_clean":"2024-03-12T17:06:30.508634+0000","last_became_active":"2024-03-12T17:06:30.017383+0000","last_became_peered":"2024-03-12T17:06:30.017383+0000","last_unstale":"2024-03-12T17:06:30.508634+0000","last_undegraded":"2024-03-12T17:06:30.508634+0000","last_fullsized":"2024-03-12T17:06:30.508634+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:17:09.055890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.013754+0000","last_change":"2024-03-12T17:06:25.958864+0000","last_active":"2024-03-12T17:06:30.013754+0000","last_peered":"2024-03-12T17:06:30.013754+0000","last_clean":"2024-03-12T17:06:30.013754+0000","last_became_active":"2024-03-12T17:06:25.958588+0000","last_became_peered":"2024-03-12T17:06:25.958588+0000","last_unstale":"2024-03-12T17:06:30.013754+0000","last_undegraded":"2024-03-12T17:06:30.013754+0000","last_fullsized":"2024-03-12T17:06:30.013754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:21:02.182215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.980396+0000","last_change":"2024-03-12T17:06:25.453348+0000","last_active":"2024-03-12T17:06:29.980396+0000","last_peered":"2024-03-12T17:06:29.980396+0000","last_clean":"2024-03-12T17:06:29.980396+0000","last_became_active":"2024-03-12T17:06:25.449918+0000","last_became_peered":"2024-03-12T17:06:25.449918+0000","last_unstale":"2024-03-12T17:06:29.980396+0000","last_undegraded":"2024-03-12T17:06:29.980396+0000","last_fullsized":"2024-03-12T17:06:29.980396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:49:52.284732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.479802+0000","last_change":"2024-03-12T17:06:30.023418+0000","last_active":"2024-03-12T17:06:30.479802+0000","last_peered":"2024-03-12T17:06:30.479802+0000","last_clean":"2024-03-12T17:06:30.479802+0000","last_became_active":"2024-03-12T17:06:30.022919+0000","last_became_peered":"2024-03-12T17:06:30.022919+0000","last_unstale":"2024-03-12T17:06:30.479802+0000","last_undegraded":"2024-03-12T17:06:30.479802+0000","last_fullsized":"2024-03-12T17:06:30.479802+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:13:55.311599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.015933+0000","last_change":"2024-03-12T17:06:25.961193+0000","last_active":"2024-03-12T17:06:30.015933+0000","last_peered":"2024-03-12T17:06:30.015933+0000","last_clean":"2024-03-12T17:06:30.015933+0000","last_became_active":"2024-03-12T17:06:25.960906+0000","last_became_peered":"2024-03-12T17:06:25.960906+0000","last_unstale":"2024-03-12T17:06:30.015933+0000","last_undegraded":"2024-03-12T17:06:30.015933+0000","last_fullsized":"2024-03-12T17:06:30.015933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:17:09.067136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.013327+0000","last_change":"2024-03-12T17:06:25.442363+0000","last_active":"2024-03-12T17:06:30.013327+0000","last_peered":"2024-03-12T17:06:30.013327+0000","last_clean":"2024-03-12T17:06:30.013327+0000","last_became_active":"2024-03-12T17:06:25.442053+0000","last_became_peered":"2024-03-12T17:06:25.442053+0000","last_unstale":"2024-03-12T17:06:30.013327+0000","last_undegraded":"2024-03-12T17:06:30.013327+0000","last_fullsized":"2024-03-12T17:06:30.013327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:06:05.702956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.481199+0000","last_change":"2024-03-12T17:06:30.020741+0000","last_active":"2024-03-12T17:06:30.481199+0000","last_peered":"2024-03-12T17:06:30.481199+0000","last_clean":"2024-03-12T17:06:30.481199+0000","last_became_active":"2024-03-12T17:06:30.018043+0000","last_became_peered":"2024-03-12T17:06:30.018043+0000","last_unstale":"2024-03-12T17:06:30.481199+0000","last_undegraded":"2024-03-12T17:06:30.481199+0000","last_fullsized":"2024-03-12T17:06:30.481199+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:11:34.105824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.016104+0000","last_change":"2024-03-12T17:06:25.958260+0000","last_active":"2024-03-12T17:06:30.016104+0000","last_peered":"2024-03-12T17:06:30.016104+0000","last_clean":"2024-03-12T17:06:30.016104+0000","last_became_active":"2024-03-12T17:06:25.955966+0000","last_became_peered":"2024-03-12T17:06:25.955966+0000","last_unstale":"2024-03-12T17:06:30.016104+0000","last_undegraded":"2024-03-12T17:06:30.016104+0000","last_fullsized":"2024-03-12T17:06:30.016104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:33:16.553805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.979851+0000","last_change":"2024-03-12T17:06:25.452222+0000","last_active":"2024-03-12T17:06:29.979851+0000","last_peered":"2024-03-12T17:06:29.979851+0000","last_clean":"2024-03-12T17:06:29.979851+0000","last_became_active":"2024-03-12T17:06:25.451696+0000","last_became_peered":"2024-03-12T17:06:25.451696+0000","last_unstale":"2024-03-12T17:06:29.979851+0000","last_undegraded":"2024-03-12T17:06:29.979851+0000","last_fullsized":"2024-03-12T17:06:29.979851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:28:25.809578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.508879+0000","last_change":"2024-03-12T17:06:30.009551+0000","last_active":"2024-03-12T17:06:30.508879+0000","last_peered":"2024-03-12T17:06:30.508879+0000","last_clean":"2024-03-12T17:06:30.508879+0000","last_became_active":"2024-03-12T17:06:30.009269+0000","last_became_peered":"2024-03-12T17:06:30.009269+0000","last_unstale":"2024-03-12T17:06:30.508879+0000","last_undegraded":"2024-03-12T17:06:30.508879+0000","last_fullsized":"2024-03-12T17:06:30.508879+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:42:49.327905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.984652+0000","last_change":"2024-03-12T17:06:25.949275+0000","last_active":"2024-03-12T17:06:29.984652+0000","last_peered":"2024-03-12T17:06:29.984652+0000","last_clean":"2024-03-12T17:06:29.984652+0000","last_became_active":"2024-03-12T17:06:25.947884+0000","last_became_peered":"2024-03-12T17:06:25.947884+0000","last_unstale":"2024-03-12T17:06:29.984652+0000","last_undegraded":"2024-03-12T17:06:29.984652+0000","last_fullsized":"2024-03-12T17:06:29.984652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:28:59.465721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.984927+0000","last_change":"2024-03-12T17:06:25.450408+0000","last_active":"2024-03-12T17:06:29.984927+0000","last_peered":"2024-03-12T17:06:29.984927+0000","last_clean":"2024-03-12T17:06:29.984927+0000","last_became_active":"2024-03-12T17:06:25.448867+0000","last_became_peered":"2024-03-12T17:06:25.448867+0000","last_unstale":"2024-03-12T17:06:29.984927+0000","last_undegraded":"2024-03-12T17:06:29.984927+0000","last_fullsized":"2024-03-12T17:06:29.984927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:49:52.284732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.477643+0000","last_change":"2024-03-12T17:06:30.022424+0000","last_active":"2024-03-12T17:06:30.477643+0000","last_peered":"2024-03-12T17:06:30.477643+0000","last_clean":"2024-03-12T17:06:30.477643+0000","last_became_active":"2024-03-12T17:06:30.021994+0000","last_became_peered":"2024-03-12T17:06:30.021994+0000","last_unstale":"2024-03-12T17:06:30.477643+0000","last_undegraded":"2024-03-12T17:06:30.477643+0000","last_fullsized":"2024-03-12T17:06:30.477643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:32:37.399700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012128+0000","last_change":"2024-03-12T17:06:25.958282+0000","last_active":"2024-03-12T17:06:30.012128+0000","last_peered":"2024-03-12T17:06:30.012128+0000","last_clean":"2024-03-12T17:06:30.012128+0000","last_became_active":"2024-03-12T17:06:25.955357+0000","last_became_peered":"2024-03-12T17:06:25.955357+0000","last_unstale":"2024-03-12T17:06:30.012128+0000","last_undegraded":"2024-03-12T17:06:30.012128+0000","last_fullsized":"2024-03-12T17:06:30.012128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:15:36.195143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.992283+0000","last_change":"2024-03-12T17:06:25.448887+0000","last_active":"2024-03-12T17:06:29.992283+0000","last_peered":"2024-03-12T17:06:29.992283+0000","last_clean":"2024-03-12T17:06:29.992283+0000","last_became_active":"2024-03-12T17:06:25.448299+0000","last_became_peered":"2024-03-12T17:06:25.448299+0000","last_unstale":"2024-03-12T17:06:29.992283+0000","last_undegraded":"2024-03-12T17:06:29.992283+0000","last_fullsized":"2024-03-12T17:06:29.992283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:00:06.114876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.477924+0000","last_change":"2024-03-12T17:06:30.022221+0000","last_active":"2024-03-12T17:06:30.477924+0000","last_peered":"2024-03-12T17:06:30.477924+0000","last_clean":"2024-03-12T17:06:30.477924+0000","last_became_active":"2024-03-12T17:06:30.021765+0000","last_became_peered":"2024-03-12T17:06:30.021765+0000","last_unstale":"2024-03-12T17:06:30.477924+0000","last_undegraded":"2024-03-12T17:06:30.477924+0000","last_fullsized":"2024-03-12T17:06:30.477924+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:28:34.686064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.996652+0000","last_change":"2024-03-12T17:06:25.944466+0000","last_active":"2024-03-12T17:06:29.996652+0000","last_peered":"2024-03-12T17:06:29.996652+0000","last_clean":"2024-03-12T17:06:29.996652+0000","last_became_active":"2024-03-12T17:06:25.944075+0000","last_became_peered":"2024-03-12T17:06:25.944075+0000","last_unstale":"2024-03-12T17:06:29.996652+0000","last_undegraded":"2024-03-12T17:06:29.996652+0000","last_fullsized":"2024-03-12T17:06:29.996652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T20:48:48.332409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.996480+0000","last_change":"2024-03-12T17:06:25.444436+0000","last_active":"2024-03-12T17:06:29.996480+0000","last_peered":"2024-03-12T17:06:29.996480+0000","last_clean":"2024-03-12T17:06:29.996480+0000","last_became_active":"2024-03-12T17:06:25.443700+0000","last_became_peered":"2024-03-12T17:06:25.443700+0000","last_unstale":"2024-03-12T17:06:29.996480+0000","last_undegraded":"2024-03-12T17:06:29.996480+0000","last_fullsized":"2024-03-12T17:06:29.996480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:55:17.158520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.483163+0000","last_change":"2024-03-12T17:06:30.022439+0000","last_active":"2024-03-12T17:06:30.483163+0000","last_peered":"2024-03-12T17:06:30.483163+0000","last_clean":"2024-03-12T17:06:30.483163+0000","last_became_active":"2024-03-12T17:06:30.017040+0000","last_became_peered":"2024-03-12T17:06:30.017040+0000","last_unstale":"2024-03-12T17:06:30.483163+0000","last_undegraded":"2024-03-12T17:06:30.483163+0000","last_fullsized":"2024-03-12T17:06:30.483163+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:08:39.661179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012020+0000","last_change":"2024-03-12T17:06:25.958251+0000","last_active":"2024-03-12T17:06:30.012020+0000","last_peered":"2024-03-12T17:06:30.012020+0000","last_clean":"2024-03-12T17:06:30.012020+0000","last_became_active":"2024-03-12T17:06:25.956505+0000","last_became_peered":"2024-03-12T17:06:25.956505+0000","last_unstale":"2024-03-12T17:06:30.012020+0000","last_undegraded":"2024-03-12T17:06:30.012020+0000","last_fullsized":"2024-03-12T17:06:30.012020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:32:08.311478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:44.758048+0000","last_change":"2024-03-12T17:06:25.448251+0000","last_active":"2024-03-12T17:06:44.758048+0000","last_peered":"2024-03-12T17:06:44.758048+0000","last_clean":"2024-03-12T17:06:44.758048+0000","last_became_active":"2024-03-12T17:06:25.447768+0000","last_became_peered":"2024-03-12T17:06:25.447768+0000","last_unstale":"2024-03-12T17:06:44.758048+0000","last_undegraded":"2024-03-12T17:06:44.758048+0000","last_fullsized":"2024-03-12T17:06:44.758048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:34:26.277720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3795,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.508684+0000","last_change":"2024-03-12T17:06:30.019328+0000","last_active":"2024-03-12T17:06:30.508684+0000","last_peered":"2024-03-12T17:06:30.508684+0000","last_clean":"2024-03-12T17:06:30.508684+0000","last_became_active":"2024-03-12T17:06:30.014815+0000","last_became_peered":"2024-03-12T17:06:30.014815+0000","last_unstale":"2024-03-12T17:06:30.508684+0000","last_undegraded":"2024-03-12T17:06:30.508684+0000","last_fullsized":"2024-03-12T17:06:30.508684+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:08:39.661179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012873+0000","last_change":"2024-03-12T17:06:25.945208+0000","last_active":"2024-03-12T17:06:30.012873+0000","last_peered":"2024-03-12T17:06:30.012873+0000","last_clean":"2024-03-12T17:06:30.012873+0000","last_became_active":"2024-03-12T17:06:25.944043+0000","last_became_peered":"2024-03-12T17:06:25.944043+0000","last_unstale":"2024-03-12T17:06:30.012873+0000","last_undegraded":"2024-03-12T17:06:30.012873+0000","last_fullsized":"2024-03-12T17:06:30.012873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:36:48.410681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.021900+0000","last_change":"2024-03-12T17:06:25.455080+0000","last_active":"2024-03-12T17:06:30.021900+0000","last_peered":"2024-03-12T17:06:30.021900+0000","last_clean":"2024-03-12T17:06:30.021900+0000","last_became_active":"2024-03-12T17:06:25.454788+0000","last_became_peered":"2024-03-12T17:06:25.454788+0000","last_unstale":"2024-03-12T17:06:30.021900+0000","last_undegraded":"2024-03-12T17:06:30.021900+0000","last_fullsized":"2024-03-12T17:06:30.021900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:33:42.671579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.482470+0000","last_change":"2024-03-12T17:06:30.018328+0000","last_active":"2024-03-12T17:06:30.482470+0000","last_peered":"2024-03-12T17:06:30.482470+0000","last_clean":"2024-03-12T17:06:30.482470+0000","last_became_active":"2024-03-12T17:06:30.017852+0000","last_became_peered":"2024-03-12T17:06:30.017852+0000","last_unstale":"2024-03-12T17:06:30.482470+0000","last_undegraded":"2024-03-12T17:06:30.482470+0000","last_fullsized":"2024-03-12T17:06:30.482470+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T02:49:07.614077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.999490+0000","last_change":"2024-03-12T17:06:25.938178+0000","last_active":"2024-03-12T17:06:29.999490+0000","last_peered":"2024-03-12T17:06:29.999490+0000","last_clean":"2024-03-12T17:06:29.999490+0000","last_became_active":"2024-03-12T17:06:25.937926+0000","last_became_peered":"2024-03-12T17:06:25.937926+0000","last_unstale":"2024-03-12T17:06:29.999490+0000","last_undegraded":"2024-03-12T17:06:29.999490+0000","last_fullsized":"2024-03-12T17:06:29.999490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:57:56.009742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990778+0000","last_change":"2024-03-12T17:06:25.451993+0000","last_active":"2024-03-12T17:06:29.990778+0000","last_peered":"2024-03-12T17:06:29.990778+0000","last_clean":"2024-03-12T17:06:29.990778+0000","last_became_active":"2024-03-12T17:06:25.451794+0000","last_became_peered":"2024-03-12T17:06:25.451794+0000","last_unstale":"2024-03-12T17:06:29.990778+0000","last_undegraded":"2024-03-12T17:06:29.990778+0000","last_fullsized":"2024-03-12T17:06:29.990778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:33:42.671579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.476846+0000","last_change":"2024-03-12T17:06:30.016232+0000","last_active":"2024-03-12T17:06:30.476846+0000","last_peered":"2024-03-12T17:06:30.476846+0000","last_clean":"2024-03-12T17:06:30.476846+0000","last_became_active":"2024-03-12T17:06:30.015825+0000","last_became_peered":"2024-03-12T17:06:30.015825+0000","last_unstale":"2024-03-12T17:06:30.476846+0000","last_undegraded":"2024-03-12T17:06:30.476846+0000","last_fullsized":"2024-03-12T17:06:30.476846+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:28:28.745853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990537+0000","last_change":"2024-03-12T17:06:25.938249+0000","last_active":"2024-03-12T17:06:29.990537+0000","last_peered":"2024-03-12T17:06:29.990537+0000","last_clean":"2024-03-12T17:06:29.990537+0000","last_became_active":"2024-03-12T17:06:25.937744+0000","last_became_peered":"2024-03-12T17:06:25.937744+0000","last_unstale":"2024-03-12T17:06:29.990537+0000","last_undegraded":"2024-03-12T17:06:29.990537+0000","last_fullsized":"2024-03-12T17:06:29.990537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:06:19.464242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990879+0000","last_change":"2024-03-12T17:06:25.452356+0000","last_active":"2024-03-12T17:06:29.990879+0000","last_peered":"2024-03-12T17:06:29.990879+0000","last_clean":"2024-03-12T17:06:29.990879+0000","last_became_active":"2024-03-12T17:06:25.452033+0000","last_became_peered":"2024-03-12T17:06:25.452033+0000","last_unstale":"2024-03-12T17:06:29.990879+0000","last_undegraded":"2024-03-12T17:06:29.990879+0000","last_fullsized":"2024-03-12T17:06:29.990879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:58:32.509940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.477019+0000","last_change":"2024-03-12T17:06:30.023373+0000","last_active":"2024-03-12T17:06:30.477019+0000","last_peered":"2024-03-12T17:06:30.477019+0000","last_clean":"2024-03-12T17:06:30.477019+0000","last_became_active":"2024-03-12T17:06:30.022331+0000","last_became_peered":"2024-03-12T17:06:30.022331+0000","last_unstale":"2024-03-12T17:06:30.477019+0000","last_undegraded":"2024-03-12T17:06:30.477019+0000","last_fullsized":"2024-03-12T17:06:30.477019+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:50:22.021457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.508638+0000","last_change":"2024-03-12T17:06:30.020503+0000","last_active":"2024-03-12T17:06:30.508638+0000","last_peered":"2024-03-12T17:06:30.508638+0000","last_clean":"2024-03-12T17:06:30.508638+0000","last_became_active":"2024-03-12T17:06:30.019897+0000","last_became_peered":"2024-03-12T17:06:30.019897+0000","last_unstale":"2024-03-12T17:06:30.508638+0000","last_undegraded":"2024-03-12T17:06:30.508638+0000","last_fullsized":"2024-03-12T17:06:30.508638+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:13:55.311599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.013985+0000","last_change":"2024-03-12T17:06:25.927847+0000","last_active":"2024-03-12T17:06:30.013985+0000","last_peered":"2024-03-12T17:06:30.013985+0000","last_clean":"2024-03-12T17:06:30.013985+0000","last_became_active":"2024-03-12T17:06:25.927572+0000","last_became_peered":"2024-03-12T17:06:25.927572+0000","last_unstale":"2024-03-12T17:06:30.013985+0000","last_undegraded":"2024-03-12T17:06:30.013985+0000","last_fullsized":"2024-03-12T17:06:30.013985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:29:49.461651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.989420+0000","last_change":"2024-03-12T17:06:25.443473+0000","last_active":"2024-03-12T17:06:29.989420+0000","last_peered":"2024-03-12T17:06:29.989420+0000","last_clean":"2024-03-12T17:06:29.989420+0000","last_became_active":"2024-03-12T17:06:25.443046+0000","last_became_peered":"2024-03-12T17:06:25.443046+0000","last_unstale":"2024-03-12T17:06:29.989420+0000","last_undegraded":"2024-03-12T17:06:29.989420+0000","last_fullsized":"2024-03-12T17:06:29.989420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:49:52.284732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.513045+0000","last_change":"2024-03-12T17:06:30.017610+0000","last_active":"2024-03-12T17:06:30.513045+0000","last_peered":"2024-03-12T17:06:30.513045+0000","last_clean":"2024-03-12T17:06:30.513045+0000","last_became_active":"2024-03-12T17:06:30.017119+0000","last_became_peered":"2024-03-12T17:06:30.017119+0000","last_unstale":"2024-03-12T17:06:30.513045+0000","last_undegraded":"2024-03-12T17:06:30.513045+0000","last_fullsized":"2024-03-12T17:06:30.513045+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:42:49.327905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.986700+0000","last_change":"2024-03-12T17:06:25.935139+0000","last_active":"2024-03-12T17:06:29.986700+0000","last_peered":"2024-03-12T17:06:29.986700+0000","last_clean":"2024-03-12T17:06:29.986700+0000","last_became_active":"2024-03-12T17:06:25.934842+0000","last_became_peered":"2024-03-12T17:06:25.934842+0000","last_unstale":"2024-03-12T17:06:29.986700+0000","last_undegraded":"2024-03-12T17:06:29.986700+0000","last_fullsized":"2024-03-12T17:06:29.986700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:28:59.465721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.989691+0000","last_change":"2024-03-12T17:06:25.447398+0000","last_active":"2024-03-12T17:06:29.989691+0000","last_peered":"2024-03-12T17:06:29.989691+0000","last_clean":"2024-03-12T17:06:29.989691+0000","last_became_active":"2024-03-12T17:06:25.447080+0000","last_became_peered":"2024-03-12T17:06:25.447080+0000","last_unstale":"2024-03-12T17:06:29.989691+0000","last_undegraded":"2024-03-12T17:06:29.989691+0000","last_fullsized":"2024-03-12T17:06:29.989691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:00:32.347614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.986717+0000","last_change":"2024-03-12T17:06:25.447858+0000","last_active":"2024-03-12T17:06:29.986717+0000","last_peered":"2024-03-12T17:06:29.986717+0000","last_clean":"2024-03-12T17:06:29.986717+0000","last_became_active":"2024-03-12T17:06:25.447551+0000","last_became_peered":"2024-03-12T17:06:25.447551+0000","last_unstale":"2024-03-12T17:06:29.986717+0000","last_undegraded":"2024-03-12T17:06:29.986717+0000","last_fullsized":"2024-03-12T17:06:29.986717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:28:25.809578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.996427+0000","last_change":"2024-03-12T17:06:25.943160+0000","last_active":"2024-03-12T17:06:29.996427+0000","last_peered":"2024-03-12T17:06:29.996427+0000","last_clean":"2024-03-12T17:06:29.996427+0000","last_became_active":"2024-03-12T17:06:25.942883+0000","last_became_peered":"2024-03-12T17:06:25.942883+0000","last_unstale":"2024-03-12T17:06:29.996427+0000","last_undegraded":"2024-03-12T17:06:29.996427+0000","last_fullsized":"2024-03-12T17:06:29.996427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:04:04.198678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.509831+0000","last_change":"2024-03-12T17:06:30.018594+0000","last_active":"2024-03-12T17:06:30.509831+0000","last_peered":"2024-03-12T17:06:30.509831+0000","last_clean":"2024-03-12T17:06:30.509831+0000","last_became_active":"2024-03-12T17:06:30.016503+0000","last_became_peered":"2024-03-12T17:06:30.016503+0000","last_unstale":"2024-03-12T17:06:30.509831+0000","last_undegraded":"2024-03-12T17:06:30.509831+0000","last_fullsized":"2024-03-12T17:06:30.509831+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:29:03.020890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.995919+0000","last_change":"2024-03-12T17:06:25.443901+0000","last_active":"2024-03-12T17:06:29.995919+0000","last_peered":"2024-03-12T17:06:29.995919+0000","last_clean":"2024-03-12T17:06:29.995919+0000","last_became_active":"2024-03-12T17:06:25.443607+0000","last_became_peered":"2024-03-12T17:06:25.443607+0000","last_unstale":"2024-03-12T17:06:29.995919+0000","last_undegraded":"2024-03-12T17:06:29.995919+0000","last_fullsized":"2024-03-12T17:06:29.995919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T05:00:49.177317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.011582+0000","last_change":"2024-03-12T17:06:25.931011+0000","last_active":"2024-03-12T17:06:30.011582+0000","last_peered":"2024-03-12T17:06:30.011582+0000","last_clean":"2024-03-12T17:06:30.011582+0000","last_became_active":"2024-03-12T17:06:25.930306+0000","last_became_peered":"2024-03-12T17:06:25.930306+0000","last_unstale":"2024-03-12T17:06:30.011582+0000","last_undegraded":"2024-03-12T17:06:30.011582+0000","last_fullsized":"2024-03-12T17:06:30.011582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:58:33.513346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.478204+0000","last_change":"2024-03-12T17:06:30.018640+0000","last_active":"2024-03-12T17:06:30.478204+0000","last_peered":"2024-03-12T17:06:30.478204+0000","last_clean":"2024-03-12T17:06:30.478204+0000","last_became_active":"2024-03-12T17:06:30.018278+0000","last_became_peered":"2024-03-12T17:06:30.018278+0000","last_unstale":"2024-03-12T17:06:30.478204+0000","last_undegraded":"2024-03-12T17:06:30.478204+0000","last_fullsized":"2024-03-12T17:06:30.478204+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:01:05.684140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.984555+0000","last_change":"2024-03-12T17:06:25.947304+0000","last_active":"2024-03-12T17:06:29.984555+0000","last_peered":"2024-03-12T17:06:29.984555+0000","last_clean":"2024-03-12T17:06:29.984555+0000","last_became_active":"2024-03-12T17:06:25.947030+0000","last_became_peered":"2024-03-12T17:06:25.947030+0000","last_unstale":"2024-03-12T17:06:29.984555+0000","last_undegraded":"2024-03-12T17:06:29.984555+0000","last_fullsized":"2024-03-12T17:06:29.984555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:29:49.461651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.979886+0000","last_change":"2024-03-12T17:06:25.929080+0000","last_active":"2024-03-12T17:06:29.979886+0000","last_peered":"2024-03-12T17:06:29.979886+0000","last_clean":"2024-03-12T17:06:29.979886+0000","last_became_active":"2024-03-12T17:06:25.928820+0000","last_became_peered":"2024-03-12T17:06:25.928820+0000","last_unstale":"2024-03-12T17:06:29.979886+0000","last_undegraded":"2024-03-12T17:06:29.979886+0000","last_fullsized":"2024-03-12T17:06:29.979886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:36:26.472050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.992041+0000","last_change":"2024-03-12T17:06:25.932309+0000","last_active":"2024-03-12T17:06:29.992041+0000","last_peered":"2024-03-12T17:06:29.992041+0000","last_clean":"2024-03-12T17:06:29.992041+0000","last_became_active":"2024-03-12T17:06:25.932014+0000","last_became_peered":"2024-03-12T17:06:25.932014+0000","last_unstale":"2024-03-12T17:06:29.992041+0000","last_undegraded":"2024-03-12T17:06:29.992041+0000","last_fullsized":"2024-03-12T17:06:29.992041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:11:15.145295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.996376+0000","last_change":"2024-03-12T17:06:25.937407+0000","last_active":"2024-03-12T17:06:29.996376+0000","last_peered":"2024-03-12T17:06:29.996376+0000","last_clean":"2024-03-12T17:06:29.996376+0000","last_became_active":"2024-03-12T17:06:25.937065+0000","last_became_peered":"2024-03-12T17:06:25.937065+0000","last_unstale":"2024-03-12T17:06:29.996376+0000","last_undegraded":"2024-03-12T17:06:29.996376+0000","last_fullsized":"2024-03-12T17:06:29.996376+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:04:28.921316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.994102+0000","last_change":"2024-03-12T17:06:25.930358+0000","last_active":"2024-03-12T17:06:29.994102+0000","last_peered":"2024-03-12T17:06:29.994102+0000","last_clean":"2024-03-12T17:06:29.994102+0000","last_became_active":"2024-03-12T17:06:25.929870+0000","last_became_peered":"2024-03-12T17:06:25.929870+0000","last_unstale":"2024-03-12T17:06:29.994102+0000","last_undegraded":"2024-03-12T17:06:29.994102+0000","last_fullsized":"2024-03-12T17:06:29.994102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:57:56.009742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.021765+0000","last_change":"2024-03-12T17:06:25.928987+0000","last_active":"2024-03-12T17:06:30.021765+0000","last_peered":"2024-03-12T17:06:30.021765+0000","last_clean":"2024-03-12T17:06:30.021765+0000","last_became_active":"2024-03-12T17:06:25.928717+0000","last_became_peered":"2024-03-12T17:06:25.928717+0000","last_unstale":"2024-03-12T17:06:30.021765+0000","last_undegraded":"2024-03-12T17:06:30.021765+0000","last_fullsized":"2024-03-12T17:06:30.021765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:23:25.984714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990238+0000","last_change":"2024-03-12T17:06:25.943871+0000","last_active":"2024-03-12T17:06:29.990238+0000","last_peered":"2024-03-12T17:06:29.990238+0000","last_clean":"2024-03-12T17:06:29.990238+0000","last_became_active":"2024-03-12T17:06:25.938802+0000","last_became_peered":"2024-03-12T17:06:25.938802+0000","last_unstale":"2024-03-12T17:06:29.990238+0000","last_undegraded":"2024-03-12T17:06:29.990238+0000","last_fullsized":"2024-03-12T17:06:29.990238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:23:25.984714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.004028+0000","last_change":"2024-03-12T17:06:25.935272+0000","last_active":"2024-03-12T17:06:30.004028+0000","last_peered":"2024-03-12T17:06:30.004028+0000","last_clean":"2024-03-12T17:06:30.004028+0000","last_became_active":"2024-03-12T17:06:25.928645+0000","last_became_peered":"2024-03-12T17:06:25.928645+0000","last_unstale":"2024-03-12T17:06:30.004028+0000","last_undegraded":"2024-03-12T17:06:30.004028+0000","last_fullsized":"2024-03-12T17:06:30.004028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:44:25.750833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.989166+0000","last_change":"2024-03-12T17:06:25.937035+0000","last_active":"2024-03-12T17:06:29.989166+0000","last_peered":"2024-03-12T17:06:29.989166+0000","last_clean":"2024-03-12T17:06:29.989166+0000","last_became_active":"2024-03-12T17:06:25.936598+0000","last_became_peered":"2024-03-12T17:06:25.936598+0000","last_unstale":"2024-03-12T17:06:29.989166+0000","last_undegraded":"2024-03-12T17:06:29.989166+0000","last_fullsized":"2024-03-12T17:06:29.989166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:21:02.182215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.989614+0000","last_change":"2024-03-12T17:06:25.931310+0000","last_active":"2024-03-12T17:06:29.989614+0000","last_peered":"2024-03-12T17:06:29.989614+0000","last_clean":"2024-03-12T17:06:29.989614+0000","last_became_active":"2024-03-12T17:06:25.930910+0000","last_became_peered":"2024-03-12T17:06:25.930910+0000","last_unstale":"2024-03-12T17:06:29.989614+0000","last_undegraded":"2024-03-12T17:06:29.989614+0000","last_fullsized":"2024-03-12T17:06:29.989614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:48:33.016370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.991821+0000","last_change":"2024-03-12T17:06:25.937111+0000","last_active":"2024-03-12T17:06:29.991821+0000","last_peered":"2024-03-12T17:06:29.991821+0000","last_clean":"2024-03-12T17:06:29.991821+0000","last_became_active":"2024-03-12T17:06:25.936895+0000","last_became_peered":"2024-03-12T17:06:25.936895+0000","last_unstale":"2024-03-12T17:06:29.991821+0000","last_undegraded":"2024-03-12T17:06:29.991821+0000","last_fullsized":"2024-03-12T17:06:29.991821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:38:01.478195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.996309+0000","last_change":"2024-03-12T17:06:25.947115+0000","last_active":"2024-03-12T17:06:29.996309+0000","last_peered":"2024-03-12T17:06:29.996309+0000","last_clean":"2024-03-12T17:06:29.996309+0000","last_became_active":"2024-03-12T17:06:25.946820+0000","last_became_peered":"2024-03-12T17:06:25.946820+0000","last_unstale":"2024-03-12T17:06:29.996309+0000","last_undegraded":"2024-03-12T17:06:29.996309+0000","last_fullsized":"2024-03-12T17:06:29.996309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:48:33.016370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.984992+0000","last_change":"2024-03-12T17:06:25.950973+0000","last_active":"2024-03-12T17:06:29.984992+0000","last_peered":"2024-03-12T17:06:29.984992+0000","last_clean":"2024-03-12T17:06:29.984992+0000","last_became_active":"2024-03-12T17:06:25.948336+0000","last_became_peered":"2024-03-12T17:06:25.948336+0000","last_unstale":"2024-03-12T17:06:29.984992+0000","last_undegraded":"2024-03-12T17:06:29.984992+0000","last_fullsized":"2024-03-12T17:06:29.984992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:04:04.198678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012237+0000","last_change":"2024-03-12T17:06:25.961536+0000","last_active":"2024-03-12T17:06:30.012237+0000","last_peered":"2024-03-12T17:06:30.012237+0000","last_clean":"2024-03-12T17:06:30.012237+0000","last_became_active":"2024-03-12T17:06:25.961147+0000","last_became_peered":"2024-03-12T17:06:25.961147+0000","last_unstale":"2024-03-12T17:06:30.012237+0000","last_undegraded":"2024-03-12T17:06:30.012237+0000","last_fullsized":"2024-03-12T17:06:30.012237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:06:19.464242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.979994+0000","last_change":"2024-03-12T17:06:25.939977+0000","last_active":"2024-03-12T17:06:29.979994+0000","last_peered":"2024-03-12T17:06:29.979994+0000","last_clean":"2024-03-12T17:06:29.979994+0000","last_became_active":"2024-03-12T17:06:25.939725+0000","last_became_peered":"2024-03-12T17:06:25.939725+0000","last_unstale":"2024-03-12T17:06:29.979994+0000","last_undegraded":"2024-03-12T17:06:29.979994+0000","last_fullsized":"2024-03-12T17:06:29.979994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:45:02.929901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.991517+0000","last_change":"2024-03-12T17:06:25.939649+0000","last_active":"2024-03-12T17:06:29.991517+0000","last_peered":"2024-03-12T17:06:29.991517+0000","last_clean":"2024-03-12T17:06:29.991517+0000","last_became_active":"2024-03-12T17:06:25.939379+0000","last_became_peered":"2024-03-12T17:06:25.939379+0000","last_unstale":"2024-03-12T17:06:29.991517+0000","last_undegraded":"2024-03-12T17:06:29.991517+0000","last_fullsized":"2024-03-12T17:06:29.991517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:17:09.067136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.988936+0000","last_change":"2024-03-12T17:06:25.948556+0000","last_active":"2024-03-12T17:06:29.988936+0000","last_peered":"2024-03-12T17:06:29.988936+0000","last_clean":"2024-03-12T17:06:29.988936+0000","last_became_active":"2024-03-12T17:06:25.948217+0000","last_became_peered":"2024-03-12T17:06:25.948217+0000","last_unstale":"2024-03-12T17:06:29.988936+0000","last_undegraded":"2024-03-12T17:06:29.988936+0000","last_fullsized":"2024-03-12T17:06:29.988936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:26:54.235648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.013984+0000","last_change":"2024-03-12T17:06:25.931098+0000","last_active":"2024-03-12T17:06:30.013984+0000","last_peered":"2024-03-12T17:06:30.013984+0000","last_clean":"2024-03-12T17:06:30.013984+0000","last_became_active":"2024-03-12T17:06:25.930273+0000","last_became_peered":"2024-03-12T17:06:25.930273+0000","last_unstale":"2024-03-12T17:06:30.013984+0000","last_undegraded":"2024-03-12T17:06:30.013984+0000","last_fullsized":"2024-03-12T17:06:30.013984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T20:48:48.332409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.996609+0000","last_change":"2024-03-12T17:06:25.933421+0000","last_active":"2024-03-12T17:06:29.996609+0000","last_peered":"2024-03-12T17:06:29.996609+0000","last_clean":"2024-03-12T17:06:29.996609+0000","last_became_active":"2024-03-12T17:06:25.933133+0000","last_became_peered":"2024-03-12T17:06:25.933133+0000","last_unstale":"2024-03-12T17:06:29.996609+0000","last_undegraded":"2024-03-12T17:06:29.996609+0000","last_fullsized":"2024-03-12T17:06:29.996609+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:28:59.465721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.991543+0000","last_change":"2024-03-12T17:06:25.948622+0000","last_active":"2024-03-12T17:06:29.991543+0000","last_peered":"2024-03-12T17:06:29.991543+0000","last_clean":"2024-03-12T17:06:29.991543+0000","last_became_active":"2024-03-12T17:06:25.948293+0000","last_became_peered":"2024-03-12T17:06:25.948293+0000","last_unstale":"2024-03-12T17:06:29.991543+0000","last_undegraded":"2024-03-12T17:06:29.991543+0000","last_fullsized":"2024-03-12T17:06:29.991543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:52:14.405796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012611+0000","last_change":"2024-03-12T17:06:25.944171+0000","last_active":"2024-03-12T17:06:30.012611+0000","last_peered":"2024-03-12T17:06:30.012611+0000","last_clean":"2024-03-12T17:06:30.012611+0000","last_became_active":"2024-03-12T17:06:25.940309+0000","last_became_peered":"2024-03-12T17:06:25.940309+0000","last_unstale":"2024-03-12T17:06:30.012611+0000","last_undegraded":"2024-03-12T17:06:30.012611+0000","last_fullsized":"2024-03-12T17:06:30.012611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:00:07.118282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.987793+0000","last_change":"2024-03-12T17:06:25.946194+0000","last_active":"2024-03-12T17:06:29.987793+0000","last_peered":"2024-03-12T17:06:29.987793+0000","last_clean":"2024-03-12T17:06:29.987793+0000","last_became_active":"2024-03-12T17:06:25.944383+0000","last_became_peered":"2024-03-12T17:06:25.944383+0000","last_unstale":"2024-03-12T17:06:29.987793+0000","last_undegraded":"2024-03-12T17:06:29.987793+0000","last_fullsized":"2024-03-12T17:06:29.987793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:48:33.016370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.989993+0000","last_change":"2024-03-12T17:06:25.936660+0000","last_active":"2024-03-12T17:06:29.989993+0000","last_peered":"2024-03-12T17:06:29.989993+0000","last_clean":"2024-03-12T17:06:29.989993+0000","last_became_active":"2024-03-12T17:06:25.936366+0000","last_became_peered":"2024-03-12T17:06:25.936366+0000","last_unstale":"2024-03-12T17:06:29.989993+0000","last_undegraded":"2024-03-12T17:06:29.989993+0000","last_fullsized":"2024-03-12T17:06:29.989993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:44:34.582031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.013702+0000","last_change":"2024-03-12T17:06:25.960447+0000","last_active":"2024-03-12T17:06:30.013702+0000","last_peered":"2024-03-12T17:06:30.013702+0000","last_clean":"2024-03-12T17:06:30.013702+0000","last_became_active":"2024-03-12T17:06:25.958764+0000","last_became_peered":"2024-03-12T17:06:25.958764+0000","last_unstale":"2024-03-12T17:06:30.013702+0000","last_undegraded":"2024-03-12T17:06:30.013702+0000","last_fullsized":"2024-03-12T17:06:30.013702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:16:50.369045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.987173+0000","last_change":"2024-03-12T17:06:25.928162+0000","last_active":"2024-03-12T17:06:29.987173+0000","last_peered":"2024-03-12T17:06:29.987173+0000","last_clean":"2024-03-12T17:06:29.987173+0000","last_became_active":"2024-03-12T17:06:25.927864+0000","last_became_peered":"2024-03-12T17:06:25.927864+0000","last_unstale":"2024-03-12T17:06:29.987173+0000","last_undegraded":"2024-03-12T17:06:29.987173+0000","last_fullsized":"2024-03-12T17:06:29.987173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:28:58.739417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.018239+0000","last_change":"2024-03-12T17:06:25.956401+0000","last_active":"2024-03-12T17:06:30.018239+0000","last_peered":"2024-03-12T17:06:30.018239+0000","last_clean":"2024-03-12T17:06:30.018239+0000","last_became_active":"2024-03-12T17:06:25.955835+0000","last_became_peered":"2024-03-12T17:06:25.955835+0000","last_unstale":"2024-03-12T17:06:30.018239+0000","last_undegraded":"2024-03-12T17:06:30.018239+0000","last_fullsized":"2024-03-12T17:06:30.018239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:11:15.145295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.985932+0000","last_change":"2024-03-12T17:06:25.952812+0000","last_active":"2024-03-12T17:06:29.985932+0000","last_peered":"2024-03-12T17:06:29.985932+0000","last_clean":"2024-03-12T17:06:29.985932+0000","last_became_active":"2024-03-12T17:06:25.946931+0000","last_became_peered":"2024-03-12T17:06:25.946931+0000","last_unstale":"2024-03-12T17:06:29.985932+0000","last_undegraded":"2024-03-12T17:06:29.985932+0000","last_fullsized":"2024-03-12T17:06:29.985932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:16:50.369045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012000+0000","last_change":"2024-03-12T17:06:25.950752+0000","last_active":"2024-03-12T17:06:30.012000+0000","last_peered":"2024-03-12T17:06:30.012000+0000","last_clean":"2024-03-12T17:06:30.012000+0000","last_became_active":"2024-03-12T17:06:25.950440+0000","last_became_peered":"2024-03-12T17:06:25.950440+0000","last_unstale":"2024-03-12T17:06:30.012000+0000","last_undegraded":"2024-03-12T17:06:30.012000+0000","last_fullsized":"2024-03-12T17:06:30.012000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:38:51.036295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990839+0000","last_change":"2024-03-12T17:06:25.944733+0000","last_active":"2024-03-12T17:06:29.990839+0000","last_peered":"2024-03-12T17:06:29.990839+0000","last_clean":"2024-03-12T17:06:29.990839+0000","last_became_active":"2024-03-12T17:06:25.942878+0000","last_became_peered":"2024-03-12T17:06:25.942878+0000","last_unstale":"2024-03-12T17:06:29.990839+0000","last_undegraded":"2024-03-12T17:06:29.990839+0000","last_fullsized":"2024-03-12T17:06:29.990839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:15:36.195143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.996580+0000","last_change":"2024-03-12T17:06:25.943005+0000","last_active":"2024-03-12T17:06:29.996580+0000","last_peered":"2024-03-12T17:06:29.996580+0000","last_clean":"2024-03-12T17:06:29.996580+0000","last_became_active":"2024-03-12T17:06:25.933118+0000","last_became_peered":"2024-03-12T17:06:25.933118+0000","last_unstale":"2024-03-12T17:06:29.996580+0000","last_undegraded":"2024-03-12T17:06:29.996580+0000","last_fullsized":"2024-03-12T17:06:29.996580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:44:25.750833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.022750+0000","last_change":"2024-03-12T17:06:25.947390+0000","last_active":"2024-03-12T17:06:30.022750+0000","last_peered":"2024-03-12T17:06:30.022750+0000","last_clean":"2024-03-12T17:06:30.022750+0000","last_became_active":"2024-03-12T17:06:25.947027+0000","last_became_peered":"2024-03-12T17:06:25.947027+0000","last_unstale":"2024-03-12T17:06:30.022750+0000","last_undegraded":"2024-03-12T17:06:30.022750+0000","last_fullsized":"2024-03-12T17:06:30.022750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:44:34.582031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.013617+0000","last_change":"2024-03-12T17:06:25.958459+0000","last_active":"2024-03-12T17:06:30.013617+0000","last_peered":"2024-03-12T17:06:30.013617+0000","last_clean":"2024-03-12T17:06:30.013617+0000","last_became_active":"2024-03-12T17:06:25.956687+0000","last_became_peered":"2024-03-12T17:06:25.956687+0000","last_unstale":"2024-03-12T17:06:30.013617+0000","last_undegraded":"2024-03-12T17:06:30.013617+0000","last_fullsized":"2024-03-12T17:06:30.013617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:26:54.235648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4593,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":864256,"data_stored":695401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4690000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.284}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.371}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.675}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4370000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":270336,"data_stored":104541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.274}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4350000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.409}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5629999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":104577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4419999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.401}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.546}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.663}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":278528,"data_stored":108246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.331}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":104451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":274432,"data_stored":104611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.125}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.268}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3140000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":872448,"data_stored":695457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.286}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.377}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.617}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3440000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-12T17:06:56.080 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-12T17:06:56.080 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-12T17:06:56.080 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-12T17:06:56.080 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-12T17:06:56.081 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-12T17:06:56.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.345+0000 7fee56e00640 1 Processor -- start 2024-03-12T17:06:56.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.345+0000 7fee56e00640 1 -- start start 2024-03-12T17:06:56.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.346+0000 7fee56e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fee500af9a0 0x7fee500cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:56.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.346+0000 7fee56e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fee500b0b40 0x7fee5015c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:56.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.346+0000 7fee56e00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee500cd830 con 0x7fee500cd3d0 2024-03-12T17:06:56.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.346+0000 7fee56e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee500cef00 con 0x7fee500b0b40 2024-03-12T17:06:56.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.346+0000 7fee56e00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee500b1480 con 0x7fee500af9a0 2024-03-12T17:06:56.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.347+0000 7fee55000640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fee500cd3d0 0x7fee500b0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:54880/0 (socket says 172.21.15.52:54880) 2024-03-12T17:06:56.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.347+0000 7fee55000640 1 -- 172.21.15.52:0/584103109 learned_addr learned my addr 172.21.15.52:0/584103109 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:06:56.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.347+0000 7fee55a00640 1 --2- 172.21.15.52:0/584103109 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fee500af9a0 0x7fee500cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:56.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.347+0000 7fee56400640 1 --2- 172.21.15.52:0/584103109 >> v2:172.21.15.102:3300/0 conn(0x7fee500b0b40 0x7fee5015c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:56.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.347+0000 7fee4fe00640 1 -- 172.21.15.52:0/584103109 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1401428429 0 0) 0x7fee500cd830 con 0x7fee500cd3d0 2024-03-12T17:06:56.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.347+0000 7fee4fe00640 1 -- 172.21.15.52:0/584103109 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee38003520 con 0x7fee500cd3d0 2024-03-12T17:06:56.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.348+0000 7fee55a00640 1 -- 172.21.15.52:0/584103109 >> v2:172.21.15.102:3300/0 conn(0x7fee500b0b40 msgr2=0x7fee5015c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:56.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.348+0000 7fee55a00640 1 --2- 172.21.15.52:0/584103109 >> v2:172.21.15.102:3300/0 conn(0x7fee500b0b40 0x7fee5015c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:56.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.348+0000 7fee55a00640 1 -- 172.21.15.52:0/584103109 >> v1:172.21.15.52:6789/0 conn(0x7fee500cd3d0 legacy=0x7fee500b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:56.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.348+0000 7fee4fe00640 1 -- 172.21.15.52:0/584103109 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3107319062 0 0) 0x7fee38003520 con 0x7fee500cd3d0 2024-03-12T17:06:56.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.348+0000 7fee55a00640 1 -- 172.21.15.52:0/584103109 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee500b11a0 con 0x7fee500af9a0 2024-03-12T17:06:56.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.349+0000 7fee55a00640 1 --2- 172.21.15.52:0/584103109 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fee500af9a0 0x7fee500cce90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fee400097a0 tx=0x7fee4002b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ff1d9e45e6b48bf server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:56.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.349+0000 7fee4fe00640 1 -- 172.21.15.52:0/584103109 <== mon.2 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee44001670 con 0x7fee500af9a0 2024-03-12T17:06:56.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.349+0000 7fee4fe00640 1 -- 172.21.15.52:0/584103109 <== mon.2 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4002e050 con 0x7fee500af9a0 2024-03-12T17:06:56.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.350+0000 7fee4fe00640 1 -- 172.21.15.52:0/584103109 <== mon.2 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee40030710 con 0x7fee500af9a0 2024-03-12T17:06:56.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.350+0000 7fee56e00640 1 -- 172.21.15.52:0/584103109 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fee500af9a0 msgr2=0x7fee500cce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:56.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.350+0000 7fee56e00640 1 --2- 172.21.15.52:0/584103109 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fee500af9a0 0x7fee500cce90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fee400097a0 tx=0x7fee4002b790 comp rx=0 tx=0).stop 2024-03-12T17:06:56.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.350+0000 7fee56e00640 1 -- 172.21.15.52:0/584103109 shutdown_connections 2024-03-12T17:06:56.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.350+0000 7fee56e00640 1 --2- 172.21.15.52:0/584103109 >> v2:172.21.15.102:3300/0 conn(0x7fee500b0b40 0x7fee5015c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:56.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.350+0000 7fee56e00640 1 --2- 172.21.15.52:0/584103109 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fee500af9a0 0x7fee500cce90 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:56.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.350+0000 7fee56e00640 1 -- 172.21.15.52:0/584103109 >> 172.21.15.52:0/584103109 conn(0x7fee500d0580 msgr2=0x7fee500d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:56.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.350+0000 7fee56e00640 1 -- 172.21.15.52:0/584103109 shutdown_connections 2024-03-12T17:06:56.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.350+0000 7fee56e00640 1 -- 172.21.15.52:0/584103109 wait complete. 2024-03-12T17:06:56.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.352+0000 7fee56e00640 1 Processor -- start 2024-03-12T17:06:56.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.352+0000 7fee56e00640 1 -- start start 2024-03-12T17:06:56.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.353+0000 7fee56e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fee500af9a0 0x7fee501f6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:56.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.353+0000 7fee56e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee500b0b40 0x7fee501f18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:56.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.353+0000 7fee56e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee500cd3d0 0x7fee500c7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:56.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.353+0000 7fee56e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5014bc60 con 0x7fee500b0b40 2024-03-12T17:06:56.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.353+0000 7fee56e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee50148500 con 0x7fee500cd3d0 2024-03-12T17:06:56.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.353+0000 7fee56e00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee501487e0 con 0x7fee500af9a0 2024-03-12T17:06:56.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.353+0000 7fee55000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee500b0b40 0x7fee501f18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:56.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.353+0000 7fee55000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee500b0b40 0x7fee501f18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60716/0 (socket says 172.21.15.52:60716) 2024-03-12T17:06:56.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.353+0000 7fee55000640 1 -- 172.21.15.52:0/772782669 learned_addr learned my addr 172.21.15.52:0/772782669 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:56.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.354+0000 7fee55a00640 1 --2- 172.21.15.52:0/772782669 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fee500af9a0 0x7fee501f6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:56.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.354+0000 7fee56400640 1 --2- 172.21.15.52:0/772782669 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee500cd3d0 0x7fee500c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:56.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.354+0000 7fee55000640 1 -- 172.21.15.52:0/772782669 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fee500af9a0 msgr2=0x7fee501f6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:56.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.354+0000 7fee55000640 1 --2- 172.21.15.52:0/772782669 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fee500af9a0 0x7fee501f6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:56.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.354+0000 7fee55000640 1 -- 172.21.15.52:0/772782669 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee500cd3d0 msgr2=0x7fee500c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:56.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.354+0000 7fee55000640 1 --2- 172.21.15.52:0/772782669 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee500cd3d0 0x7fee500c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:56.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.354+0000 7fee55000640 1 -- 172.21.15.52:0/772782669 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee40009400 con 0x7fee500b0b40 2024-03-12T17:06:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.354+0000 7fee55a00640 1 --2- 172.21.15.52:0/772782669 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fee500af9a0 0x7fee501f6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.354+0000 7fee56400640 1 --2- 172.21.15.52:0/772782669 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee500cd3d0 0x7fee500c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.354+0000 7fee55000640 1 --2- 172.21.15.52:0/772782669 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee500b0b40 0x7fee501f18c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fee4400c750 tx=0x7fee4400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.354+0000 7fee4d600640 1 -- 172.21.15.52:0/772782669 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee44005ad0 con 0x7fee500b0b40 2024-03-12T17:06:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.354+0000 7fee4d600640 1 -- 172.21.15.52:0/772782669 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee44005c30 con 0x7fee500b0b40 2024-03-12T17:06:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.354+0000 7fee4d600640 1 -- 172.21.15.52:0/772782669 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee44007070 con 0x7fee500b0b40 2024-03-12T17:06:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.354+0000 7fee56e00640 1 -- 172.21.15.52:0/772782669 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee501f5730 con 0x7fee500b0b40 2024-03-12T17:06:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.355+0000 7fee56e00640 1 -- 172.21.15.52:0/772782669 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee501f5cf0 con 0x7fee500b0b40 2024-03-12T17:06:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.356+0000 7fee56e00640 1 -- 172.21.15.52:0/772782669 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee501f5f10 con 0x7fee500b0b40 2024-03-12T17:06:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.357+0000 7fee4d600640 1 -- 172.21.15.52:0/772782669 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fee440071d0 con 0x7fee500b0b40 2024-03-12T17:06:56.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.358+0000 7fee4d600640 1 --2- 172.21.15.52:0/772782669 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fee3406ac30 0x7fee3406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:56.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.358+0000 7fee4d600640 1 -- 172.21.15.52:0/772782669 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fee44088d80 con 0x7fee500b0b40 2024-03-12T17:06:56.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.359+0000 7fee55a00640 1 --2- 172.21.15.52:0/772782669 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fee3406ac30 0x7fee3406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:56.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.360+0000 7fee55a00640 1 --2- 172.21.15.52:0/772782669 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fee3406ac30 0x7fee3406d0f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fee40034000 tx=0x7fee4002f000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:56.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.363+0000 7fee4d600640 1 -- 172.21.15.52:0/772782669 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fee44053c60 con 0x7fee500b0b40 2024-03-12T17:06:56.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.601+0000 7fee56e00640 1 -- 172.21.15.52:0/772782669 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fee5006c330 con 0x7fee500b0b40 2024-03-12T17:06:56.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.611+0000 7fee4d600640 1 -- 172.21.15.52:0/772782669 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+170936 (secure 0 0 0) 0x7fee44017020 con 0x7fee500b0b40 2024-03-12T17:06:56.606 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:06:56.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.616+0000 7fee56e00640 1 -- 172.21.15.52:0/772782669 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fee3406ac30 msgr2=0x7fee3406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:56.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.616+0000 7fee56e00640 1 --2- 172.21.15.52:0/772782669 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fee3406ac30 0x7fee3406d0f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fee40034000 tx=0x7fee4002f000 comp rx=0 tx=0).stop 2024-03-12T17:06:56.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.616+0000 7fee56e00640 1 -- 172.21.15.52:0/772782669 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee500b0b40 msgr2=0x7fee501f18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:56.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.616+0000 7fee56e00640 1 --2- 172.21.15.52:0/772782669 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee500b0b40 0x7fee501f18c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fee4400c750 tx=0x7fee4400cc10 comp rx=0 tx=0).stop 2024-03-12T17:06:56.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.616+0000 7fee56e00640 1 -- 172.21.15.52:0/772782669 shutdown_connections 2024-03-12T17:06:56.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.616+0000 7fee56e00640 1 --2- 172.21.15.52:0/772782669 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fee3406ac30 0x7fee3406d0f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:56.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.616+0000 7fee56e00640 1 --2- 172.21.15.52:0/772782669 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee500cd3d0 0x7fee500c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:56.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.616+0000 7fee56e00640 1 --2- 172.21.15.52:0/772782669 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee500b0b40 0x7fee501f18c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:56.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.616+0000 7fee56e00640 1 --2- 172.21.15.52:0/772782669 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fee500af9a0 0x7fee501f6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:56.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.616+0000 7fee56e00640 1 -- 172.21.15.52:0/772782669 >> 172.21.15.52:0/772782669 conn(0x7fee500d0580 msgr2=0x7fee500cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:56.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.616+0000 7fee56e00640 1 -- 172.21.15.52:0/772782669 shutdown_connections 2024-03-12T17:06:56.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.616+0000 7fee56e00640 1 -- 172.21.15.52:0/772782669 wait complete. 2024-03-12T17:06:56.629 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":5,"active_gid":4109,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":36622},{"type":"v1","addr":"172.21.15.102:6833","nonce":36622}]},"active_addr":"172.21.15.102:6833/36622","active_change":"2024-03-12T17:06:11.122621+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.102:0","nonce":1375791540}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.102:0","nonce":2042311966}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.102:0","nonce":4006268510}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.102:0","nonce":2087051380}]}]} 2024-03-12T17:06:56.634 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-12T17:06:56.634 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-12T17:06:56.634 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:06:56.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.861+0000 7f052ba00640 1 Processor -- start 2024-03-12T17:06:56.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.862+0000 7f052ba00640 1 -- start start 2024-03-12T17:06:56.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.863+0000 7f052ba00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f052415cd70 0x7f05240d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:56.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.863+0000 7f052ba00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05240d17d0 con 0x7f05240b4a50 2024-03-12T17:06:56.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.863+0000 7f052ba00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05240d1ab0 con 0x7f05240d1330 2024-03-12T17:06:56.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.863+0000 7f052ba00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f052415d150 con 0x7f052415cd70 2024-03-12T17:06:56.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.863+0000 7f0529a00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f05240b4a50 0x7f05240b5ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:54890/0 (socket says 172.21.15.52:54890) 2024-03-12T17:06:56.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.863+0000 7f0529a00640 1 -- 172.21.15.52:0/3628173744 learned_addr learned my addr 172.21.15.52:0/3628173744 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:06:56.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.864+0000 7f0529000640 1 --2- 172.21.15.52:0/3628173744 >> v2:172.21.15.102:3300/0 conn(0x7f052415cd70 0x7f05240d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:56.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.864+0000 7f0523400640 1 -- 172.21.15.52:0/3628173744 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3864152447 0 0) 0x7f05240d17d0 con 0x7f05240b4a50 2024-03-12T17:06:56.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.864+0000 7f0523400640 1 -- 172.21.15.52:0/3628173744 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f050c003520 con 0x7f05240b4a50 2024-03-12T17:06:56.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.864+0000 7f0523400640 1 -- 172.21.15.52:0/3628173744 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1774663466 0 0) 0x7f050c003520 con 0x7f05240b4a50 2024-03-12T17:06:56.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.864+0000 7f0523400640 1 -- 172.21.15.52:0/3628173744 >> v2:172.21.15.102:3300/0 conn(0x7f052415cd70 msgr2=0x7f05240d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:56.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.864+0000 7f0523400640 1 --2- 172.21.15.52:0/3628173744 >> v2:172.21.15.102:3300/0 conn(0x7f052415cd70 0x7f05240d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:56.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.864+0000 7f0523400640 1 -- 172.21.15.52:0/3628173744 >> v1:172.21.15.102:6789/0 conn(0x7f05240d1330 legacy=0x7f05240b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:56.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.864+0000 7f0523400640 1 -- 172.21.15.52:0/3628173744 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05240d1d90 con 0x7f05240b4a50 2024-03-12T17:06:56.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.864+0000 7f0523400640 1 -- 172.21.15.52:0/3628173744 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f051c0019f0 con 0x7f05240b4a50 2024-03-12T17:06:56.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.864+0000 7f0523400640 1 -- 172.21.15.52:0/3628173744 >> v1:172.21.15.52:6789/0 conn(0x7f05240b4a50 legacy=0x7f05240b5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:56.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.864+0000 7f0523400640 1 --2- 172.21.15.52:0/3628173744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f050c0039c0 0x7f050c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:56.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.864+0000 7f0523400640 1 --2- 172.21.15.52:0/3628173744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f050c0047c0 0x7f050c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:56.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.864+0000 7f0523400640 1 --2- 172.21.15.52:0/3628173744 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f050c0051d0 0x7f050c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:56.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.865+0000 7f0523400640 1 -- 172.21.15.52:0/3628173744 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050c00c0f0 con 0x7f050c0039c0 2024-03-12T17:06:56.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.865+0000 7f0523400640 1 -- 172.21.15.52:0/3628173744 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050c00c260 con 0x7f050c0047c0 2024-03-12T17:06:56.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.865+0000 7f0523400640 1 -- 172.21.15.52:0/3628173744 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f050c00c420 con 0x7f050c0051d0 2024-03-12T17:06:56.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.865+0000 7f0529000640 1 --2- 172.21.15.52:0/3628173744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f050c0039c0 0x7f050c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:56.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.865+0000 7f0523e00640 1 --2- 172.21.15.52:0/3628173744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f050c0047c0 0x7f050c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:56.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.865+0000 7f0529000640 1 -- 172.21.15.52:0/3628173744 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f050c0051d0 msgr2=0x7f050c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:56.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.865+0000 7f0529a00640 1 --2- 172.21.15.52:0/3628173744 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f050c0051d0 0x7f050c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:56.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.865+0000 7f0529000640 1 --2- 172.21.15.52:0/3628173744 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f050c0051d0 0x7f050c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:56.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.865+0000 7f0529000640 1 -- 172.21.15.52:0/3628173744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f050c0047c0 msgr2=0x7f050c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:56.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.865+0000 7f0529000640 1 --2- 172.21.15.52:0/3628173744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f050c0047c0 0x7f050c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:56.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.865+0000 7f0529000640 1 -- 172.21.15.52:0/3628173744 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f050c003520 con 0x7f050c0039c0 2024-03-12T17:06:56.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.865+0000 7f0529a00640 1 -- 172.21.15.52:0/3628173744 reap_dead start 2024-03-12T17:06:56.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.865+0000 7f0523e00640 1 --2- 172.21.15.52:0/3628173744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f050c0047c0 0x7f050c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:56.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.866+0000 7f0529000640 1 --2- 172.21.15.52:0/3628173744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f050c0039c0 0x7f050c003de0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0518009a90 tx=0x7f050c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dee4c972813b47b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:56.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.867+0000 7f0523400640 1 -- 172.21.15.52:0/3628173744 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051802c6b0 con 0x7f050c0039c0 2024-03-12T17:06:56.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.867+0000 7f0523400640 1 -- 172.21.15.52:0/3628173744 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051802c810 con 0x7f050c0039c0 2024-03-12T17:06:56.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.867+0000 7f0523400640 1 -- 172.21.15.52:0/3628173744 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051802cba0 con 0x7f050c0039c0 2024-03-12T17:06:56.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.867+0000 7f052ba00640 1 -- 172.21.15.52:0/3628173744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f050c0039c0 msgr2=0x7f050c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:56.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.867+0000 7f052ba00640 1 --2- 172.21.15.52:0/3628173744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f050c0039c0 0x7f050c003de0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0518009a90 tx=0x7f050c0057f0 comp rx=0 tx=0).stop 2024-03-12T17:06:56.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.868+0000 7f052ba00640 1 -- 172.21.15.52:0/3628173744 shutdown_connections 2024-03-12T17:06:56.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.868+0000 7f052ba00640 1 --2- 172.21.15.52:0/3628173744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f050c0039c0 0x7f050c003de0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:56.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.868+0000 7f052ba00640 1 -- 172.21.15.52:0/3628173744 >> 172.21.15.52:0/3628173744 conn(0x7f05240cf340 msgr2=0x7f05240cf750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:56.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.868+0000 7f052ba00640 1 -- 172.21.15.52:0/3628173744 shutdown_connections 2024-03-12T17:06:56.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.868+0000 7f052ba00640 1 -- 172.21.15.52:0/3628173744 wait complete. 2024-03-12T17:06:56.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.869+0000 7f052ba00640 1 Processor -- start 2024-03-12T17:06:56.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.870+0000 7f052ba00640 1 -- start start 2024-03-12T17:06:56.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.871+0000 7f052ba00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f050c0039c0 0x7f05240cc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:56.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.871+0000 7f052ba00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05240c9ff0 0x7f0524158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:56.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.871+0000 7f052ba00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05241f3680 0x7f05240c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:56.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.871+0000 7f052ba00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0524159370 con 0x7f050c0039c0 2024-03-12T17:06:56.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.871+0000 7f052ba00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0524159060 con 0x7f05241f3680 2024-03-12T17:06:56.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.871+0000 7f052ba00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05240b2ea0 con 0x7f05240c9ff0 2024-03-12T17:06:56.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.871+0000 7f0529000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f050c0039c0 0x7f05240cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:56.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.871+0000 7f0529000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f050c0039c0 0x7f05240cc470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60742/0 (socket says 172.21.15.52:60742) 2024-03-12T17:06:56.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.871+0000 7f0529000640 1 -- 172.21.15.52:0/3227714009 learned_addr learned my addr 172.21.15.52:0/3227714009 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:56.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.871+0000 7f0523e00640 1 --2- 172.21.15.52:0/3227714009 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05240c9ff0 0x7f0524158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:56.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.871+0000 7f0529000640 1 -- 172.21.15.52:0/3227714009 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05240c9ff0 msgr2=0x7f0524158b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:56.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.871+0000 7f0529000640 1 --2- 172.21.15.52:0/3227714009 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05240c9ff0 0x7f0524158b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:56.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.871+0000 7f0529000640 1 -- 172.21.15.52:0/3227714009 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05241f3680 msgr2=0x7f05240c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:56.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.871+0000 7f0529000640 1 --2- 172.21.15.52:0/3227714009 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05241f3680 0x7f05240c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:56.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.871+0000 7f0529000640 1 -- 172.21.15.52:0/3227714009 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05240b3180 con 0x7f050c0039c0 2024-03-12T17:06:56.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.872+0000 7f0529000640 1 --2- 172.21.15.52:0/3227714009 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f050c0039c0 0x7f05240cc470 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0518009a90 tx=0x7f0518004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:56.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.872+0000 7f0520c00640 1 -- 172.21.15.52:0/3227714009 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0518003810 con 0x7f050c0039c0 2024-03-12T17:06:56.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.872+0000 7f0520c00640 1 -- 172.21.15.52:0/3227714009 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0518003970 con 0x7f050c0039c0 2024-03-12T17:06:56.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.872+0000 7f0520c00640 1 -- 172.21.15.52:0/3227714009 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0518003d00 con 0x7f050c0039c0 2024-03-12T17:06:56.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.872+0000 7f052ba00640 1 -- 172.21.15.52:0/3227714009 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05240b3460 con 0x7f050c0039c0 2024-03-12T17:06:56.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.872+0000 7f052ba00640 1 -- 172.21.15.52:0/3227714009 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05240b3a20 con 0x7f050c0039c0 2024-03-12T17:06:56.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.874+0000 7f052ba00640 1 -- 172.21.15.52:0/3227714009 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05240d0c60 con 0x7f050c0039c0 2024-03-12T17:06:56.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.874+0000 7f0520c00640 1 -- 172.21.15.52:0/3227714009 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0518040450 con 0x7f050c0039c0 2024-03-12T17:06:56.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.875+0000 7f0520c00640 1 --2- 172.21.15.52:0/3227714009 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f050406ac30 0x7f050406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:56.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.875+0000 7f0520c00640 1 -- 172.21.15.52:0/3227714009 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f05180ab5c0 con 0x7f050c0039c0 2024-03-12T17:06:56.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.876+0000 7f0523e00640 1 --2- 172.21.15.52:0/3227714009 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f050406ac30 0x7f050406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:56.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.877+0000 7f0523e00640 1 --2- 172.21.15.52:0/3227714009 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f050406ac30 0x7f050406d0f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f052415c6c0 tx=0x7f0510009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:56.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:56.880+0000 7f0520c00640 1 -- 172.21.15.52:0/3227714009 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f0518076410 con 0x7f050c0039c0 2024-03-12T17:06:57.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.079+0000 7f052ba00640 1 -- 172.21.15.52:0/3227714009 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0524073390 con 0x7f050c0039c0 2024-03-12T17:06:57.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.081+0000 7f0520c00640 1 -- 172.21.15.52:0/3227714009 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7f05180044e0 con 0x7f050c0039c0 2024-03-12T17:06:57.076 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:06:57.076 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":25,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:06:30.468476+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T17:06:19.218543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T17:06:23.525474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T17:06:24.911574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-12T17:06:28.017172+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.733031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.775178+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.591522+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.820838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.829252+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.582481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:06:57.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.084+0000 7f052ba00640 1 -- 172.21.15.52:0/3227714009 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f050406ac30 msgr2=0x7f050406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.084+0000 7f052ba00640 1 --2- 172.21.15.52:0/3227714009 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f050406ac30 0x7f050406d0f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f052415c6c0 tx=0x7f0510009040 comp rx=0 tx=0).stop 2024-03-12T17:06:57.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.084+0000 7f052ba00640 1 -- 172.21.15.52:0/3227714009 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f050c0039c0 msgr2=0x7f05240cc470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.084+0000 7f052ba00640 1 --2- 172.21.15.52:0/3227714009 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f050c0039c0 0x7f05240cc470 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0518009a90 tx=0x7f0518004730 comp rx=0 tx=0).stop 2024-03-12T17:06:57.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.084+0000 7f052ba00640 1 -- 172.21.15.52:0/3227714009 shutdown_connections 2024-03-12T17:06:57.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.084+0000 7f052ba00640 1 --2- 172.21.15.52:0/3227714009 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f050406ac30 0x7f050406d0f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.084+0000 7f052ba00640 1 --2- 172.21.15.52:0/3227714009 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05240c9ff0 0x7f0524158b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.084+0000 7f052ba00640 1 --2- 172.21.15.52:0/3227714009 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05241f3680 0x7f05240c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.084+0000 7f052ba00640 1 --2- 172.21.15.52:0/3227714009 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f050c0039c0 0x7f05240cc470 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.084+0000 7f052ba00640 1 -- 172.21.15.52:0/3227714009 >> 172.21.15.52:0/3227714009 conn(0x7f05240cf340 msgr2=0x7f05240b3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:57.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.084+0000 7f052ba00640 1 -- 172.21.15.52:0/3227714009 shutdown_connections 2024-03-12T17:06:57.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.084+0000 7f052ba00640 1 -- 172.21.15.52:0/3227714009 wait complete. 2024-03-12T17:06:57.100 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-12T17:06:57.100 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:06:57.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.331+0000 7f791dc00640 1 Processor -- start 2024-03-12T17:06:57.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.331+0000 7f791dc00640 1 -- start start 2024-03-12T17:06:57.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.332+0000 7f791dc00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f79180b4aa0 0x7f79180d2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.332+0000 7f791dc00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f791815d090 con 0x7f79180d0af0 2024-03-12T17:06:57.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.332+0000 7f791dc00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79180d0f30 con 0x7f791815ccb0 2024-03-12T17:06:57.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.332+0000 7f791dc00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79180d27d0 con 0x7f79180b4aa0 2024-03-12T17:06:57.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.333+0000 7f7917400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f79180b4aa0 0x7f79180d2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.333+0000 7f7916000640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f79180d0af0 0x7f79180b4390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:54900/0 (socket says 172.21.15.52:54900) 2024-03-12T17:06:57.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.333+0000 7f7916000640 1 -- 172.21.15.52:0/2958681262 learned_addr learned my addr 172.21.15.52:0/2958681262 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:06:57.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.333+0000 7f7917400640 1 -- 172.21.15.52:0/2958681262 >> v1:172.21.15.102:6789/0 conn(0x7f791815ccb0 legacy=0x7f79180cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:57.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.333+0000 7f7917400640 1 -- 172.21.15.52:0/2958681262 >> v1:172.21.15.52:6789/0 conn(0x7f79180d0af0 legacy=0x7f79180b4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.333+0000 7f7917400640 1 -- 172.21.15.52:0/2958681262 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79180d11f0 con 0x7f79180b4aa0 2024-03-12T17:06:57.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.333+0000 7f7915600640 1 -- 172.21.15.52:0/2958681262 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2119895788 0 0) 0x7f791815d090 con 0x7f79180d0af0 2024-03-12T17:06:57.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.334+0000 7f7917400640 1 --2- 172.21.15.52:0/2958681262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79180b4aa0 0x7f79180d2290 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f79100097a0 tx=0x7f791002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d006eb70ac7e05e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.335+0000 7f7915600640 1 -- 172.21.15.52:0/2958681262 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7910030610 con 0x7f79180b4aa0 2024-03-12T17:06:57.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.335+0000 7f7915600640 1 -- 172.21.15.52:0/2958681262 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7910030770 con 0x7f79180b4aa0 2024-03-12T17:06:57.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.335+0000 7f7915600640 1 -- 172.21.15.52:0/2958681262 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7910030b00 con 0x7f79180b4aa0 2024-03-12T17:06:57.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.335+0000 7f791dc00640 1 -- 172.21.15.52:0/2958681262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79180b4aa0 msgr2=0x7f79180d2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.335+0000 7f791dc00640 1 --2- 172.21.15.52:0/2958681262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79180b4aa0 0x7f79180d2290 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f79100097a0 tx=0x7f791002b960 comp rx=0 tx=0).stop 2024-03-12T17:06:57.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.336+0000 7f791dc00640 1 -- 172.21.15.52:0/2958681262 shutdown_connections 2024-03-12T17:06:57.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.336+0000 7f791dc00640 1 --2- 172.21.15.52:0/2958681262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79180b4aa0 0x7f79180d2290 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.336+0000 7f791dc00640 1 -- 172.21.15.52:0/2958681262 >> 172.21.15.52:0/2958681262 conn(0x7f79180717b0 msgr2=0x7f7918071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:57.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.336+0000 7f791dc00640 1 -- 172.21.15.52:0/2958681262 shutdown_connections 2024-03-12T17:06:57.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.336+0000 7f791dc00640 1 -- 172.21.15.52:0/2958681262 wait complete. 2024-03-12T17:06:57.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.337+0000 7f791dc00640 1 Processor -- start 2024-03-12T17:06:57.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.338+0000 7f791dc00640 1 -- start start 2024-03-12T17:06:57.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.338+0000 7f791dc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79180b4aa0 0x7f7918156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.338+0000 7f791dc00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79180d0af0 0x7f79181536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.338+0000 7f791dc00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f791815ccb0 0x7f7918158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.338+0000 7f791dc00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7918159970 con 0x7f79180d0af0 2024-03-12T17:06:57.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.338+0000 7f791dc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7918156ce0 con 0x7f79180b4aa0 2024-03-12T17:06:57.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.338+0000 7f791dc00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7918156ad0 con 0x7f791815ccb0 2024-03-12T17:06:57.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.339+0000 7f7916000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79180d0af0 0x7f79181536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.339+0000 7f7916000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79180d0af0 0x7f79181536b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60766/0 (socket says 172.21.15.52:60766) 2024-03-12T17:06:57.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.339+0000 7f7916000640 1 -- 172.21.15.52:0/4037589496 learned_addr learned my addr 172.21.15.52:0/4037589496 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:57.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.339+0000 7f7916a00640 1 --2- 172.21.15.52:0/4037589496 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79180b4aa0 0x7f7918156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.339+0000 7f7917400640 1 --2- 172.21.15.52:0/4037589496 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f791815ccb0 0x7f7918158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.339+0000 7f7916000640 1 -- 172.21.15.52:0/4037589496 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f791815ccb0 msgr2=0x7f7918158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.339+0000 7f7916000640 1 --2- 172.21.15.52:0/4037589496 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f791815ccb0 0x7f7918158a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.339+0000 7f7916000640 1 -- 172.21.15.52:0/4037589496 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79180b4aa0 msgr2=0x7f7918156590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.339+0000 7f7916000640 1 --2- 172.21.15.52:0/4037589496 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79180b4aa0 0x7f7918156590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.339+0000 7f7916000640 1 -- 172.21.15.52:0/4037589496 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7910009400 con 0x7f79180d0af0 2024-03-12T17:06:57.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.339+0000 7f7917400640 1 --2- 172.21.15.52:0/4037589496 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f791815ccb0 0x7f7918158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:57.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.340+0000 7f7916000640 1 --2- 172.21.15.52:0/4037589496 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79180d0af0 0x7f79181536b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f790800c890 tx=0x7f790800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.340+0000 7f790ea00640 1 -- 172.21.15.52:0/4037589496 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790800a640 con 0x7f79180d0af0 2024-03-12T17:06:57.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.340+0000 7f790ea00640 1 -- 172.21.15.52:0/4037589496 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790800a7c0 con 0x7f79180d0af0 2024-03-12T17:06:57.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.340+0000 7f790ea00640 1 -- 172.21.15.52:0/4037589496 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790800ab70 con 0x7f79180d0af0 2024-03-12T17:06:57.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.340+0000 7f791dc00640 1 -- 172.21.15.52:0/4037589496 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79180b3cc0 con 0x7f79180d0af0 2024-03-12T17:06:57.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.340+0000 7f791dc00640 1 -- 172.21.15.52:0/4037589496 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79180b2f00 con 0x7f79180d0af0 2024-03-12T17:06:57.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.341+0000 7f791dc00640 1 -- 172.21.15.52:0/4037589496 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79180b8920 con 0x7f79180d0af0 2024-03-12T17:06:57.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.342+0000 7f790ea00640 1 -- 172.21.15.52:0/4037589496 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f790800acd0 con 0x7f79180d0af0 2024-03-12T17:06:57.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.343+0000 7f790ea00640 1 --2- 172.21.15.52:0/4037589496 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f78f406ac00 0x7f78f406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.343+0000 7f790ea00640 1 -- 172.21.15.52:0/4037589496 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7908088c10 con 0x7f79180d0af0 2024-03-12T17:06:57.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.344+0000 7f7916a00640 1 --2- 172.21.15.52:0/4037589496 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f78f406ac00 0x7f78f406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.345+0000 7f7916a00640 1 --2- 172.21.15.52:0/4037589496 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f78f406ac00 0x7f78f406d0c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7900004ea0 tx=0x7f7900009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.348+0000 7f790ea00640 1 -- 172.21.15.52:0/4037589496 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f7908053a60 con 0x7f79180d0af0 2024-03-12T17:06:57.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.548+0000 7f791dc00640 1 -- 172.21.15.52:0/4037589496 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7918066bb0 con 0x7f79180d0af0 2024-03-12T17:06:57.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.551+0000 7f790ea00640 1 -- 172.21.15.52:0/4037589496 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7f7908018020 con 0x7f79180d0af0 2024-03-12T17:06:57.546 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:06:57.546 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":25,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:06:30.468476+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T17:06:19.218543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T17:06:23.525474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T17:06:24.911574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-12T17:06:28.017172+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.733031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.775178+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.591522+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.820838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.829252+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.582481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:06:57.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.554+0000 7f791dc00640 1 -- 172.21.15.52:0/4037589496 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f78f406ac00 msgr2=0x7f78f406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.554+0000 7f791dc00640 1 --2- 172.21.15.52:0/4037589496 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f78f406ac00 0x7f78f406d0c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7900004ea0 tx=0x7f7900009040 comp rx=0 tx=0).stop 2024-03-12T17:06:57.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.554+0000 7f791dc00640 1 -- 172.21.15.52:0/4037589496 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79180d0af0 msgr2=0x7f79181536b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.554+0000 7f791dc00640 1 --2- 172.21.15.52:0/4037589496 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79180d0af0 0x7f79181536b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f790800c890 tx=0x7f790800cd50 comp rx=0 tx=0).stop 2024-03-12T17:06:57.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.554+0000 7f791dc00640 1 -- 172.21.15.52:0/4037589496 shutdown_connections 2024-03-12T17:06:57.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.554+0000 7f791dc00640 1 --2- 172.21.15.52:0/4037589496 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f78f406ac00 0x7f78f406d0c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.554+0000 7f791dc00640 1 --2- 172.21.15.52:0/4037589496 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f791815ccb0 0x7f7918158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.554+0000 7f791dc00640 1 --2- 172.21.15.52:0/4037589496 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79180d0af0 0x7f79181536b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.554+0000 7f791dc00640 1 --2- 172.21.15.52:0/4037589496 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79180b4aa0 0x7f7918156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.554+0000 7f791dc00640 1 -- 172.21.15.52:0/4037589496 >> 172.21.15.52:0/4037589496 conn(0x7f79180717b0 msgr2=0x7f7918071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:57.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.554+0000 7f791dc00640 1 -- 172.21.15.52:0/4037589496 shutdown_connections 2024-03-12T17:06:57.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.555+0000 7f791dc00640 1 -- 172.21.15.52:0/4037589496 wait complete. 2024-03-12T17:06:57.566 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-12T17:06:57.567 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-12T17:06:57.567 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-12T17:06:57.568 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-12T17:06:57.568 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-12T17:06:57.569 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-12T17:06:57.569 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-12T17:06:57.569 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-12T17:06:57.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.874+0000 7f52c7e00640 1 Processor -- start 2024-03-12T17:06:57.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.874+0000 7f52c7e00640 1 -- start start 2024-03-12T17:06:57.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.876+0000 7f52c7e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f52c0144070 0x7f52c00700d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.876+0000 7f52c7e00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c00b2370 con 0x7f52c015be50 2024-03-12T17:06:57.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.876+0000 7f52c7e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c00cbfb0 con 0x7f52c00caf30 2024-03-12T17:06:57.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.876+0000 7f52c7e00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c00cb430 con 0x7f52c0144070 2024-03-12T17:06:57.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.877+0000 7f52c5400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f52c0144070 0x7f52c00700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.877+0000 7f52c5400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f52c0144070 0x7f52c00700d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60782/0 (socket says 172.21.15.52:60782) 2024-03-12T17:06:57.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.877+0000 7f52c5400640 1 -- 172.21.15.52:0/1453712823 learned_addr learned my addr 172.21.15.52:0/1453712823 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:57.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.879+0000 7f18f1800640 1 Processor -- start 2024-03-12T17:06:57.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.879+0000 7f52c5400640 1 -- 172.21.15.52:0/1453712823 >> v1:172.21.15.102:6789/0 conn(0x7f52c00caf30 legacy=0x7f52c00b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:57.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.879+0000 7f52c5400640 1 -- 172.21.15.52:0/1453712823 >> v1:172.21.15.52:6789/0 conn(0x7f52c015be50 legacy=0x7f52c00b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.879+0000 7f18f1800640 1 -- start start 2024-03-12T17:06:57.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.879+0000 7f52c5400640 1 -- 172.21.15.52:0/1453712823 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c00cb710 con 0x7f52c0144070 2024-03-12T17:06:57.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.881+0000 7fb614a00640 1 Processor -- start 2024-03-12T17:06:57.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.881+0000 7fb614a00640 1 -- start start 2024-03-12T17:06:57.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.882+0000 7f52c5400640 1 --2- 172.21.15.52:0/1453712823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52c0144070 0x7f52c00700d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f52b4013150 tx=0x7f52b4034120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9fd6133abfea1e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.883+0000 7f52bfe00640 1 -- 172.21.15.52:0/1453712823 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b40110d0 con 0x7f52c0144070 2024-03-12T17:06:57.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.883+0000 7f7de8e00640 1 Processor -- start 2024-03-12T17:06:57.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.883+0000 7fb614a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb6100718b0 0x7fb61006c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.883+0000 7fb614a00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb6100b1a90 0x7fb61006cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.883+0000 7fb614a00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6100716a0 con 0x7fb61006aca0 2024-03-12T17:06:57.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.883+0000 7fb614a00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb61015dc60 con 0x7fb6100b1a90 2024-03-12T17:06:57.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.883+0000 7fb614a00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb61015cd20 con 0x7fb6100718b0 2024-03-12T17:06:57.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.881+0000 7f18f1800640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f18ec0d0640 0x7f18ec0d0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.881+0000 7f18f1800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f18ec0d11b0 0x7f18ec0b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.882+0000 7f18f1800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18ec0d1600 con 0x7f18ec15c830 2024-03-12T17:06:57.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.882+0000 7f18f1800640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18ec0d0a80 con 0x7f18ec0d0640 2024-03-12T17:06:57.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.882+0000 7f18f1800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18ec0cda60 con 0x7f18ec0d11b0 2024-03-12T17:06:57.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.882+0000 7f18ebe00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f18ec15c830 0x7f18ec0cd1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.52:35732/0 (socket says 172.21.15.52:35732) 2024-03-12T17:06:57.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.882+0000 7f18ebe00640 1 -- 172.21.15.52:0/728120727 learned_addr learned my addr 172.21.15.52:0/728120727 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:06:57.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.883+0000 7f18f0e00640 1 --2- 172.21.15.52:0/728120727 >> v2:172.21.15.102:3300/0 conn(0x7f18ec0d11b0 0x7f18ec0b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.883+0000 7f18eb400640 1 --2- 172.21.15.52:0/728120727 >> v2:172.21.15.52:3300/0 conn(0x7f18ec0d0640 0x7f18ec0d0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.883+0000 7f52bfe00640 1 -- 172.21.15.52:0/1453712823 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b4011230 con 0x7f52c0144070 2024-03-12T17:06:57.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.883+0000 7f52bfe00640 1 -- 172.21.15.52:0/1453712823 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b40115c0 con 0x7f52c0144070 2024-03-12T17:06:57.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.884+0000 7f18eaa00640 1 -- 172.21.15.52:0/728120727 <== mon.1 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2703981581 0 0) 0x7f18d8001110 con 0x7f18ec15c830 2024-03-12T17:06:57.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.884+0000 7f18eaa00640 1 -- 172.21.15.52:0/728120727 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18d0003520 con 0x7f18ec15c830 2024-03-12T17:06:57.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.884+0000 7f18eb400640 1 -- 172.21.15.52:0/728120727 >> v2:172.21.15.102:3300/0 conn(0x7f18ec0d11b0 msgr2=0x7f18ec0b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:57.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.884+0000 7f18eb400640 1 --2- 172.21.15.52:0/728120727 >> v2:172.21.15.102:3300/0 conn(0x7f18ec0d11b0 0x7f18ec0b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.884+0000 7fb60ea00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb6100718b0 0x7fb61006c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.884+0000 7fb60ea00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb6100718b0 0x7fb61006c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.52:55636/0 (socket says 172.21.15.52:55636) 2024-03-12T17:06:57.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.884+0000 7fb60ea00640 1 -- 172.21.15.52:0/3882746896 learned_addr learned my addr 172.21.15.52:0/3882746896 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:57.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.884+0000 7f18eaa00640 1 -- 172.21.15.52:0/728120727 <== mon.1 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 489332900 0 0) 0x7f18d0003520 con 0x7f18ec15c830 2024-03-12T17:06:57.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.883+0000 7f7de8e00640 1 -- start start 2024-03-12T17:06:57.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.885+0000 7f52c7e00640 1 -- 172.21.15.52:0/1453712823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52c0144070 msgr2=0x7f52c00700d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.885+0000 7f52c7e00640 1 --2- 172.21.15.52:0/1453712823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52c0144070 0x7f52c00700d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f52b4013150 tx=0x7f52b4034120 comp rx=0 tx=0).stop 2024-03-12T17:06:57.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.885+0000 7fb60fe00640 1 --2- 172.21.15.52:0/3882746896 >> v2:172.21.15.52:3300/0 conn(0x7fb6100b1a90 0x7fb61006cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.885+0000 7f52c7e00640 1 -- 172.21.15.52:0/1453712823 shutdown_connections 2024-03-12T17:06:57.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.885+0000 7f52c7e00640 1 --2- 172.21.15.52:0/1453712823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52c0144070 0x7f52c00700d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.885+0000 7f52c7e00640 1 -- 172.21.15.52:0/1453712823 >> 172.21.15.52:0/1453712823 conn(0x7f52c0069e70 msgr2=0x7f52c006a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:57.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.885+0000 7f18eb400640 1 -- 172.21.15.52:0/728120727 >> v1:172.21.15.102:6789/0 conn(0x7f18ec15c830 legacy=0x7f18ec0cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.885+0000 7f52c7e00640 1 -- 172.21.15.52:0/1453712823 shutdown_connections 2024-03-12T17:06:57.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.885+0000 7f7de8e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7ddc0ebb50 0x7f7ddc055300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.885+0000 7f18eb400640 1 -- 172.21.15.52:0/728120727 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ec0cdd40 con 0x7f18ec0d0640 2024-03-12T17:06:57.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.885+0000 7f18f0e00640 1 --2- 172.21.15.52:0/728120727 >> v2:172.21.15.102:3300/0 conn(0x7f18ec0d11b0 0x7f18ec0b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T17:06:57.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.886+0000 7f7de8e00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ddc057e30 con 0x7f7ddc0061a0 2024-03-12T17:06:57.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.886+0000 7f7de8e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ddc0579a0 con 0x7f7ddc0eb060 2024-03-12T17:06:57.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.886+0000 7f7de8e00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ddc0ec000 con 0x7f7ddc0ebb50 2024-03-12T17:06:57.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.886+0000 7fb60fe00640 1 -- 172.21.15.52:0/3882746896 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb6100718b0 msgr2=0x7fb61006c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:57.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.886+0000 7fb60fe00640 1 --2- 172.21.15.52:0/3882746896 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb6100718b0 0x7fb61006c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.886+0000 7fb60fe00640 1 -- 172.21.15.52:0/3882746896 >> v1:172.21.15.52:6789/0 conn(0x7fb61006aca0 legacy=0x7fb61006b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.886+0000 7fb60fe00640 1 -- 172.21.15.52:0/3882746896 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb61015cfe0 con 0x7fb6100b1a90 2024-03-12T17:06:57.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.886+0000 7f52c7e00640 1 -- 172.21.15.52:0/1453712823 wait complete. 2024-03-12T17:06:57.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.887+0000 7fb60ea00640 1 --2- 172.21.15.52:0/3882746896 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb6100718b0 0x7fb61006c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T17:06:57.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.886+0000 7f7de2800640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f7ddc0061a0 0x7f7ddc006610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:54924/0 (socket says 172.21.15.52:54924) 2024-03-12T17:06:57.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.886+0000 7f7de2800640 1 -- 172.21.15.52:0/2282806269 learned_addr learned my addr 172.21.15.52:0/2282806269 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:06:57.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.886+0000 7f7de1400640 1 --2- 172.21.15.52:0/2282806269 >> v2:172.21.15.52:3300/0 conn(0x7f7ddc0ebb50 0x7f7ddc055300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.887+0000 7f7de1400640 1 -- 172.21.15.52:0/2282806269 >> v1:172.21.15.102:6789/0 conn(0x7f7ddc0eb060 legacy=0x7f7ddc0eb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:57.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.887+0000 7f7de1400640 1 -- 172.21.15.52:0/2282806269 >> v1:172.21.15.52:6789/0 conn(0x7f7ddc0061a0 legacy=0x7f7ddc006610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.887+0000 7f7de1400640 1 -- 172.21.15.52:0/2282806269 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ddc0387e0 con 0x7f7ddc0ebb50 2024-03-12T17:06:57.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.887+0000 7f18eb400640 1 --2- 172.21.15.52:0/728120727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18ec0d0640 0x7f18ec0d0c70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f18ec08c5a0 tx=0x7f18dc02b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40d66df27874376 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.888+0000 7fb60fe00640 1 --2- 172.21.15.52:0/3882746896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6100b1a90 0x7fb61006cf20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb608009dc0 tx=0x7fb60802be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9300f6d8fc9ac80f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.888+0000 7f18eaa00640 1 -- 172.21.15.52:0/728120727 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d8001800 con 0x7f18ec0d0640 2024-03-12T17:06:57.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.889+0000 7f18eaa00640 1 -- 172.21.15.52:0/728120727 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18dc02e050 con 0x7f18ec0d0640 2024-03-12T17:06:57.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.889+0000 7f18eaa00640 1 -- 172.21.15.52:0/728120727 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc030730 con 0x7f18ec0d0640 2024-03-12T17:06:57.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.889+0000 7f7de1400640 1 --2- 172.21.15.52:0/2282806269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ddc0ebb50 0x7f7ddc055300 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00097a0 tx=0x7f7dd002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8420945edd9409c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.889+0000 7f52c7e00640 1 Processor -- start 2024-03-12T17:06:57.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.889+0000 7fb60e000640 1 -- 172.21.15.52:0/3882746896 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60802cc30 con 0x7fb6100b1a90 2024-03-12T17:06:57.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.889+0000 7fb60e000640 1 -- 172.21.15.52:0/3882746896 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60802cd90 con 0x7fb6100b1a90 2024-03-12T17:06:57.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.889+0000 7fb60e000640 1 -- 172.21.15.52:0/3882746896 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60802cc30 con 0x7fb6100b1a90 2024-03-12T17:06:57.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.889+0000 7f7de0a00640 1 -- 172.21.15.52:0/2282806269 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd002e070 con 0x7f7ddc0ebb50 2024-03-12T17:06:57.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.890+0000 7f18f1800640 1 -- 172.21.15.52:0/728120727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18ec0d0640 msgr2=0x7f18ec0d0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.890+0000 7f18f1800640 1 --2- 172.21.15.52:0/728120727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18ec0d0640 0x7f18ec0d0c70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f18ec08c5a0 tx=0x7f18dc02b710 comp rx=0 tx=0).stop 2024-03-12T17:06:57.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.890+0000 7f7de0a00640 1 -- 172.21.15.52:0/2282806269 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd00315a0 con 0x7f7ddc0ebb50 2024-03-12T17:06:57.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.890+0000 7f18f1800640 1 -- 172.21.15.52:0/728120727 shutdown_connections 2024-03-12T17:06:57.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.890+0000 7f18f1800640 1 --2- 172.21.15.52:0/728120727 >> v2:172.21.15.102:3300/0 conn(0x7f18ec0d11b0 0x7f18ec0b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.890+0000 7f18f1800640 1 --2- 172.21.15.52:0/728120727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18ec0d0640 0x7f18ec0d0c70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.890+0000 7f18f1800640 1 -- 172.21.15.52:0/728120727 >> 172.21.15.52:0/728120727 conn(0x7f18ec089fd0 msgr2=0x7f18ec0d26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:57.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.890+0000 7f7de0a00640 1 -- 172.21.15.52:0/2282806269 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd002e070 con 0x7f7ddc0ebb50 2024-03-12T17:06:57.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.891+0000 7fb614a00640 1 -- 172.21.15.52:0/3882746896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6100b1a90 msgr2=0x7fb61006cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.891+0000 7f7de8e00640 1 -- 172.21.15.52:0/2282806269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ddc0ebb50 msgr2=0x7f7ddc055300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.891+0000 7f7de8e00640 1 --2- 172.21.15.52:0/2282806269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ddc0ebb50 0x7f7ddc055300 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00097a0 tx=0x7f7dd002b840 comp rx=0 tx=0).stop 2024-03-12T17:06:57.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.889+0000 7f52c7e00640 1 -- start start 2024-03-12T17:06:57.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.891+0000 7fb614a00640 1 --2- 172.21.15.52:0/3882746896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6100b1a90 0x7fb61006cf20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb608009dc0 tx=0x7fb60802be80 comp rx=0 tx=0).stop 2024-03-12T17:06:57.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.891+0000 7f52c7e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52c00caf30 0x7f52c0159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.891+0000 7f52c7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52c015be50 0x7f52c013b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.891+0000 7f52c7e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f52c0142460 0x7f52c0136dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.891+0000 7f52c7e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c013bb50 con 0x7f52c00caf30 2024-03-12T17:06:57.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.891+0000 7f52c7e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c013bdd0 con 0x7f52c015be50 2024-03-12T17:06:57.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.891+0000 7f52c7e00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c013c0b0 con 0x7f52c0142460 2024-03-12T17:06:57.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.891+0000 7f18f1800640 1 -- 172.21.15.52:0/728120727 shutdown_connections 2024-03-12T17:06:57.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.891+0000 7f18f1800640 1 -- 172.21.15.52:0/728120727 wait complete. 2024-03-12T17:06:57.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.891+0000 7f52c5400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52c00caf30 0x7f52c0159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.891+0000 7fb614a00640 1 -- 172.21.15.52:0/3882746896 shutdown_connections 2024-03-12T17:06:57.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.891+0000 7fb614a00640 1 --2- 172.21.15.52:0/3882746896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6100b1a90 0x7fb61006cf20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.891+0000 7fb614a00640 1 --2- 172.21.15.52:0/3882746896 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb6100718b0 0x7fb61006c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.891+0000 7fb614a00640 1 -- 172.21.15.52:0/3882746896 >> 172.21.15.52:0/3882746896 conn(0x7fb610070880 msgr2=0x7fb610070c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:57.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.892+0000 7f7de8e00640 1 -- 172.21.15.52:0/2282806269 shutdown_connections 2024-03-12T17:06:57.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.892+0000 7f7de8e00640 1 --2- 172.21.15.52:0/2282806269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ddc0ebb50 0x7f7ddc055300 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.892+0000 7f7de8e00640 1 -- 172.21.15.52:0/2282806269 >> 172.21.15.52:0/2282806269 conn(0x7f7ddc0559e0 msgr2=0x7f7ddc055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:57.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.892+0000 7fb614a00640 1 -- 172.21.15.52:0/3882746896 shutdown_connections 2024-03-12T17:06:57.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.891+0000 7f52c5400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52c00caf30 0x7f52c0159cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60830/0 (socket says 172.21.15.52:60830) 2024-03-12T17:06:57.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.892+0000 7fb614a00640 1 -- 172.21.15.52:0/3882746896 wait complete. 2024-03-12T17:06:57.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.892+0000 7f7de8e00640 1 -- 172.21.15.52:0/2282806269 shutdown_connections 2024-03-12T17:06:57.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.891+0000 7f52c5400640 1 -- 172.21.15.52:0/3056925370 learned_addr learned my addr 172.21.15.52:0/3056925370 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:57.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.892+0000 7f52c5e00640 1 --2- 172.21.15.52:0/3056925370 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f52c0142460 0x7f52c0136dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.892+0000 7f52c4a00640 1 --2- 172.21.15.52:0/3056925370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52c015be50 0x7f52c013b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.892+0000 7f52c5400640 1 -- 172.21.15.52:0/3056925370 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f52c0142460 msgr2=0x7f52c0136dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.892+0000 7f52c5400640 1 --2- 172.21.15.52:0/3056925370 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f52c0142460 0x7f52c0136dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.892+0000 7f52c5400640 1 -- 172.21.15.52:0/3056925370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52c015be50 msgr2=0x7f52c013b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.892+0000 7f52c5400640 1 --2- 172.21.15.52:0/3056925370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52c015be50 0x7f52c013b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.892+0000 7f52c5400640 1 -- 172.21.15.52:0/3056925370 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b4011d70 con 0x7f52c00caf30 2024-03-12T17:06:57.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.893+0000 7f52c5400640 1 --2- 172.21.15.52:0/3056925370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52c00caf30 0x7f52c0159cb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f52b403e040 tx=0x7f52b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.893+0000 7f284f200640 1 Processor -- start 2024-03-12T17:06:57.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.893+0000 7f8a7e200640 1 Processor -- start 2024-03-12T17:06:57.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.893+0000 7f52bd600640 1 -- 172.21.15.52:0/3056925370 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b40110d0 con 0x7f52c00caf30 2024-03-12T17:06:57.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.893+0000 7f52bd600640 1 -- 172.21.15.52:0/3056925370 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b4046660 con 0x7f52c00caf30 2024-03-12T17:06:57.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.893+0000 7f52bd600640 1 -- 172.21.15.52:0/3056925370 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b40110d0 con 0x7f52c00caf30 2024-03-12T17:06:57.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.892+0000 7f7de8e00640 1 -- 172.21.15.52:0/2282806269 wait complete. 2024-03-12T17:06:57.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.894+0000 7f8a7e200640 1 -- start start 2024-03-12T17:06:57.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.894+0000 7f18f1800640 1 Processor -- start 2024-03-12T17:06:57.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.894+0000 7f18f1800640 1 -- start start 2024-03-12T17:06:57.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.895+0000 7f52c7e00640 1 -- 172.21.15.52:0/3056925370 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c013c390 con 0x7f52c00caf30 2024-03-12T17:06:57.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.896+0000 7f7de8e00640 1 Processor -- start 2024-03-12T17:06:57.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.893+0000 7f284f200640 1 -- start start 2024-03-12T17:06:57.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.895+0000 7f284f200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2850160ee0 0x7f28500cad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.895+0000 7f284f200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f28500cb2b0 0x7f28500cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.895+0000 7f284f200640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28500cb710 con 0x7f28500cca00 2024-03-12T17:06:57.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.895+0000 7f284f200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28500cd1b0 con 0x7f2850160ee0 2024-03-12T17:06:57.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.895+0000 7f284f200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28500cd490 con 0x7f28500cb2b0 2024-03-12T17:06:57.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.895+0000 7f8a7e200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8a700e3080 0x7f8a70057d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.896+0000 7f8a7e200640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a70058290 con 0x7f8a700542c0 2024-03-12T17:06:57.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.896+0000 7f8a7e200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a70058530 con 0x7f8a7003e800 2024-03-12T17:06:57.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.896+0000 7f8a7e200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a70006160 con 0x7f8a700e3080 2024-03-12T17:06:57.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.896+0000 7f7de8e00640 1 -- start start 2024-03-12T17:06:57.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.896+0000 7f52c7e00640 1 -- 172.21.15.52:0/3056925370 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c0139200 con 0x7f52c00caf30 2024-03-12T17:06:57.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.896+0000 7fb614a00640 1 Processor -- start 2024-03-12T17:06:57.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.896+0000 7f8a77400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8a700e3080 0x7f8a70057d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.896+0000 7f8a76a00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f8a7003e800 0x7f8a70053bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.52:35740/0 (socket says 172.21.15.52:35740) 2024-03-12T17:06:57.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.896+0000 7f8a76a00640 1 -- 172.21.15.52:0/1435808283 learned_addr learned my addr 172.21.15.52:0/1435808283 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:06:57.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.897+0000 7f18f1800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18ec0d0640 0x7f18ec1f14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.897+0000 7f18f1800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f18ec0d11b0 0x7f18ec1f19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.897+0000 7f18f1800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18ec15c830 0x7f18ec1f5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.897+0000 7f284de00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2850160ee0 0x7f28500cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.897+0000 7f284de00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2850160ee0 0x7f28500cad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60834/0 (socket says 172.21.15.52:60834) 2024-03-12T17:06:57.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.897+0000 7f284de00640 1 -- 172.21.15.52:0/2285060125 learned_addr learned my addr 172.21.15.52:0/2285060125 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:57.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.897+0000 7f18f1800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec1f1f30 con 0x7f18ec0d0640 2024-03-12T17:06:57.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.897+0000 7f18f1800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec0cc180 con 0x7f18ec15c830 2024-03-12T17:06:57.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.897+0000 7f18f1800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec0cc460 con 0x7f18ec0d11b0 2024-03-12T17:06:57.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.898+0000 7f18ebe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18ec0d0640 0x7f18ec1f14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.897+0000 7f8a77400640 1 -- 172.21.15.52:0/1435808283 >> v1:172.21.15.102:6789/0 conn(0x7f8a7003e800 legacy=0x7f8a70053bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.898+0000 7f8a77400640 1 -- 172.21.15.52:0/1435808283 >> v1:172.21.15.52:6789/0 conn(0x7f8a700542c0 legacy=0x7f8a70006890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.898+0000 7f8a77400640 1 -- 172.21.15.52:0/1435808283 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a70058810 con 0x7f8a700e3080 2024-03-12T17:06:57.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.898+0000 7f18eb400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f18ec0d11b0 0x7f18ec1f19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.898+0000 7f7de8e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ddc0061a0 0x7f7ddc0e74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.897+0000 7f284d400640 1 --2- 172.21.15.52:0/2285060125 >> v2:172.21.15.102:3300/0 conn(0x7f28500cb2b0 0x7f28500cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.897+0000 7fb614a00640 1 -- start start 2024-03-12T17:06:57.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.898+0000 7fb614a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb61006aca0 0x7fb6101f99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.898+0000 7fb614a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6100718b0 0x7fb610156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.898+0000 7fb614a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6100b1a90 0x7fb6101546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.898+0000 7fb614a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6101fa840 con 0x7fb6100b1a90 2024-03-12T17:06:57.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.898+0000 7fb614a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6101fa280 con 0x7fb6100718b0 2024-03-12T17:06:57.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.898+0000 7fb614a00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6101fa560 con 0x7fb61006aca0 2024-03-12T17:06:57.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.898+0000 7f7de8e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7ddc0eb060 0x7f7ddc052800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.898+0000 7f18f0e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18ec15c830 0x7f18ec1f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.898+0000 7f18f0e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18ec15c830 0x7f18ec1f5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:38550/0 (socket says 172.21.15.52:38550) 2024-03-12T17:06:57.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.898+0000 7f18f0e00640 1 -- 172.21.15.52:0/2025597461 learned_addr learned my addr 172.21.15.52:0/2025597461 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:57.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.899+0000 7f7de8e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ddc0ebb50 0x7f7ddc17fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.899+0000 7f18ebe00640 1 -- 172.21.15.52:0/2025597461 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f18ec0d11b0 msgr2=0x7f18ec1f19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.899+0000 7fb60ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6100718b0 0x7fb610156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.899+0000 7f18ebe00640 1 --2- 172.21.15.52:0/2025597461 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f18ec0d11b0 0x7f18ec1f19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.899+0000 7f18ebe00640 1 -- 172.21.15.52:0/2025597461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18ec15c830 msgr2=0x7f18ec1f5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.899+0000 7f7de1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ddc0061a0 0x7f7ddc0e74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.899+0000 7f7de1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ddc0061a0 0x7f7ddc0e74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:38560/0 (socket says 172.21.15.52:38560) 2024-03-12T17:06:57.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.899+0000 7f7de8e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc0dd8f0 con 0x7f7ddc0ebb50 2024-03-12T17:06:57.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.899+0000 7f7de8e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc04ef90 con 0x7f7ddc0061a0 2024-03-12T17:06:57.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.899+0000 7f7de8e00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc052e90 con 0x7f7ddc0eb060 2024-03-12T17:06:57.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.899+0000 7fb60fe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6100b1a90 0x7fb6101546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.899+0000 7fb60ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6100718b0 0x7fb610156690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:38556/0 (socket says 172.21.15.52:38556) 2024-03-12T17:06:57.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.900+0000 7fb60ea00640 1 -- 172.21.15.52:0/138744444 learned_addr learned my addr 172.21.15.52:0/138744444 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:57.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.900+0000 7f18ebe00640 1 --2- 172.21.15.52:0/2025597461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18ec15c830 0x7f18ec1f5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.900+0000 7f18ebe00640 1 -- 172.21.15.52:0/2025597461 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e4009a70 con 0x7f18ec0d0640 2024-03-12T17:06:57.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.900+0000 7f284de00640 1 -- 172.21.15.52:0/2285060125 >> v2:172.21.15.102:3300/0 conn(0x7f28500cb2b0 msgr2=0x7f28500cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:57.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.900+0000 7f284de00640 1 --2- 172.21.15.52:0/2285060125 >> v2:172.21.15.102:3300/0 conn(0x7f28500cb2b0 0x7f28500cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.900+0000 7f284de00640 1 -- 172.21.15.52:0/2285060125 >> v1:172.21.15.52:6789/0 conn(0x7f28500cca00 legacy=0x7f2850070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.899+0000 7f8a77400640 1 --2- 172.21.15.52:0/1435808283 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8a700e3080 0x7f8a70057d50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8a78063b00 tx=0x7f8a780a5700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b660ff96dad6eeea server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.900+0000 7f7de1400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7ddc0eb060 0x7f7ddc052800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.900+0000 7f7de1e00640 1 -- 172.21.15.52:0/278680511 learned_addr learned my addr 172.21.15.52:0/278680511 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:57.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.900+0000 7f18ebe00640 1 --2- 172.21.15.52:0/2025597461 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18ec0d0640 0x7f18ec1f14b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f18d800c750 tx=0x7f18d800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.900+0000 7f284de00640 1 -- 172.21.15.52:0/2285060125 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28500b1ca0 con 0x7f2850160ee0 2024-03-12T17:06:57.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.900+0000 7f8a76000640 1 -- 172.21.15.52:0/1435808283 <== mon.2 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a780aa7d0 con 0x7f8a700e3080 2024-03-12T17:06:57.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.900+0000 7f8a76000640 1 -- 172.21.15.52:0/1435808283 <== mon.2 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a780a8050 con 0x7f8a700e3080 2024-03-12T17:06:57.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.900+0000 7f8a76000640 1 -- 172.21.15.52:0/1435808283 <== mon.2 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a780aabb0 con 0x7f8a700e3080 2024-03-12T17:06:57.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.901+0000 7f9520400640 1 Processor -- start 2024-03-12T17:06:57.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.899+0000 7f52bd600640 1 -- 172.21.15.52:0/3056925370 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f52b40467d0 con 0x7f52c00caf30 2024-03-12T17:06:57.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.900+0000 7f52c7e00640 1 -- 172.21.15.52:0/3056925370 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f52c00cbcd0 con 0x7f52c00caf30 2024-03-12T17:06:57.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.901+0000 7f52bd600640 1 --2- 172.21.15.52:0/3056925370 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f52a006ac90 0x7f52a006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.901+0000 7f7de2800640 1 --2- 172.21.15.52:0/278680511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ddc0ebb50 0x7f7ddc17fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.900+0000 7fb60f400640 1 --2- 172.21.15.52:0/138744444 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb61006aca0 0x7fb6101f99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.901+0000 7f18e3e00640 1 -- 172.21.15.52:0/2025597461 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d8005ad0 con 0x7f18ec0d0640 2024-03-12T17:06:57.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.901+0000 7f18e3e00640 1 -- 172.21.15.52:0/2025597461 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d8005c30 con 0x7f18ec0d0640 2024-03-12T17:06:57.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.901+0000 7f18e3e00640 1 -- 172.21.15.52:0/2025597461 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d8007270 con 0x7f18ec0d0640 2024-03-12T17:06:57.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.901+0000 7f9520400640 1 -- start start 2024-03-12T17:06:57.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.900+0000 7f284d400640 1 --2- 172.21.15.52:0/2285060125 >> v2:172.21.15.102:3300/0 conn(0x7f28500cb2b0 0x7f28500cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:06:57.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.901+0000 7f7de1e00640 1 -- 172.21.15.52:0/278680511 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7ddc0eb060 msgr2=0x7f7ddc052800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.901+0000 7f7de1e00640 1 --2- 172.21.15.52:0/278680511 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7ddc0eb060 0x7f7ddc052800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.901+0000 7f7de1e00640 1 -- 172.21.15.52:0/278680511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ddc0ebb50 msgr2=0x7f7ddc17fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.902+0000 7f8a7e200640 1 -- 172.21.15.52:0/1435808283 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8a700e3080 msgr2=0x7f8a70057d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.902+0000 7f8a7e200640 1 --2- 172.21.15.52:0/1435808283 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8a700e3080 0x7f8a70057d50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8a78063b00 tx=0x7f8a780a5700 comp rx=0 tx=0).stop 2024-03-12T17:06:57.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.902+0000 7f8a7e200640 1 -- 172.21.15.52:0/1435808283 shutdown_connections 2024-03-12T17:06:57.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.902+0000 7f52c4a00640 1 --2- 172.21.15.52:0/3056925370 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f52a006ac90 0x7f52a006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.902+0000 7f52bd600640 1 -- 172.21.15.52:0/3056925370 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f52b4084e30 con 0x7f52c00caf30 2024-03-12T17:06:57.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.902+0000 7f7de1e00640 1 --2- 172.21.15.52:0/278680511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ddc0ebb50 0x7f7ddc17fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.902+0000 7f7de1e00640 1 -- 172.21.15.52:0/278680511 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd0009400 con 0x7f7ddc0061a0 2024-03-12T17:06:57.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.902+0000 7f8a7e200640 1 --2- 172.21.15.52:0/1435808283 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8a700e3080 0x7f8a70057d50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.902+0000 7f8a7e200640 1 -- 172.21.15.52:0/1435808283 >> 172.21.15.52:0/1435808283 conn(0x7f8a700559a0 msgr2=0x7f8a70055db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:57.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.902+0000 7f18f1800640 1 -- 172.21.15.52:0/2025597461 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18dc009400 con 0x7f18ec0d0640 2024-03-12T17:06:57.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.902+0000 7f18f1800640 1 -- 172.21.15.52:0/2025597461 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18ec15cd90 con 0x7f18ec0d0640 2024-03-12T17:06:57.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.903+0000 7f7de1e00640 1 --2- 172.21.15.52:0/278680511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ddc0061a0 0x7f7ddc0e74c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7dd400c410 tx=0x7f7dd400c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.903+0000 7f52bd600640 1 --2- 172.21.15.52:0/3056925370 >> [v2:172.21.15.52:6800/37214,v1:172.21.15.52:6801/37214] conn(0x7f52a00741d0 0x7f52a0076610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.903+0000 7f52bd600640 1 -- 172.21.15.52:0/3056925370 --> [v2:172.21.15.52:6800/37214,v1:172.21.15.52:6801/37214] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f52a0076ce0 con 0x7f52a00741d0 2024-03-12T17:06:57.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.903+0000 7f52bd600640 1 -- 172.21.15.52:0/3056925370 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52b407e7c0 con 0x7f52c00caf30 2024-03-12T17:06:57.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.903+0000 7f284de00640 1 --2- 172.21.15.52:0/2285060125 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2850160ee0 0x7f28500cad70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f285006a970 tx=0x7f284002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02406b0d9da4aa0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.902+0000 7f9520400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f9518160cd0 0x7f951806c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.903+0000 7f9520400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f95180d0500 0x7f95180b2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.902+0000 7fb60fe00640 1 -- 172.21.15.52:0/138744444 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb61006aca0 msgr2=0x7fb6101f99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.903+0000 7f9520400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95180d0970 con 0x7f95181610b0 2024-03-12T17:06:57.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.903+0000 7f9520400640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95180c3ec0 con 0x7f9518160cd0 2024-03-12T17:06:57.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.903+0000 7f9520400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f951806b050 con 0x7f95180d0500 2024-03-12T17:06:57.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.902+0000 7fb60fe00640 1 --2- 172.21.15.52:0/138744444 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb61006aca0 0x7fb6101f99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.902+0000 7fb60fe00640 1 -- 172.21.15.52:0/138744444 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6100718b0 msgr2=0x7fb610156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.902+0000 7fb60fe00640 1 --2- 172.21.15.52:0/138744444 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6100718b0 0x7fb610156690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.902+0000 7fb60fe00640 1 -- 172.21.15.52:0/138744444 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6000094d0 con 0x7fb6100b1a90 2024-03-12T17:06:57.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.903+0000 7fb60ea00640 1 --2- 172.21.15.52:0/138744444 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6100718b0 0x7fb610156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:06:57.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.903+0000 7fb60f400640 1 --2- 172.21.15.52:0/138744444 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb61006aca0 0x7fb6101f99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:57.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.903+0000 7f7dda000640 1 -- 172.21.15.52:0/278680511 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd40035f0 con 0x7f7ddc0061a0 2024-03-12T17:06:57.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.903+0000 7fa3d2600640 1 Processor -- start 2024-03-12T17:06:57.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.904+0000 7f52c5e00640 1 --2- 172.21.15.52:0/3056925370 >> [v2:172.21.15.52:6800/37214,v1:172.21.15.52:6801/37214] conn(0x7f52a00741d0 0x7f52a0076610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.904+0000 7fb60fe00640 1 --2- 172.21.15.52:0/138744444 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6100b1a90 0x7fb6101546e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb608033000 tx=0x7fb608004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.904+0000 7f52c4a00640 1 --2- 172.21.15.52:0/3056925370 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f52a006ac90 0x7f52a006d150 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f52c00cc5f0 tx=0x7f52ac009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.904+0000 7f7de8e00640 1 -- 172.21.15.52:0/278680511 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ddc04e660 con 0x7f7ddc0061a0 2024-03-12T17:06:57.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.904+0000 7f7de8e00640 1 -- 172.21.15.52:0/278680511 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ddc04ec20 con 0x7f7ddc0061a0 2024-03-12T17:06:57.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.904+0000 7f7dda000640 1 -- 172.21.15.52:0/278680511 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd4003750 con 0x7f7ddc0061a0 2024-03-12T17:06:57.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.904+0000 7f7dda000640 1 -- 172.21.15.52:0/278680511 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd4003b40 con 0x7f7ddc0061a0 2024-03-12T17:06:57.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.904+0000 7f951da00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f9518160cd0 0x7f951806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.904+0000 7f951d000640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f95181610b0 0x7f95180cacf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.52:35746/0 (socket says 172.21.15.52:35746) 2024-03-12T17:06:57.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.904+0000 7f951d000640 1 -- 172.21.15.52:0/3561190540 learned_addr learned my addr 172.21.15.52:0/3561190540 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:06:57.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.904+0000 7f18f1800640 1 -- 172.21.15.52:0/2025597461 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f18ec15cc10 con 0x7f18ec0d0640 2024-03-12T17:06:57.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.904+0000 7f951e400640 1 --2- 172.21.15.52:0/3561190540 >> v2:172.21.15.102:3300/0 conn(0x7f95180d0500 0x7f95180b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.905+0000 7f951da00640 1 -- 172.21.15.52:0/3561190540 >> v2:172.21.15.102:3300/0 conn(0x7f95180d0500 msgr2=0x7f95180b2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:57.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.905+0000 7f951da00640 1 --2- 172.21.15.52:0/3561190540 >> v2:172.21.15.102:3300/0 conn(0x7f95180d0500 0x7f95180b2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.905+0000 7f951da00640 1 -- 172.21.15.52:0/3561190540 >> v1:172.21.15.102:6789/0 conn(0x7f95181610b0 legacy=0x7f95180cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.905+0000 7f284ca00640 1 -- 172.21.15.52:0/2285060125 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2840030860 con 0x7f2850160ee0 2024-03-12T17:06:57.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.905+0000 7f284ca00640 1 -- 172.21.15.52:0/2285060125 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28400309c0 con 0x7f2850160ee0 2024-03-12T17:06:57.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.905+0000 7f284ca00640 1 -- 172.21.15.52:0/2285060125 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2840030d50 con 0x7f2850160ee0 2024-03-12T17:06:57.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.904+0000 7fa3d2600640 1 -- start start 2024-03-12T17:06:57.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.902+0000 7f8a7e200640 1 -- 172.21.15.52:0/1435808283 shutdown_connections 2024-03-12T17:06:57.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.902+0000 7f8a7e200640 1 -- 172.21.15.52:0/1435808283 wait complete. 2024-03-12T17:06:57.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.905+0000 7f8a7e200640 1 Processor -- start 2024-03-12T17:06:57.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.905+0000 7f8a7e200640 1 -- start start 2024-03-12T17:06:57.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.905+0000 7f52c5e00640 1 --2- 172.21.15.52:0/3056925370 >> [v2:172.21.15.52:6800/37214,v1:172.21.15.52:6801/37214] conn(0x7f52a00741d0 0x7f52a0076610 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.905+0000 7f951da00640 1 -- 172.21.15.52:0/3561190540 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95180d0bb0 con 0x7f9518160cd0 2024-03-12T17:06:57.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.906+0000 7f7de8e00640 1 -- 172.21.15.52:0/278680511 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7dbc000fc0 con 0x7f7ddc0061a0 2024-03-12T17:06:57.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.906+0000 7f951e400640 1 --2- 172.21.15.52:0/3561190540 >> v2:172.21.15.102:3300/0 conn(0x7f95180d0500 0x7f95180b2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:57.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.906+0000 7fb607400640 1 -- 172.21.15.52:0/138744444 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb608004800 con 0x7fb6100b1a90 2024-03-12T17:06:57.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.906+0000 7fb607400640 1 -- 172.21.15.52:0/138744444 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb608009450 con 0x7fb6100b1a90 2024-03-12T17:06:57.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.906+0000 7fb607400640 1 -- 172.21.15.52:0/138744444 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb608034680 con 0x7fb6100b1a90 2024-03-12T17:06:57.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.906+0000 7fb614a00640 1 -- 172.21.15.52:0/138744444 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb608009a70 con 0x7fb6100b1a90 2024-03-12T17:06:57.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.906+0000 7f52bd600640 1 -- 172.21.15.52:0/3056925370 <== osd.0 v2:172.21.15.52:6800/37214 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f52a0076ce0 con 0x7f52a00741d0 2024-03-12T17:06:57.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.906+0000 7f284f200640 1 -- 172.21.15.52:0/2285060125 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2850160ee0 msgr2=0x7f28500cad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.906+0000 7f284f200640 1 --2- 172.21.15.52:0/2285060125 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2850160ee0 0x7f28500cad70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f285006a970 tx=0x7f284002b9a0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.906+0000 7fb614a00640 1 -- 172.21.15.52:0/138744444 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6101541c0 con 0x7fb6100b1a90 2024-03-12T17:06:57.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.907+0000 7f18e3e00640 1 -- 172.21.15.52:0/2025597461 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f18d80073d0 con 0x7f18ec0d0640 2024-03-12T17:06:57.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.907+0000 7f284f200640 1 -- 172.21.15.52:0/2285060125 shutdown_connections 2024-03-12T17:06:57.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.907+0000 7f284f200640 1 --2- 172.21.15.52:0/2285060125 >> v2:172.21.15.102:3300/0 conn(0x7f28500cb2b0 0x7f28500cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.907+0000 7f284f200640 1 --2- 172.21.15.52:0/2285060125 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2850160ee0 0x7f28500cad70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.907+0000 7f284f200640 1 -- 172.21.15.52:0/2285060125 >> 172.21.15.52:0/2285060125 conn(0x7f2850065e20 msgr2=0x7f2850069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:57.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.907+0000 7f8a7e200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a7003e800 0x7f8a70182b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.907+0000 7f8a7e200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8a700542c0 0x7f8a7004f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.907+0000 7f8a7e200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a700e3080 0x7f8a7003cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.908+0000 7fb604c00640 1 -- 172.21.15.52:0/138744444 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb5e4000fc0 con 0x7fb6100b1a90 2024-03-12T17:06:57.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.908+0000 7fa3d2600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa3cc165340 0x7fa3cc0ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.908+0000 7fa3d2600640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3cc1657c0 con 0x7fa3cc1627c0 2024-03-12T17:06:57.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.908+0000 7fa3d2600640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3cc06cdc0 con 0x7fa3cc0ce930 2024-03-12T17:06:57.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.908+0000 7fa3d2600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3cc0cf250 con 0x7fa3cc165340 2024-03-12T17:06:57.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.908+0000 7f284f200640 1 -- 172.21.15.52:0/2285060125 shutdown_connections 2024-03-12T17:06:57.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.908+0000 7f18e3e00640 1 --2- 172.21.15.52:0/2025597461 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f18cc06ac50 0x7f18cc06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.908+0000 7f18e3e00640 1 -- 172.21.15.52:0/2025597461 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f18d8088f80 con 0x7f18ec0d0640 2024-03-12T17:06:57.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.908+0000 7f951da00640 1 --2- 172.21.15.52:0/3561190540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9518160cd0 0x7f951806c990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f95180c40f0 tx=0x7f950802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf9d9f41ac11f2e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.908+0000 7fa3caa00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa3cc165340 0x7fa3cc0ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.908+0000 7fa3caa00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa3cc165340 0x7fa3cc0ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:38598/0 (socket says 172.21.15.52:38598) 2024-03-12T17:06:57.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.908+0000 7fa3caa00640 1 -- 172.21.15.52:0/2471415818 learned_addr learned my addr 172.21.15.52:0/2471415818 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:57.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.909+0000 7f18e3e00640 1 --2- 172.21.15.52:0/2025597461 >> [v2:172.21.15.52:6816/37217,v1:172.21.15.52:6818/37217] conn(0x7f18cc074210 0x7f18cc076650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.908+0000 7f8a77400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a7003e800 0x7f8a70182b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.908+0000 7f8a76a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8a700542c0 0x7f8a7004f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.909+0000 7f8a76a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8a700542c0 0x7f8a7004f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.52:55708/0 (socket says 172.21.15.52:55708) 2024-03-12T17:06:57.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.909+0000 7f8a77400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a7003e800 0x7f8a70182b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:38584/0 (socket says 172.21.15.52:38584) 2024-03-12T17:06:57.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.909+0000 7f8a7e200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a70186310 con 0x7f8a700e3080 2024-03-12T17:06:57.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.909+0000 7f8a7e200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7003c7d0 con 0x7f8a7003e800 2024-03-12T17:06:57.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.909+0000 7f8a7e200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a70185690 con 0x7f8a700542c0 2024-03-12T17:06:57.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.909+0000 7f8a76a00640 1 -- 172.21.15.52:0/2275790810 learned_addr learned my addr 172.21.15.52:0/2275790810 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:57.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.909+0000 7f9517e00640 1 -- 172.21.15.52:0/3561190540 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950802cd10 con 0x7f9518160cd0 2024-03-12T17:06:57.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.909+0000 7f7dda000640 1 -- 172.21.15.52:0/278680511 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7dd4003ca0 con 0x7f7ddc0061a0 2024-03-12T17:06:57.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.909+0000 7f18e3e00640 1 -- 172.21.15.52:0/2025597461 --> [v2:172.21.15.52:6816/37217,v1:172.21.15.52:6818/37217] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f18cc076d20 con 0x7f18cc074210 2024-03-12T17:06:57.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.909+0000 7f18f0e00640 1 --2- 172.21.15.52:0/2025597461 >> [v2:172.21.15.52:6816/37217,v1:172.21.15.52:6818/37217] conn(0x7f18cc074210 0x7f18cc076650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.908+0000 7f284f200640 1 -- 172.21.15.52:0/2285060125 wait complete. 2024-03-12T17:06:57.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.910+0000 7f9517e00640 1 -- 172.21.15.52:0/3561190540 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f950802ce70 con 0x7f9518160cd0 2024-03-12T17:06:57.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.910+0000 7f9517e00640 1 -- 172.21.15.52:0/3561190540 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950802cd10 con 0x7f9518160cd0 2024-03-12T17:06:57.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.910+0000 7f8a77400640 1 -- 172.21.15.52:0/2275790810 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8a700542c0 msgr2=0x7f8a7004f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.910+0000 7f8a77400640 1 --2- 172.21.15.52:0/2275790810 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8a700542c0 0x7f8a7004f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.910+0000 7f8a77400640 1 -- 172.21.15.52:0/2275790810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a700e3080 msgr2=0x7f8a7003cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.910+0000 7f8a77400640 1 --2- 172.21.15.52:0/2275790810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a700e3080 0x7f8a7003cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.910+0000 7f8a77400640 1 -- 172.21.15.52:0/2275790810 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a78073170 con 0x7f8a7003e800 2024-03-12T17:06:57.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.910+0000 7f18eb400640 1 --2- 172.21.15.52:0/2025597461 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f18cc06ac50 0x7f18cc06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.910+0000 7f18e3e00640 1 -- 172.21.15.52:0/2025597461 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18d8053fe0 con 0x7f18ec0d0640 2024-03-12T17:06:57.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.910+0000 7f8a77400640 1 --2- 172.21.15.52:0/2275790810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a7003e800 0x7f8a70182b90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8a7804ede0 tx=0x7f8a78068280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.911+0000 7f9520400640 1 -- 172.21.15.52:0/3561190540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9518160cd0 msgr2=0x7f951806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.911+0000 7f9520400640 1 --2- 172.21.15.52:0/3561190540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9518160cd0 0x7f951806c990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f95180c40f0 tx=0x7f950802bf60 comp rx=0 tx=0).stop 2024-03-12T17:06:57.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.909+0000 7fa3caa00640 1 -- 172.21.15.52:0/2471415818 >> v1:172.21.15.102:6789/0 conn(0x7fa3cc0ce930 legacy=0x7fa3cc1612d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.909+0000 7fa3caa00640 1 -- 172.21.15.52:0/2471415818 >> v1:172.21.15.52:6789/0 conn(0x7fa3cc1627c0 legacy=0x7fa3cc06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:57.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.909+0000 7fa3caa00640 1 -- 172.21.15.52:0/2471415818 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3cc161ab0 con 0x7fa3cc165340 2024-03-12T17:06:57.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.911+0000 7f18f0e00640 1 --2- 172.21.15.52:0/2025597461 >> [v2:172.21.15.52:6816/37217,v1:172.21.15.52:6818/37217] conn(0x7f18cc074210 0x7f18cc076650 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.911+0000 7f7dda000640 1 --2- 172.21.15.52:0/278680511 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f7dc406ac00 0x7f7dc406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.911+0000 7fa3caa00640 1 --2- 172.21.15.52:0/2471415818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3cc165340 0x7fa3cc0ced10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc06e550 tx=0x7fa3b802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67d2d25d64985a45 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.911+0000 7f8a6b400640 1 -- 172.21.15.52:0/2275790810 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a780acac0 con 0x7f8a7003e800 2024-03-12T17:06:57.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.911+0000 7f9520400640 1 -- 172.21.15.52:0/3561190540 shutdown_connections 2024-03-12T17:06:57.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.911+0000 7f9520400640 1 --2- 172.21.15.52:0/3561190540 >> v2:172.21.15.102:3300/0 conn(0x7f95180d0500 0x7f95180b2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.911+0000 7f9520400640 1 --2- 172.21.15.52:0/3561190540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9518160cd0 0x7f951806c990 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.911+0000 7f9520400640 1 -- 172.21.15.52:0/3561190540 >> 172.21.15.52:0/3561190540 conn(0x7f9518069c90 msgr2=0x7f951806a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:57.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.911+0000 7f7dda000640 1 -- 172.21.15.52:0/278680511 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7dd4088fa0 con 0x7f7ddc0061a0 2024-03-12T17:06:57.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.912+0000 7f7dda000640 1 --2- 172.21.15.52:0/278680511 >> [v2:172.21.15.52:6817/37218,v1:172.21.15.52:6819/37218] conn(0x7f7dc40741c0 0x7f7dc4076600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.912+0000 7f7dda000640 1 -- 172.21.15.52:0/278680511 --> [v2:172.21.15.52:6817/37218,v1:172.21.15.52:6819/37218] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7dc4076cd0 con 0x7f7dc40741c0 2024-03-12T17:06:57.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.912+0000 7f7dda000640 1 -- 172.21.15.52:0/278680511 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7dd40893f0 con 0x7f7ddc0061a0 2024-03-12T17:06:57.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.912+0000 7f7de1400640 1 --2- 172.21.15.52:0/278680511 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f7dc406ac00 0x7f7dc406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.912+0000 7f18eb400640 1 --2- 172.21.15.52:0/2025597461 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f18cc06ac50 0x7f18cc06d110 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f18dc034000 tx=0x7f18dc008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.912+0000 7f9520400640 1 -- 172.21.15.52:0/3561190540 shutdown_connections 2024-03-12T17:06:57.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.912+0000 7f284f200640 1 Processor -- start 2024-03-12T17:06:57.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.911+0000 7f8a6b400640 1 -- 172.21.15.52:0/2275790810 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a780acc20 con 0x7f8a7003e800 2024-03-12T17:06:57.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.911+0000 7f8a6b400640 1 -- 172.21.15.52:0/2275790810 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a780b6410 con 0x7f8a7003e800 2024-03-12T17:06:57.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.912+0000 7fa3ca000640 1 -- 172.21.15.52:0/2471415818 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8030840 con 0x7fa3cc165340 2024-03-12T17:06:57.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.912+0000 7f8a7e200640 1 -- 172.21.15.52:0/2275790810 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a70185970 con 0x7f8a7003e800 2024-03-12T17:06:57.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.912+0000 7f8a7e200640 1 -- 172.21.15.52:0/2275790810 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a70185f30 con 0x7f8a7003e800 2024-03-12T17:06:57.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.912+0000 7fa3ca000640 1 -- 172.21.15.52:0/2471415818 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b80309a0 con 0x7fa3cc165340 2024-03-12T17:06:57.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.912+0000 7fa3ca000640 1 -- 172.21.15.52:0/2471415818 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8030d30 con 0x7fa3cc165340 2024-03-12T17:06:57.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.912+0000 7f9520400640 1 -- 172.21.15.52:0/3561190540 wait complete. 2024-03-12T17:06:57.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.913+0000 7f7de2800640 1 --2- 172.21.15.52:0/278680511 >> [v2:172.21.15.52:6817/37218,v1:172.21.15.52:6819/37218] conn(0x7f7dc40741c0 0x7f7dc4076600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.913+0000 7f8a7e200640 1 -- 172.21.15.52:0/2275790810 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8a7003bf80 con 0x7f8a7003e800 2024-03-12T17:06:57.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.913+0000 7f284f200640 1 -- start start 2024-03-12T17:06:57.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.913+0000 7fb607400640 1 -- 172.21.15.52:0/138744444 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb6080347e0 con 0x7fb6100b1a90 2024-03-12T17:06:57.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.913+0000 7fa3d2600640 1 -- 172.21.15.52:0/2471415818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3cc165340 msgr2=0x7fa3cc0ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.913+0000 7fa3d2600640 1 --2- 172.21.15.52:0/2471415818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3cc165340 0x7fa3cc0ced10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc06e550 tx=0x7fa3b802b980 comp rx=0 tx=0).stop 2024-03-12T17:06:57.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.914+0000 7f18e3e00640 1 -- 172.21.15.52:0/2025597461 <== osd.3 v2:172.21.15.52:6816/37217 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f18cc076d20 con 0x7f18cc074210 2024-03-12T17:06:57.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.914+0000 7f284f200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28500cb2b0 0x7f28500af120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.914+0000 7f284f200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f28500cca00 0x7f2850070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.914+0000 7f284f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2850160ee0 0x7f2850070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.914+0000 7f284f200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2850162ac0 con 0x7f28500cb2b0 2024-03-12T17:06:57.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.914+0000 7f284f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2850146f90 con 0x7f2850160ee0 2024-03-12T17:06:57.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.914+0000 7f284f200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2850161830 con 0x7f28500cca00 2024-03-12T17:06:57.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.915+0000 7fa3d2600640 1 -- 172.21.15.52:0/2471415818 shutdown_connections 2024-03-12T17:06:57.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.913+0000 7f7de1400640 1 --2- 172.21.15.52:0/278680511 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f7dc406ac00 0x7f7dc406d0c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0036000 tx=0x7f7dd0008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.914+0000 7f7de2800640 1 --2- 172.21.15.52:0/278680511 >> [v2:172.21.15.52:6817/37218,v1:172.21.15.52:6819/37218] conn(0x7f7dc40741c0 0x7f7dc4076600 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.915+0000 7fa3d2600640 1 --2- 172.21.15.52:0/2471415818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3cc165340 0x7fa3cc0ced10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.915+0000 7fa3d2600640 1 -- 172.21.15.52:0/2471415818 >> 172.21.15.52:0/2471415818 conn(0x7fa3cc06a0b0 msgr2=0x7fa3cc06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:57.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.915+0000 7fa3d2600640 1 -- 172.21.15.52:0/2471415818 shutdown_connections 2024-03-12T17:06:57.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.915+0000 7fa3d2600640 1 -- 172.21.15.52:0/2471415818 wait complete. 2024-03-12T17:06:57.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.915+0000 7f284e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2850160ee0 0x7f2850070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.915+0000 7f284e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2850160ee0 0x7f2850070560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:38606/0 (socket says 172.21.15.52:38606) 2024-03-12T17:06:57.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.915+0000 7f284e800640 1 -- 172.21.15.52:0/4264660140 learned_addr learned my addr 172.21.15.52:0/4264660140 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:57.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.915+0000 7f9520400640 1 Processor -- start 2024-03-12T17:06:57.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.915+0000 7fb607400640 1 --2- 172.21.15.52:0/138744444 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb5f006ac00 0x7fb5f006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.915+0000 7fb607400640 1 -- 172.21.15.52:0/138744444 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb6080ac740 con 0x7fb6100b1a90 2024-03-12T17:06:57.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.915+0000 7fb607400640 1 --2- 172.21.15.52:0/138744444 >> [v2:172.21.15.52:6808/37216,v1:172.21.15.52:6809/37216] conn(0x7fb5f00741c0 0x7fb5f0076600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.915+0000 7fb607400640 1 -- 172.21.15.52:0/138744444 --> [v2:172.21.15.52:6808/37216,v1:172.21.15.52:6809/37216] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb5f0076cd0 con 0x7fb5f00741c0 2024-03-12T17:06:57.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.915+0000 7fb607400640 1 -- 172.21.15.52:0/138744444 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6080acb90 con 0x7fb6100b1a90 2024-03-12T17:06:57.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.915+0000 7f7dda000640 1 -- 172.21.15.52:0/278680511 <== osd.1 v2:172.21.15.52:6817/37218 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f7dc4076cd0 con 0x7f7dc40741c0 2024-03-12T17:06:57.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.916+0000 7f284d400640 1 --2- 172.21.15.52:0/4264660140 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f28500cca00 0x7f2850070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.916+0000 7f284de00640 1 --2- 172.21.15.52:0/4264660140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28500cb2b0 0x7f28500af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.916+0000 7fb60ea00640 1 --2- 172.21.15.52:0/138744444 >> [v2:172.21.15.52:6808/37216,v1:172.21.15.52:6809/37216] conn(0x7fb5f00741c0 0x7fb5f0076600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.916+0000 7f284e800640 1 -- 172.21.15.52:0/4264660140 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f28500cca00 msgr2=0x7f2850070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.916+0000 7f284e800640 1 --2- 172.21.15.52:0/4264660140 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f28500cca00 0x7f2850070020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.916+0000 7f284e800640 1 -- 172.21.15.52:0/4264660140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28500cb2b0 msgr2=0x7f28500af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.916+0000 7f284e800640 1 --2- 172.21.15.52:0/4264660140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28500cb2b0 0x7f28500af120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.916+0000 7f284e800640 1 -- 172.21.15.52:0/4264660140 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28400094d0 con 0x7f2850160ee0 2024-03-12T17:06:57.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.916+0000 7f284de00640 1 --2- 172.21.15.52:0/4264660140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28500cb2b0 0x7f28500af120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:06:57.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.916+0000 7f8a6b400640 1 -- 172.21.15.52:0/2275790810 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8a780b3050 con 0x7f8a7003e800 2024-03-12T17:06:57.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.917+0000 7f284e800640 1 --2- 172.21.15.52:0/4264660140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2850160ee0 0x7f2850070560 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f284800c710 tx=0x7f284800cbd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.917+0000 7f2846000640 1 -- 172.21.15.52:0/4264660140 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2848006d50 con 0x7f2850160ee0 2024-03-12T17:06:57.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.917+0000 7f284f200640 1 -- 172.21.15.52:0/4264660140 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2850147250 con 0x7f2850160ee0 2024-03-12T17:06:57.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.917+0000 7f284f200640 1 -- 172.21.15.52:0/4264660140 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2850147810 con 0x7f2850160ee0 2024-03-12T17:06:57.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.917+0000 7f2846000640 1 -- 172.21.15.52:0/4264660140 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284800ce20 con 0x7f2850160ee0 2024-03-12T17:06:57.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.917+0000 7f2846000640 1 -- 172.21.15.52:0/4264660140 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2848003bb0 con 0x7f2850160ee0 2024-03-12T17:06:57.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.918+0000 7f8a6b400640 1 --2- 172.21.15.52:0/2275790810 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f8a5806acd0 0x7f8a5806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.918+0000 7f8a6b400640 1 -- 172.21.15.52:0/2275790810 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8a781245b0 con 0x7f8a7003e800 2024-03-12T17:06:57.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.918+0000 7fa3d2600640 1 Processor -- start 2024-03-12T17:06:57.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.918+0000 7f8a6b400640 1 --2- 172.21.15.52:0/2275790810 >> [v2:172.21.15.102:6824/37214,v1:172.21.15.102:6825/37214] conn(0x7f8a58074310 0x7f8a58076750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.918+0000 7f8a6b400640 1 -- 172.21.15.52:0/2275790810 --> [v2:172.21.15.102:6824/37214,v1:172.21.15.102:6825/37214] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8a58076e20 con 0x7f8a58074310 2024-03-12T17:06:57.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.918+0000 7f8a6b400640 1 -- 172.21.15.52:0/2275790810 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a78124960 con 0x7f8a7003e800 2024-03-12T17:06:57.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.917+0000 7f9520400640 1 -- start start 2024-03-12T17:06:57.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.919+0000 7f8a76a00640 1 --2- 172.21.15.52:0/2275790810 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f8a5806acd0 0x7f8a5806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.918+0000 7f284f200640 1 -- 172.21.15.52:0/4264660140 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2828000fc0 con 0x7f2850160ee0 2024-03-12T17:06:57.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.919+0000 7f9520400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95180d0500 0x7f9518157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.919+0000 7f9520400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9518160cd0 0x7f95181f7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.919+0000 7f951da00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95180d0500 0x7f9518157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.919+0000 7f951da00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95180d0500 0x7f9518157dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60886/0 (socket says 172.21.15.52:60886) 2024-03-12T17:06:57.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.919+0000 7f9520400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95181610b0 0x7f95181f8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.919+0000 7f9520400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95181f8b00 con 0x7f95180d0500 2024-03-12T17:06:57.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.919+0000 7f951da00640 1 -- 172.21.15.52:0/2234509420 learned_addr learned my addr 172.21.15.52:0/2234509420 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:57.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.919+0000 7f9520400640 1 -- 172.21.15.52:0/2234509420 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9518158310 con 0x7f9518160cd0 2024-03-12T17:06:57.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.919+0000 7f9520400640 1 -- 172.21.15.52:0/2234509420 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95181f7e80 con 0x7f95181610b0 2024-03-12T17:06:57.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.917+0000 7fb60ea00640 1 --2- 172.21.15.52:0/138744444 >> [v2:172.21.15.52:6808/37216,v1:172.21.15.52:6809/37216] conn(0x7fb5f00741c0 0x7fb5f0076600 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.917+0000 7fb60f400640 1 --2- 172.21.15.52:0/138744444 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb5f006ac00 0x7fb5f006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.920+0000 7fb607400640 1 -- 172.21.15.52:0/138744444 <== osd.2 v2:172.21.15.52:6808/37216 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fb5f0076cd0 con 0x7fb5f00741c0 2024-03-12T17:06:57.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.920+0000 7f951e400640 1 --2- 172.21.15.52:0/2234509420 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95181610b0 0x7f95181f8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.920+0000 7f951d000640 1 --2- 172.21.15.52:0/2234509420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9518160cd0 0x7f95181f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.920+0000 7f951da00640 1 -- 172.21.15.52:0/2234509420 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95181610b0 msgr2=0x7f95181f8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.920+0000 7f8a76a00640 1 --2- 172.21.15.52:0/2275790810 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f8a5806acd0 0x7f8a5806d190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8a70006080 tx=0x7f8a64009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.920+0000 7f951da00640 1 --2- 172.21.15.52:0/2234509420 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95181610b0 0x7f95181f8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.920+0000 7f951da00640 1 -- 172.21.15.52:0/2234509420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9518160cd0 msgr2=0x7f95181f7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.920+0000 7f951da00640 1 --2- 172.21.15.52:0/2234509420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9518160cd0 0x7f95181f7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.920+0000 7f951da00640 1 -- 172.21.15.52:0/2234509420 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9508009a70 con 0x7f95180d0500 2024-03-12T17:06:57.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.920+0000 7f951e400640 1 --2- 172.21.15.52:0/2234509420 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95181610b0 0x7f95181f8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:57.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.920+0000 7f951d000640 1 --2- 172.21.15.52:0/2234509420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9518160cd0 0x7f95181f7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:57.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.921+0000 7f8a77e00640 1 --2- 172.21.15.52:0/2275790810 >> [v2:172.21.15.102:6824/37214,v1:172.21.15.102:6825/37214] conn(0x7f8a58074310 0x7f8a58076750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.919+0000 7fa3d2600640 1 -- start start 2024-03-12T17:06:57.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.921+0000 7fa3d2600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3cc0ce930 0x7fa3cc15c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.921+0000 7fa3d2600640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa3cc1627c0 0x7fa3cc147220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.921+0000 7fa3d2600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3cc165340 0x7fa3cc15a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.921+0000 7fa3d2600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc148820 con 0x7fa3cc0ce930 2024-03-12T17:06:57.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.921+0000 7fa3d2600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc147c00 con 0x7fa3cc165340 2024-03-12T17:06:57.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.921+0000 7fa3d2600640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc147ee0 con 0x7fa3cc1627c0 2024-03-12T17:06:57.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.922+0000 7fb60f400640 1 --2- 172.21.15.52:0/138744444 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb5f006ac00 0x7fb5f006d0c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc000f60 tx=0x7fb5fc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.922+0000 7fa3cb400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3cc0ce930 0x7fa3cc15c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.922+0000 7fa3cbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3cc165340 0x7fa3cc15a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.922+0000 7f8a77e00640 1 --2- 172.21.15.52:0/2275790810 >> [v2:172.21.15.102:6824/37214,v1:172.21.15.102:6825/37214] conn(0x7f8a58074310 0x7f8a58076750 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.922+0000 7fa3cbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3cc165340 0x7fa3cc15a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:38628/0 (socket says 172.21.15.52:38628) 2024-03-12T17:06:57.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.922+0000 7fa3cbe00640 1 -- 172.21.15.52:0/1845369396 learned_addr learned my addr 172.21.15.52:0/1845369396 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:57.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.922+0000 7f2846000640 1 -- 172.21.15.52:0/4264660140 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2848003d10 con 0x7f2850160ee0 2024-03-12T17:06:57.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.922+0000 7fa3cb400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3cc0ce930 0x7fa3cc15c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60888/0 (socket says 172.21.15.52:60888) 2024-03-12T17:06:57.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.922+0000 7fa3caa00640 1 --2- 172.21.15.52:0/1845369396 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa3cc1627c0 0x7fa3cc147220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.923+0000 7f2846000640 1 --2- 172.21.15.52:0/4264660140 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f283006ac30 0x7f283006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.923+0000 7fa3cb400640 1 -- 172.21.15.52:0/1845369396 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa3cc1627c0 msgr2=0x7fa3cc147220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.924+0000 7fa3cb400640 1 --2- 172.21.15.52:0/1845369396 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa3cc1627c0 0x7fa3cc147220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.924+0000 7fa3cb400640 1 -- 172.21.15.52:0/1845369396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3cc165340 msgr2=0x7fa3cc15a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.924+0000 7fa3cb400640 1 --2- 172.21.15.52:0/1845369396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3cc165340 0x7fa3cc15a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.924+0000 7f2846000640 1 -- 172.21.15.52:0/4264660140 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2848089f40 con 0x7f2850160ee0 2024-03-12T17:06:57.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.922+0000 7f951da00640 1 --2- 172.21.15.52:0/2234509420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95180d0500 0x7f9518157dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9508009e80 tx=0x7f9508004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.923+0000 7f9515600640 1 -- 172.21.15.52:0/2234509420 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9508040450 con 0x7f95180d0500 2024-03-12T17:06:57.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.923+0000 7f9515600640 1 -- 172.21.15.52:0/2234509420 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95080405d0 con 0x7f95180d0500 2024-03-12T17:06:57.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.923+0000 7f9515600640 1 -- 172.21.15.52:0/2234509420 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9508040980 con 0x7f95180d0500 2024-03-12T17:06:57.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.923+0000 7f9520400640 1 -- 172.21.15.52:0/2234509420 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95181f8160 con 0x7f95180d0500 2024-03-12T17:06:57.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.924+0000 7f9520400640 1 -- 172.21.15.52:0/2234509420 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95181f8720 con 0x7f95180d0500 2024-03-12T17:06:57.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.924+0000 7fa3cb400640 1 -- 172.21.15.52:0/1845369396 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b80094d0 con 0x7fa3cc0ce930 2024-03-12T17:06:57.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.924+0000 7f2846000640 1 --2- 172.21.15.52:0/4264660140 >> [v2:172.21.15.102:6816/37201,v1:172.21.15.102:6817/37201] conn(0x7f2830074270 0x7f28300766b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.924+0000 7f2846000640 1 -- 172.21.15.52:0/4264660140 --> [v2:172.21.15.102:6816/37201,v1:172.21.15.102:6817/37201] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2830076d80 con 0x7f2830074270 2024-03-12T17:06:57.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.924+0000 7f2846000640 1 -- 172.21.15.52:0/4264660140 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f284808a2f0 con 0x7f2850160ee0 2024-03-12T17:06:57.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.924+0000 7fa3caa00640 1 --2- 172.21.15.52:0/1845369396 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa3cc1627c0 0x7fa3cc147220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:06:57.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.924+0000 7fa3cbe00640 1 --2- 172.21.15.52:0/1845369396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3cc165340 0x7fa3cc15a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:06:57.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.924+0000 7f8a6b400640 1 -- 172.21.15.52:0/2275790810 <== osd.7 v2:172.21.15.102:6824/37214 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f8a58076e20 con 0x7f8a58074310 2024-03-12T17:06:57.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.925+0000 7f284de00640 1 --2- 172.21.15.52:0/4264660140 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f283006ac30 0x7f283006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.926+0000 7f9520400640 1 -- 172.21.15.52:0/2234509420 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f95180d1450 con 0x7f95180d0500 2024-03-12T17:06:57.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.926+0000 7f284d400640 1 --2- 172.21.15.52:0/4264660140 >> [v2:172.21.15.102:6816/37201,v1:172.21.15.102:6817/37201] conn(0x7f2830074270 0x7f28300766b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.926+0000 7fa3cb400640 1 --2- 172.21.15.52:0/1845369396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3cc0ce930 0x7fa3cc15c0f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00c340 tx=0x7fa3bc00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.926+0000 7fa3c3400640 1 -- 172.21.15.52:0/1845369396 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc0052d0 con 0x7fa3cc0ce930 2024-03-12T17:06:57.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.927+0000 7fa3c3400640 1 -- 172.21.15.52:0/1845369396 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3bc005450 con 0x7fa3cc0ce930 2024-03-12T17:06:57.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.927+0000 7fa3c3400640 1 -- 172.21.15.52:0/1845369396 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc007260 con 0x7fa3cc0ce930 2024-03-12T17:06:57.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.928+0000 7fa3d2600640 1 -- 172.21.15.52:0/1845369396 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3cc1481c0 con 0x7fa3cc0ce930 2024-03-12T17:06:57.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.928+0000 7fa3d2600640 1 -- 172.21.15.52:0/1845369396 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3cc158210 con 0x7fa3cc0ce930 2024-03-12T17:06:57.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.929+0000 7f9515600640 1 -- 172.21.15.52:0/2234509420 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9508040ae0 con 0x7f95180d0500 2024-03-12T17:06:57.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.931+0000 7f9515600640 1 --2- 172.21.15.52:0/2234509420 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f94f806ac00 0x7f94f806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.931+0000 7f9515600640 1 -- 172.21.15.52:0/2234509420 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f95080acdb0 con 0x7f95180d0500 2024-03-12T17:06:57.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.931+0000 7f9515600640 1 --2- 172.21.15.52:0/2234509420 >> [v2:172.21.15.102:6800/37191,v1:172.21.15.102:6801/37191] conn(0x7f94f80741c0 0x7f94f8076600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.932+0000 7f9515600640 1 -- 172.21.15.52:0/2234509420 --> [v2:172.21.15.102:6800/37191,v1:172.21.15.102:6801/37191] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f94f8076cd0 con 0x7f94f80741c0 2024-03-12T17:06:57.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.932+0000 7f9515600640 1 -- 172.21.15.52:0/2234509420 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95080ad160 con 0x7f95180d0500 2024-03-12T17:06:57.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.932+0000 7fa3d2600640 1 -- 172.21.15.52:0/1845369396 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa3cc06d010 con 0x7fa3cc0ce930 2024-03-12T17:06:57.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.932+0000 7f951e400640 1 --2- 172.21.15.52:0/2234509420 >> [v2:172.21.15.102:6800/37191,v1:172.21.15.102:6801/37191] conn(0x7f94f80741c0 0x7f94f8076600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.932+0000 7f284de00640 1 --2- 172.21.15.52:0/4264660140 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f283006ac30 0x7f283006d0f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2840036000 tx=0x7f284003b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.933+0000 7f284d400640 1 --2- 172.21.15.52:0/4264660140 >> [v2:172.21.15.102:6816/37201,v1:172.21.15.102:6817/37201] conn(0x7f2830074270 0x7f28300766b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.933+0000 7f951d000640 1 --2- 172.21.15.52:0/2234509420 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f94f806ac00 0x7f94f806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.934+0000 7fa3c3400640 1 -- 172.21.15.52:0/1845369396 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa3bc0073c0 con 0x7fa3cc0ce930 2024-03-12T17:06:57.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.934+0000 7f951e400640 1 --2- 172.21.15.52:0/2234509420 >> [v2:172.21.15.102:6800/37191,v1:172.21.15.102:6801/37191] conn(0x7f94f80741c0 0x7f94f8076600 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.935+0000 7fa3c3400640 1 --2- 172.21.15.52:0/1845369396 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa3ac06ac50 0x7fa3ac06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.935+0000 7fa3c3400640 1 -- 172.21.15.52:0/1845369396 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa3bc0888e0 con 0x7fa3cc0ce930 2024-03-12T17:06:57.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.935+0000 7fa3c3400640 1 --2- 172.21.15.52:0/1845369396 >> [v2:172.21.15.102:6808/37192,v1:172.21.15.102:6809/37192] conn(0x7fa3ac074290 0x7fa3ac0766d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:57.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.935+0000 7fa3c3400640 1 -- 172.21.15.52:0/1845369396 --> [v2:172.21.15.102:6808/37192,v1:172.21.15.102:6809/37192] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa3ac076da0 con 0x7fa3ac074290 2024-03-12T17:06:57.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.935+0000 7fa3c3400640 1 -- 172.21.15.52:0/1845369396 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3bc08d2c0 con 0x7fa3cc0ce930 2024-03-12T17:06:57.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.936+0000 7fa3caa00640 1 --2- 172.21.15.52:0/1845369396 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa3ac06ac50 0x7fa3ac06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.936+0000 7f2846000640 1 -- 172.21.15.52:0/4264660140 <== osd.5 v2:172.21.15.102:6816/37201 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f2830076d80 con 0x7f2830074270 2024-03-12T17:06:57.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.936+0000 7fa3cbe00640 1 --2- 172.21.15.52:0/1845369396 >> [v2:172.21.15.102:6808/37192,v1:172.21.15.102:6809/37192] conn(0x7fa3ac074290 0x7fa3ac0766d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:57.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.938+0000 7f951d000640 1 --2- 172.21.15.52:0/2234509420 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f94f806ac00 0x7f94f806d0c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f95181618a0 tx=0x7f9504009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.938+0000 7fa3cbe00640 1 --2- 172.21.15.52:0/1845369396 >> [v2:172.21.15.102:6808/37192,v1:172.21.15.102:6809/37192] conn(0x7fa3ac074290 0x7fa3ac0766d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.938+0000 7fa3caa00640 1 --2- 172.21.15.52:0/1845369396 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa3ac06ac50 0x7fa3ac06d110 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8036000 tx=0x7fa3b803b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:57.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.939+0000 7f9515600640 1 -- 172.21.15.52:0/2234509420 <== osd.4 v2:172.21.15.102:6800/37191 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f94f8076cd0 con 0x7f94f80741c0 2024-03-12T17:06:57.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.942+0000 7fa3c3400640 1 -- 172.21.15.52:0/1845369396 <== osd.6 v2:172.21.15.102:6808/37192 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fa3ac076da0 con 0x7fa3ac074290 2024-03-12T17:06:57.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.946+0000 7f7de8e00640 1 -- 172.21.15.52:0/278680511 --> [v2:172.21.15.52:6817/37218,v1:172.21.15.52:6819/37218] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7dbc002cb0 con 0x7f7dc40741c0 2024-03-12T17:06:57.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.947+0000 7f7dda000640 1 -- 172.21.15.52:0/278680511 <== osd.1 v2:172.21.15.52:6817/37218 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7dbc002cb0 con 0x7f7dc40741c0 2024-03-12T17:06:57.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.947+0000 7f52c7e00640 1 -- 172.21.15.52:0/3056925370 --> [v2:172.21.15.52:6800/37214,v1:172.21.15.52:6801/37214] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f52c0138d80 con 0x7f52a00741d0 2024-03-12T17:06:57.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.948+0000 7f7de8e00640 1 -- 172.21.15.52:0/278680511 >> [v2:172.21.15.52:6817/37218,v1:172.21.15.52:6819/37218] conn(0x7f7dc40741c0 msgr2=0x7f7dc4076600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.948+0000 7f7de8e00640 1 --2- 172.21.15.52:0/278680511 >> [v2:172.21.15.52:6817/37218,v1:172.21.15.52:6819/37218] conn(0x7f7dc40741c0 0x7f7dc4076600 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.948+0000 7f7de8e00640 1 -- 172.21.15.52:0/278680511 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f7dc406ac00 msgr2=0x7f7dc406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.948+0000 7f7de8e00640 1 --2- 172.21.15.52:0/278680511 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f7dc406ac00 0x7f7dc406d0c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0036000 tx=0x7f7dd0008030 comp rx=0 tx=0).stop 2024-03-12T17:06:57.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.948+0000 7f7de8e00640 1 -- 172.21.15.52:0/278680511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ddc0061a0 msgr2=0x7f7ddc0e74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.948+0000 7f7de8e00640 1 --2- 172.21.15.52:0/278680511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ddc0061a0 0x7f7ddc0e74c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7dd400c410 tx=0x7f7dd400c8d0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.948+0000 7f7de2800640 1 -- 172.21.15.52:0/278680511 reap_dead start 2024-03-12T17:06:57.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.948+0000 7f7de8e00640 1 -- 172.21.15.52:0/278680511 shutdown_connections 2024-03-12T17:06:57.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.948+0000 7f7de8e00640 1 -- 172.21.15.52:0/278680511 >> 172.21.15.52:0/278680511 conn(0x7f7ddc0559e0 msgr2=0x7f7ddc03c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:57.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.949+0000 7f7de8e00640 1 -- 172.21.15.52:0/278680511 shutdown_connections 2024-03-12T17:06:57.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.949+0000 7f52bd600640 1 -- 172.21.15.52:0/3056925370 <== osd.0 v2:172.21.15.52:6800/37214 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f52c0138d80 con 0x7f52a00741d0 2024-03-12T17:06:57.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.949+0000 7f7de8e00640 1 -- 172.21.15.52:0/278680511 wait complete. 2024-03-12T17:06:57.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.951+0000 7f52c7e00640 1 -- 172.21.15.52:0/3056925370 >> [v2:172.21.15.52:6800/37214,v1:172.21.15.52:6801/37214] conn(0x7f52a00741d0 msgr2=0x7f52a0076610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.951+0000 7f52c7e00640 1 --2- 172.21.15.52:0/3056925370 >> [v2:172.21.15.52:6800/37214,v1:172.21.15.52:6801/37214] conn(0x7f52a00741d0 0x7f52a0076610 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.951+0000 7f52c7e00640 1 -- 172.21.15.52:0/3056925370 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f52a006ac90 msgr2=0x7f52a006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.951+0000 7f52c7e00640 1 --2- 172.21.15.52:0/3056925370 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f52a006ac90 0x7f52a006d150 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f52c00cc5f0 tx=0x7f52ac009040 comp rx=0 tx=0).stop 2024-03-12T17:06:57.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.951+0000 7f52c7e00640 1 -- 172.21.15.52:0/3056925370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52c00caf30 msgr2=0x7f52c0159cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.951+0000 7f52c7e00640 1 --2- 172.21.15.52:0/3056925370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52c00caf30 0x7f52c0159cb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f52b403e040 tx=0x7f52b40046c0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.952+0000 7f52c5e00640 1 -- 172.21.15.52:0/3056925370 reap_dead start 2024-03-12T17:06:57.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.951+0000 7f18f1800640 1 -- 172.21.15.52:0/2025597461 --> [v2:172.21.15.52:6816/37217,v1:172.21.15.52:6818/37217] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f18ec0ce300 con 0x7f18cc074210 2024-03-12T17:06:57.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.952+0000 7f52c7e00640 1 -- 172.21.15.52:0/3056925370 shutdown_connections 2024-03-12T17:06:57.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.952+0000 7f52c7e00640 1 -- 172.21.15.52:0/3056925370 >> 172.21.15.52:0/3056925370 conn(0x7f52c0069e70 msgr2=0x7f52c00ae1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:57.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.952+0000 7f52c7e00640 1 -- 172.21.15.52:0/3056925370 shutdown_connections 2024-03-12T17:06:57.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.952+0000 7f18e3e00640 1 -- 172.21.15.52:0/2025597461 <== osd.3 v2:172.21.15.52:6816/37217 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f18e400f040 con 0x7f18cc074210 2024-03-12T17:06:57.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.953+0000 7f52c7e00640 1 -- 172.21.15.52:0/3056925370 wait complete. 2024-03-12T17:06:57.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.953+0000 7f18f1800640 1 -- 172.21.15.52:0/2025597461 >> [v2:172.21.15.52:6816/37217,v1:172.21.15.52:6818/37217] conn(0x7f18cc074210 msgr2=0x7f18cc076650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.953+0000 7f18f1800640 1 --2- 172.21.15.52:0/2025597461 >> [v2:172.21.15.52:6816/37217,v1:172.21.15.52:6818/37217] conn(0x7f18cc074210 0x7f18cc076650 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.954+0000 7f18f1800640 1 -- 172.21.15.52:0/2025597461 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f18cc06ac50 msgr2=0x7f18cc06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.954+0000 7f18f1800640 1 --2- 172.21.15.52:0/2025597461 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f18cc06ac50 0x7f18cc06d110 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f18dc034000 tx=0x7f18dc008030 comp rx=0 tx=0).stop 2024-03-12T17:06:57.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.954+0000 7f18f1800640 1 -- 172.21.15.52:0/2025597461 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18ec0d0640 msgr2=0x7f18ec1f14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.954+0000 7f18f1800640 1 --2- 172.21.15.52:0/2025597461 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18ec0d0640 0x7f18ec1f14b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f18d800c750 tx=0x7f18d800cc10 comp rx=0 tx=0).stop 2024-03-12T17:06:57.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.954+0000 7f18f0e00640 1 -- 172.21.15.52:0/2025597461 reap_dead start 2024-03-12T17:06:57.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.954+0000 7f18f1800640 1 -- 172.21.15.52:0/2025597461 shutdown_connections 2024-03-12T17:06:57.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.954+0000 7f18f1800640 1 -- 172.21.15.52:0/2025597461 >> 172.21.15.52:0/2025597461 conn(0x7f18ec089fd0 msgr2=0x7f18ec06c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:57.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.954+0000 7f18f1800640 1 -- 172.21.15.52:0/2025597461 shutdown_connections 2024-03-12T17:06:57.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.955+0000 7f18f1800640 1 -- 172.21.15.52:0/2025597461 wait complete. 2024-03-12T17:06:57.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.957+0000 7f8a7e200640 1 -- 172.21.15.52:0/2275790810 --> [v2:172.21.15.102:6824/37214,v1:172.21.15.102:6825/37214] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8a70006370 con 0x7f8a58074310 2024-03-12T17:06:57.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.958+0000 7fb604c00640 1 -- 172.21.15.52:0/138744444 --> [v2:172.21.15.52:6808/37216,v1:172.21.15.52:6809/37216] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb5e4002cb0 con 0x7fb5f00741c0 2024-03-12T17:06:57.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.959+0000 7fb607400640 1 -- 172.21.15.52:0/138744444 <== osd.2 v2:172.21.15.52:6808/37216 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb5e4002cb0 con 0x7fb5f00741c0 2024-03-12T17:06:57.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.959+0000 7f8a6b400640 1 -- 172.21.15.52:0/2275790810 <== osd.7 v2:172.21.15.102:6824/37214 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8a6c007040 con 0x7f8a58074310 2024-03-12T17:06:57.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.960+0000 7fb604c00640 1 -- 172.21.15.52:0/138744444 >> [v2:172.21.15.52:6808/37216,v1:172.21.15.52:6809/37216] conn(0x7fb5f00741c0 msgr2=0x7fb5f0076600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.960+0000 7fb604c00640 1 --2- 172.21.15.52:0/138744444 >> [v2:172.21.15.52:6808/37216,v1:172.21.15.52:6809/37216] conn(0x7fb5f00741c0 0x7fb5f0076600 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.960+0000 7f8a7e200640 1 -- 172.21.15.52:0/2275790810 >> [v2:172.21.15.102:6824/37214,v1:172.21.15.102:6825/37214] conn(0x7f8a58074310 msgr2=0x7f8a58076750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.960+0000 7f8a7e200640 1 --2- 172.21.15.52:0/2275790810 >> [v2:172.21.15.102:6824/37214,v1:172.21.15.102:6825/37214] conn(0x7f8a58074310 0x7f8a58076750 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.960+0000 7f8a7e200640 1 -- 172.21.15.52:0/2275790810 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f8a5806acd0 msgr2=0x7f8a5806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.960+0000 7f8a7e200640 1 --2- 172.21.15.52:0/2275790810 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f8a5806acd0 0x7f8a5806d190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8a70006080 tx=0x7f8a64009040 comp rx=0 tx=0).stop 2024-03-12T17:06:57.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.960+0000 7f8a7e200640 1 -- 172.21.15.52:0/2275790810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a7003e800 msgr2=0x7f8a70182b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.960+0000 7f8a7e200640 1 --2- 172.21.15.52:0/2275790810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a7003e800 0x7f8a70182b90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8a7804ede0 tx=0x7f8a78068280 comp rx=0 tx=0).stop 2024-03-12T17:06:57.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.960+0000 7f8a77e00640 1 -- 172.21.15.52:0/2275790810 reap_dead start 2024-03-12T17:06:57.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.960+0000 7f8a7e200640 1 -- 172.21.15.52:0/2275790810 shutdown_connections 2024-03-12T17:06:57.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.960+0000 7f8a7e200640 1 -- 172.21.15.52:0/2275790810 >> 172.21.15.52:0/2275790810 conn(0x7f8a700559a0 msgr2=0x7f8a70052980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:57.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.962+0000 7fb604c00640 1 -- 172.21.15.52:0/138744444 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb5f006ac00 msgr2=0x7fb5f006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.962+0000 7f8a7e200640 1 -- 172.21.15.52:0/2275790810 shutdown_connections 2024-03-12T17:06:57.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.962+0000 7fb604c00640 1 --2- 172.21.15.52:0/138744444 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb5f006ac00 0x7fb5f006d0c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc000f60 tx=0x7fb5fc009040 comp rx=0 tx=0).stop 2024-03-12T17:06:57.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.962+0000 7fb604c00640 1 -- 172.21.15.52:0/138744444 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6100b1a90 msgr2=0x7fb6101546e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.962+0000 7fb604c00640 1 --2- 172.21.15.52:0/138744444 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6100b1a90 0x7fb6101546e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb608033000 tx=0x7fb608004a40 comp rx=0 tx=0).stop 2024-03-12T17:06:57.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.962+0000 7fb60fe00640 1 -- 172.21.15.52:0/138744444 reap_dead start 2024-03-12T17:06:57.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.962+0000 7fb604c00640 1 -- 172.21.15.52:0/138744444 shutdown_connections 2024-03-12T17:06:57.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.962+0000 7fb604c00640 1 -- 172.21.15.52:0/138744444 >> 172.21.15.52:0/138744444 conn(0x7fb610070880 msgr2=0x7fb610070c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:57.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.962+0000 7f8a7e200640 1 -- 172.21.15.52:0/2275790810 wait complete. 2024-03-12T17:06:57.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.963+0000 7fb604c00640 1 -- 172.21.15.52:0/138744444 shutdown_connections 2024-03-12T17:06:57.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.963+0000 7fb604c00640 1 -- 172.21.15.52:0/138744444 wait complete. 2024-03-12T17:06:57.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.963+0000 7f284f200640 1 -- 172.21.15.52:0/4264660140 --> [v2:172.21.15.102:6816/37201,v1:172.21.15.102:6817/37201] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2828002cb0 con 0x7f2830074270 2024-03-12T17:06:57.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.965+0000 7f2846000640 1 -- 172.21.15.52:0/4264660140 <== osd.5 v2:172.21.15.102:6816/37201 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2828002cb0 con 0x7f2830074270 2024-03-12T17:06:57.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.966+0000 7f284f200640 1 -- 172.21.15.52:0/4264660140 >> [v2:172.21.15.102:6816/37201,v1:172.21.15.102:6817/37201] conn(0x7f2830074270 msgr2=0x7f28300766b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.966+0000 7f284f200640 1 --2- 172.21.15.52:0/4264660140 >> [v2:172.21.15.102:6816/37201,v1:172.21.15.102:6817/37201] conn(0x7f2830074270 0x7f28300766b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.968+0000 7f284f200640 1 -- 172.21.15.52:0/4264660140 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f283006ac30 msgr2=0x7f283006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.968+0000 7f284f200640 1 --2- 172.21.15.52:0/4264660140 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f283006ac30 0x7f283006d0f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2840036000 tx=0x7f284003b040 comp rx=0 tx=0).stop 2024-03-12T17:06:57.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.968+0000 7f284f200640 1 -- 172.21.15.52:0/4264660140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2850160ee0 msgr2=0x7f2850070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.968+0000 7f284f200640 1 --2- 172.21.15.52:0/4264660140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2850160ee0 0x7f2850070560 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f284800c710 tx=0x7f284800cbd0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.968+0000 7f284e800640 1 -- 172.21.15.52:0/4264660140 reap_dead start 2024-03-12T17:06:57.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.968+0000 7f284f200640 1 -- 172.21.15.52:0/4264660140 shutdown_connections 2024-03-12T17:06:57.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.968+0000 7f284f200640 1 -- 172.21.15.52:0/4264660140 >> 172.21.15.52:0/4264660140 conn(0x7f2850065e20 msgr2=0x7f2850150110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:57.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.969+0000 7f284f200640 1 -- 172.21.15.52:0/4264660140 shutdown_connections 2024-03-12T17:06:57.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.969+0000 7f284f200640 1 -- 172.21.15.52:0/4264660140 wait complete. 2024-03-12T17:06:57.989 INFO:teuthology.orchestra.run.smithi052.stdout:55834574857 2024-03-12T17:06:57.990 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-12T17:06:57.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.977+0000 7f9520400640 1 -- 172.21.15.52:0/2234509420 --> [v2:172.21.15.102:6800/37191,v1:172.21.15.102:6801/37191] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f95180d1170 con 0x7f94f80741c0 2024-03-12T17:06:57.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.978+0000 7f9515600640 1 -- 172.21.15.52:0/2234509420 <== osd.4 v2:172.21.15.102:6800/37191 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f951000c040 con 0x7f94f80741c0 2024-03-12T17:06:57.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.979+0000 7f9520400640 1 -- 172.21.15.52:0/2234509420 >> [v2:172.21.15.102:6800/37191,v1:172.21.15.102:6801/37191] conn(0x7f94f80741c0 msgr2=0x7f94f8076600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.979+0000 7f9520400640 1 --2- 172.21.15.52:0/2234509420 >> [v2:172.21.15.102:6800/37191,v1:172.21.15.102:6801/37191] conn(0x7f94f80741c0 0x7f94f8076600 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.979+0000 7f9520400640 1 -- 172.21.15.52:0/2234509420 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f94f806ac00 msgr2=0x7f94f806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.979+0000 7f9520400640 1 --2- 172.21.15.52:0/2234509420 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f94f806ac00 0x7f94f806d0c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f95181618a0 tx=0x7f9504009040 comp rx=0 tx=0).stop 2024-03-12T17:06:57.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.979+0000 7f9520400640 1 -- 172.21.15.52:0/2234509420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95180d0500 msgr2=0x7f9518157dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.979+0000 7f9520400640 1 --2- 172.21.15.52:0/2234509420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95180d0500 0x7f9518157dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9508009e80 tx=0x7f9508004a40 comp rx=0 tx=0).stop 2024-03-12T17:06:57.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.979+0000 7f951e400640 1 -- 172.21.15.52:0/2234509420 reap_dead start 2024-03-12T17:06:57.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.978+0000 7fa3d2600640 1 -- 172.21.15.52:0/1845369396 --> [v2:172.21.15.102:6808/37192,v1:172.21.15.102:6809/37192] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa3cc162070 con 0x7fa3ac074290 2024-03-12T17:06:57.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.979+0000 7f9520400640 1 -- 172.21.15.52:0/2234509420 shutdown_connections 2024-03-12T17:06:57.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.979+0000 7f9520400640 1 -- 172.21.15.52:0/2234509420 >> 172.21.15.52:0/2234509420 conn(0x7f9518069c90 msgr2=0x7f95180ce990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:57.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.979+0000 7f9520400640 1 -- 172.21.15.52:0/2234509420 shutdown_connections 2024-03-12T17:06:57.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.980+0000 7f9520400640 1 -- 172.21.15.52:0/2234509420 wait complete. 2024-03-12T17:06:57.993 INFO:teuthology.orchestra.run.smithi052.stdout:55834574857 2024-03-12T17:06:57.994 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-12T17:06:57.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.980+0000 7fa3c3400640 1 -- 172.21.15.52:0/1845369396 <== osd.6 v2:172.21.15.102:6808/37192 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa3ac076da0 con 0x7fa3ac074290 2024-03-12T17:06:57.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.980+0000 7fa3d2600640 1 -- 172.21.15.52:0/1845369396 >> [v2:172.21.15.102:6808/37192,v1:172.21.15.102:6809/37192] conn(0x7fa3ac074290 msgr2=0x7fa3ac0766d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.981+0000 7fa3d2600640 1 --2- 172.21.15.52:0/1845369396 >> [v2:172.21.15.102:6808/37192,v1:172.21.15.102:6809/37192] conn(0x7fa3ac074290 0x7fa3ac0766d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:57.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.981+0000 7fa3d2600640 1 -- 172.21.15.52:0/1845369396 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa3ac06ac50 msgr2=0x7fa3ac06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.981+0000 7fa3d2600640 1 --2- 172.21.15.52:0/1845369396 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa3ac06ac50 0x7fa3ac06d110 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8036000 tx=0x7fa3b803b040 comp rx=0 tx=0).stop 2024-03-12T17:06:57.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.981+0000 7fa3d2600640 1 -- 172.21.15.52:0/1845369396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3cc0ce930 msgr2=0x7fa3cc15c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:57.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.981+0000 7fa3d2600640 1 --2- 172.21.15.52:0/1845369396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3cc0ce930 0x7fa3cc15c0f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00c340 tx=0x7fa3bc00c800 comp rx=0 tx=0).stop 2024-03-12T17:06:57.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.981+0000 7fa3cbe00640 1 -- 172.21.15.52:0/1845369396 reap_dead start 2024-03-12T17:06:57.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.981+0000 7fa3d2600640 1 -- 172.21.15.52:0/1845369396 shutdown_connections 2024-03-12T17:06:57.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.981+0000 7fa3d2600640 1 -- 172.21.15.52:0/1845369396 >> 172.21.15.52:0/1845369396 conn(0x7fa3cc06a0b0 msgr2=0x7fa3cc158d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:57.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.981+0000 7fa3d2600640 1 -- 172.21.15.52:0/1845369396 shutdown_connections 2024-03-12T17:06:57.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:57.981+0000 7fa3d2600640 1 -- 172.21.15.52:0/1845369396 wait complete. 2024-03-12T17:06:57.996 INFO:teuthology.orchestra.run.smithi052.stdout:55834574858 2024-03-12T17:06:57.997 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-12T17:06:57.998 INFO:teuthology.orchestra.run.smithi052.stdout:55834574857 2024-03-12T17:06:57.998 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-12T17:06:57.999 INFO:teuthology.orchestra.run.smithi052.stdout:55834574857 2024-03-12T17:06:57.999 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-12T17:06:58.000 INFO:teuthology.orchestra.run.smithi052.stdout:55834574857 2024-03-12T17:06:58.001 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-12T17:06:58.002 INFO:teuthology.orchestra.run.smithi052.stdout:55834574860 2024-03-12T17:06:58.002 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-12T17:06:58.003 INFO:teuthology.orchestra.run.smithi052.stdout:55834574858 2024-03-12T17:06:58.003 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-12T17:06:58.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.335+0000 7f1273c00640 1 Processor -- start 2024-03-12T17:06:58.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.336+0000 7f1273c00640 1 -- start start 2024-03-12T17:06:58.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.338+0000 7f1273c00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f126c15cda0 0x7f126c0cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.339+0000 7f1271c00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f126c0b4a60 0x7f126c0d1cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:54940/0 (socket says 172.21.15.52:54940) 2024-03-12T17:06:58.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.339+0000 7f126be00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f126c0b5a30 0x7f126c0b4350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.52:35772/0 (socket says 172.21.15.52:35772) 2024-03-12T17:06:58.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.339+0000 7f1271200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f126c15cda0 0x7f126c0cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.340+0000 7f1271200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f126c15cda0 0x7f126c0cdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60902/0 (socket says 172.21.15.52:60902) 2024-03-12T17:06:58.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.340+0000 7f1271c00640 1 -- 172.21.15.52:0/1599760105 learned_addr learned my addr 172.21.15.52:0/1599760105 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:06:58.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.341+0000 7f1273c00640 1 -- 172.21.15.52:0/1599760105 --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f126c0d1830 con 0x7f126c0b4a60 2024-03-12T17:06:58.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.341+0000 7f1273c00640 1 -- 172.21.15.52:0/1599760105 --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f126c0b1f20 con 0x7f126c0b5a30 2024-03-12T17:06:58.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.341+0000 7f126b400640 1 -- 172.21.15.52:0/1599760105 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3022376939 0 0) 0x7f126c0d1830 con 0x7f126c0b4a60 2024-03-12T17:06:58.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.341+0000 7f1273c00640 1 -- 172.21.15.52:0/1599760105 --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f126c0d0c20 con 0x7f126c15cda0 2024-03-12T17:06:58.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.342+0000 7f126b400640 1 -- 172.21.15.52:0/1599760105 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1250003520 con 0x7f126c0b4a60 2024-03-12T17:06:58.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.342+0000 7f126b400640 1 -- 172.21.15.52:0/1599760105 <== mon.1 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 406883821 0 0) 0x7f126c0b1f20 con 0x7f126c0b5a30 2024-03-12T17:06:58.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.342+0000 7f126b400640 1 -- 172.21.15.52:0/1599760105 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f126c0d1830 con 0x7f126c0b5a30 2024-03-12T17:06:58.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.343+0000 7f15c0000640 1 Processor -- start 2024-03-12T17:06:58.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.344+0000 7f15c0000640 1 -- start start 2024-03-12T17:06:58.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.343+0000 7f1271200640 1 -- 172.21.15.52:0/1599760105 >> v1:172.21.15.102:6789/0 conn(0x7f126c0b5a30 legacy=0x7f126c0b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.343+0000 7f1271200640 1 -- 172.21.15.52:0/1599760105 >> v1:172.21.15.52:6789/0 conn(0x7f126c0b4a60 legacy=0x7f126c0d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.343+0000 7f1271200640 1 -- 172.21.15.52:0/1599760105 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f126c0d0f00 con 0x7f126c15cda0 2024-03-12T17:06:58.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.345+0000 7f1271200640 1 --2- 172.21.15.52:0/1599760105 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f126c15cda0 0x7f126c0cdb40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1264009d90 tx=0x7f126402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ebb1ced41f0a5e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:58.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.345+0000 7f126b400640 1 -- 172.21.15.52:0/1599760105 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126402cc60 con 0x7f126c15cda0 2024-03-12T17:06:58.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.345+0000 7f126b400640 1 -- 172.21.15.52:0/1599760105 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f126402cdc0 con 0x7f126c15cda0 2024-03-12T17:06:58.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.346+0000 7f15c0000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f15b80b1bc0 0x7f15b8098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.346+0000 7f15c0000640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f15b80afd20 0x7f15b8098f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.346+0000 7f15c0000640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15b806ada0 con 0x7f15b80cc0e0 2024-03-12T17:06:58.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.346+0000 7f15c0000640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15b80b1fa0 con 0x7f15b80afd20 2024-03-12T17:06:58.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.346+0000 7f15c0000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15b80ccc00 con 0x7f15b80b1bc0 2024-03-12T17:06:58.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.345+0000 7f126b400640 1 -- 172.21.15.52:0/1599760105 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1264034660 con 0x7f126c15cda0 2024-03-12T17:06:58.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.346+0000 7f15bcc00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f15b80afd20 0x7f15b8098f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.347+0000 7f15bcc00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f15b80afd20 0x7f15b8098f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60916/0 (socket says 172.21.15.52:60916) 2024-03-12T17:06:58.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.347+0000 7f15bcc00640 1 -- 172.21.15.52:0/2774017332 learned_addr learned my addr 172.21.15.52:0/2774017332 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:58.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.347+0000 7f15bd600640 1 --2- 172.21.15.52:0/2774017332 >> v2:172.21.15.102:3300/0 conn(0x7f15b80b1bc0 0x7f15b8098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.347+0000 7f15bcc00640 1 -- 172.21.15.52:0/2774017332 >> v2:172.21.15.102:3300/0 conn(0x7f15b80b1bc0 msgr2=0x7f15b8098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:58.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.347+0000 7f15bcc00640 1 --2- 172.21.15.52:0/2774017332 >> v2:172.21.15.102:3300/0 conn(0x7f15b80b1bc0 0x7f15b8098a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.347+0000 7f15bcc00640 1 -- 172.21.15.52:0/2774017332 >> v1:172.21.15.52:6789/0 conn(0x7f15b80cc0e0 legacy=0x7f15b80cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:58.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.347+0000 7f15bcc00640 1 -- 172.21.15.52:0/2774017332 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b806b900 con 0x7f15b80afd20 2024-03-12T17:06:58.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.347+0000 7f15bd600640 1 --2- 172.21.15.52:0/2774017332 >> v2:172.21.15.102:3300/0 conn(0x7f15b80b1bc0 0x7f15b8098a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:06:58.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.348+0000 7f1273c00640 1 -- 172.21.15.52:0/1599760105 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f126c15cda0 msgr2=0x7f126c0cdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.348+0000 7f1273c00640 1 --2- 172.21.15.52:0/1599760105 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f126c15cda0 0x7f126c0cdb40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1264009d90 tx=0x7f126402beb0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.348+0000 7f1273c00640 1 -- 172.21.15.52:0/1599760105 shutdown_connections 2024-03-12T17:06:58.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.348+0000 7f1273c00640 1 --2- 172.21.15.52:0/1599760105 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f126c15cda0 0x7f126c0cdb40 secure :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1264009d90 tx=0x7f126402beb0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.348+0000 7f1273c00640 1 -- 172.21.15.52:0/1599760105 >> 172.21.15.52:0/1599760105 conn(0x7f126c0cf350 msgr2=0x7f126c0cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:58.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.349+0000 7f1273c00640 1 -- 172.21.15.52:0/1599760105 shutdown_connections 2024-03-12T17:06:58.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.349+0000 7f67c8800640 1 Processor -- start 2024-03-12T17:06:58.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.349+0000 7f1273c00640 1 -- 172.21.15.52:0/1599760105 wait complete. 2024-03-12T17:06:58.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.349+0000 7f67c8800640 1 -- start start 2024-03-12T17:06:58.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.350+0000 7f15bcc00640 1 --2- 172.21.15.52:0/2774017332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15b80afd20 0x7f15b8098f80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f15b80ce190 tx=0x7f15b002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ddfe977b7334b94 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:58.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.351+0000 7f15b7e00640 1 -- 172.21.15.52:0/2774017332 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b002ca80 con 0x7f15b80afd20 2024-03-12T17:06:58.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.351+0000 7f15b7e00640 1 -- 172.21.15.52:0/2774017332 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b002cbe0 con 0x7f15b80afd20 2024-03-12T17:06:58.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.351+0000 7f15b7e00640 1 -- 172.21.15.52:0/2774017332 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b00343f0 con 0x7f15b80afd20 2024-03-12T17:06:58.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.352+0000 7f1273c00640 1 Processor -- start 2024-03-12T17:06:58.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.352+0000 7f67c8800640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f67bc04fc40 0x7f67bc052fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.352+0000 7f67c5e00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f67bc0e7630 0x7f67bc04f530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:54958/0 (socket says 172.21.15.52:54958) 2024-03-12T17:06:58.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.352+0000 7f1273c00640 1 -- start start 2024-03-12T17:06:58.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.353+0000 7f15c0000640 1 -- 172.21.15.52:0/2774017332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15b80afd20 msgr2=0x7f15b8098f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.353+0000 7f15c0000640 1 --2- 172.21.15.52:0/2774017332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15b80afd20 0x7f15b8098f80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f15b80ce190 tx=0x7f15b002bee0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.352+0000 7f67c8800640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67bc052b40 con 0x7f67bc0e7630 2024-03-12T17:06:58.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.353+0000 7f67c8800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67bc007260 con 0x7f67bc034220 2024-03-12T17:06:58.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.353+0000 7f67c8800640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67bc007540 con 0x7f67bc04fc40 2024-03-12T17:06:58.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.353+0000 7f67c5e00640 1 -- 172.21.15.52:0/1395910311 learned_addr learned my addr 172.21.15.52:0/1395910311 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:06:58.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.353+0000 7f15c0000640 1 -- 172.21.15.52:0/2774017332 shutdown_connections 2024-03-12T17:06:58.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.353+0000 7f15c0000640 1 --2- 172.21.15.52:0/2774017332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15b80afd20 0x7f15b8098f80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.353+0000 7f15c0000640 1 --2- 172.21.15.52:0/2774017332 >> v2:172.21.15.102:3300/0 conn(0x7f15b80b1bc0 0x7f15b8098a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.353+0000 7f15c0000640 1 -- 172.21.15.52:0/2774017332 >> 172.21.15.52:0/2774017332 conn(0x7f15b80b2140 msgr2=0x7f15b80b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:58.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.354+0000 7f67c4a00640 1 -- 172.21.15.52:0/1395910311 <== mon.1 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 394438227 0 0) 0x7f67bc007260 con 0x7f67bc034220 2024-03-12T17:06:58.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.354+0000 7f67c4a00640 1 -- 172.21.15.52:0/1395910311 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67a8003520 con 0x7f67bc034220 2024-03-12T17:06:58.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.354+0000 7f67c5400640 1 --2- 172.21.15.52:0/1395910311 >> v2:172.21.15.52:3300/0 conn(0x7f67bc04fc40 0x7f67bc052fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.354+0000 7f15c0000640 1 -- 172.21.15.52:0/2774017332 shutdown_connections 2024-03-12T17:06:58.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.354+0000 7f15c0000640 1 -- 172.21.15.52:0/2774017332 wait complete. 2024-03-12T17:06:58.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.354+0000 7f67c4a00640 1 -- 172.21.15.52:0/1395910311 <== mon.1 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3152741119 0 0) 0x7f67a8003520 con 0x7f67bc034220 2024-03-12T17:06:58.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.354+0000 7f67c4a00640 1 -- 172.21.15.52:0/1395910311 >> v2:172.21.15.52:3300/0 conn(0x7f67bc04fc40 msgr2=0x7f67bc052fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:58.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.354+0000 7f67c4a00640 1 --2- 172.21.15.52:0/1395910311 >> v2:172.21.15.52:3300/0 conn(0x7f67bc04fc40 0x7f67bc052fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.354+0000 7f67c4a00640 1 -- 172.21.15.52:0/1395910311 >> v1:172.21.15.52:6789/0 conn(0x7f67bc0e7630 legacy=0x7f67bc04f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.354+0000 7f67c4a00640 1 -- 172.21.15.52:0/1395910311 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67bc007820 con 0x7f67bc034220 2024-03-12T17:06:58.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.354+0000 7f67c4a00640 1 -- 172.21.15.52:0/1395910311 <== mon.1 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f67c004e550 con 0x7f67bc034220 2024-03-12T17:06:58.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.355+0000 7f67c4a00640 1 -- 172.21.15.52:0/1395910311 >> v1:172.21.15.102:6789/0 conn(0x7f67bc034220 legacy=0x7f67bc053510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.355+0000 7f67c4a00640 1 --2- 172.21.15.52:0/1395910311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67a8003970 0x7f67a8003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.355+0000 7f67c4a00640 1 --2- 172.21.15.52:0/1395910311 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67a8004370 0x7f67a8004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.355+0000 7f67c4a00640 1 --2- 172.21.15.52:0/1395910311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67a8005090 0x7f67a800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.355+0000 7f67c4a00640 1 -- 172.21.15.52:0/1395910311 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc007540 con 0x7f67a8005090 2024-03-12T17:06:58.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.355+0000 7f67c4a00640 1 -- 172.21.15.52:0/1395910311 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a800bfd0 con 0x7f67a8003970 2024-03-12T17:06:58.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.355+0000 7f67c4a00640 1 -- 172.21.15.52:0/1395910311 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a800c190 con 0x7f67a8004370 2024-03-12T17:06:58.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.355+0000 7f1273c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f126c0b4a60 0x7f126c15a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.355+0000 7f67c6800640 1 --2- 172.21.15.52:0/1395910311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67a8005090 0x7f67a800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.356+0000 7f67c6800640 1 -- 172.21.15.52:0/1395910311 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67a8004370 msgr2=0x7f67a8004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:58.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.356+0000 7f67c6800640 1 --2- 172.21.15.52:0/1395910311 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67a8004370 0x7f67a8004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.356+0000 7f67c6800640 1 -- 172.21.15.52:0/1395910311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67a8003970 msgr2=0x7f67a8003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:58.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.356+0000 7f67c6800640 1 --2- 172.21.15.52:0/1395910311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67a8003970 0x7f67a8003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.356+0000 7f67c6800640 1 -- 172.21.15.52:0/1395910311 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67bc007820 con 0x7f67a8005090 2024-03-12T17:06:58.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.356+0000 7f1273c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f126c0b5a30 0x7f126c15ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.356+0000 7f1273c00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f126c15df40 0x7f126c15f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.356+0000 7f1273c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126c15a7e0 con 0x7f126c0b4a60 2024-03-12T17:06:58.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.356+0000 7f1273c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126c15d210 con 0x7f126c0b5a30 2024-03-12T17:06:58.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.356+0000 7f1273c00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f126c15f9d0 con 0x7f126c15df40 2024-03-12T17:06:58.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.356+0000 7f1271200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f126c0b4a60 0x7f126c15a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.356+0000 7f1271c00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f126c15df40 0x7f126c15f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.357+0000 7f1271c00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f126c15df40 0x7f126c15f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.52:55740/0 (socket says 172.21.15.52:55740) 2024-03-12T17:06:58.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.357+0000 7f1271c00640 1 -- 172.21.15.52:0/1755954139 learned_addr learned my addr 172.21.15.52:0/1755954139 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:58.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.357+0000 7f15c0000640 1 Processor -- start 2024-03-12T17:06:58.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.357+0000 7f126be00640 1 --2- 172.21.15.52:0/1755954139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f126c0b5a30 0x7f126c15ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.357+0000 7f15c0000640 1 -- start start 2024-03-12T17:06:58.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.358+0000 7f24bca00640 1 Processor -- start 2024-03-12T17:06:58.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.358+0000 7f67c6800640 1 -- 172.21.15.52:0/1395910311 reap_dead start 2024-03-12T17:06:58.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.358+0000 7f67c6800640 1 --2- 172.21.15.52:0/1395910311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67a8005090 0x7f67a800b980 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f67c004f930 tx=0x7f67a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c48e89edc1ef667 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:58.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.358+0000 7f15c0000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15b80afd20 0x7f15b80c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.358+0000 7f15c0000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15b80b1bc0 0x7f15b80cce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.358+0000 7f15c0000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f15b80cc0e0 0x7f15b80b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.358+0000 7f15c0000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b80cd890 con 0x7f15b80b1bc0 2024-03-12T17:06:58.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.358+0000 7f15c0000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b80cdb70 con 0x7f15b80afd20 2024-03-12T17:06:58.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.358+0000 7f15c0000640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b80cde50 con 0x7f15b80cc0e0 2024-03-12T17:06:58.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.359+0000 7f15bcc00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15b80b1bc0 0x7f15b80cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.359+0000 7f15bcc00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15b80b1bc0 0x7f15b80cce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60944/0 (socket says 172.21.15.52:60944) 2024-03-12T17:06:58.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.359+0000 7f15bd600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15b80afd20 0x7f15b80c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.359+0000 7f15bcc00640 1 -- 172.21.15.52:0/3087183359 learned_addr learned my addr 172.21.15.52:0/3087183359 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:58.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.359+0000 7f15be000640 1 --2- 172.21.15.52:0/3087183359 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f15b80cc0e0 0x7f15b80b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.359+0000 7f1271c00640 1 -- 172.21.15.52:0/1755954139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f126c0b5a30 msgr2=0x7f126c15ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.359+0000 7f1271c00640 1 --2- 172.21.15.52:0/1755954139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f126c0b5a30 0x7f126c15ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.359+0000 7f1271c00640 1 -- 172.21.15.52:0/1755954139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f126c0b4a60 msgr2=0x7f126c15a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.359+0000 7f67c4a00640 1 -- 172.21.15.52:0/1395910311 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c00a6b10 con 0x7f67a8005090 2024-03-12T17:06:58.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.359+0000 7f67c4a00640 1 -- 172.21.15.52:0/1395910311 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c00a6c70 con 0x7f67a8005090 2024-03-12T17:06:58.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.360+0000 7f15bcc00640 1 -- 172.21.15.52:0/3087183359 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f15b80cc0e0 msgr2=0x7f15b80b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.358+0000 7f24bca00640 1 -- start start 2024-03-12T17:06:58.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.360+0000 7f15bcc00640 1 --2- 172.21.15.52:0/3087183359 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f15b80cc0e0 0x7f15b80b3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.360+0000 7f15bcc00640 1 -- 172.21.15.52:0/3087183359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15b80afd20 msgr2=0x7f15b80c9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.360+0000 7f15bcc00640 1 --2- 172.21.15.52:0/3087183359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15b80afd20 0x7f15b80c9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.360+0000 7f15bcc00640 1 -- 172.21.15.52:0/3087183359 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b0009b30 con 0x7f15b80b1bc0 2024-03-12T17:06:58.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.360+0000 7f67c4a00640 1 -- 172.21.15.52:0/1395910311 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c00ae440 con 0x7f67a8005090 2024-03-12T17:06:58.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.360+0000 7f1271c00640 1 --2- 172.21.15.52:0/1755954139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f126c0b4a60 0x7f126c15a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.360+0000 7f15bcc00640 1 --2- 172.21.15.52:0/3087183359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15b80b1bc0 0x7f15b80cce40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f15b0000f80 tx=0x7f15b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:58.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.360+0000 7f1271c00640 1 -- 172.21.15.52:0/1755954139 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1264009a70 con 0x7f126c15df40 2024-03-12T17:06:58.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.361+0000 7f24bca00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f24b814aa40 0x7f24b80cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.361+0000 7f15b5600640 1 -- 172.21.15.52:0/3087183359 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b0009510 con 0x7f15b80b1bc0 2024-03-12T17:06:58.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.361+0000 7f15b5600640 1 -- 172.21.15.52:0/3087183359 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b003e450 con 0x7f15b80b1bc0 2024-03-12T17:06:58.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.361+0000 7f15b5600640 1 -- 172.21.15.52:0/3087183359 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b003e800 con 0x7f15b80b1bc0 2024-03-12T17:06:58.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.361+0000 7f15c0000640 1 -- 172.21.15.52:0/3087183359 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15b80ce130 con 0x7f15b80b1bc0 2024-03-12T17:06:58.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.361+0000 7f126be00640 1 --2- 172.21.15.52:0/1755954139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f126c0b5a30 0x7f126c15ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:06:58.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.361+0000 7f24bca00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f24b80cc7b0 0x7f24b80ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.361+0000 7f15c0000640 1 -- 172.21.15.52:0/3087183359 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15b80ca450 con 0x7f15b80b1bc0 2024-03-12T17:06:58.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.361+0000 7f24b5e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f24b814aa40 0x7f24b80cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.362+0000 7f24b5e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f24b814aa40 0x7f24b80cd7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60948/0 (socket says 172.21.15.52:60948) 2024-03-12T17:06:58.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.362+0000 7f24b5e00640 1 -- 172.21.15.52:0/4276205162 learned_addr learned my addr 172.21.15.52:0/4276205162 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:58.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.362+0000 7f24bca00640 1 -- 172.21.15.52:0/4276205162 --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24b806da70 con 0x7f24b80b27e0 2024-03-12T17:06:58.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.362+0000 7f24bca00640 1 -- 172.21.15.52:0/4276205162 --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24b80b1d00 con 0x7f24b814aa40 2024-03-12T17:06:58.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.362+0000 7f67c8800640 1 -- 172.21.15.52:0/1395910311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67a8005090 msgr2=0x7f67a800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.362+0000 7f67c8800640 1 --2- 172.21.15.52:0/1395910311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67a8005090 0x7f67a800b980 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f67c004f930 tx=0x7f67a80057f0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.362+0000 7f24bca00640 1 -- 172.21.15.52:0/4276205162 --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24b80b1fe0 con 0x7f24b80cc7b0 2024-03-12T17:06:58.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.361+0000 7f1271200640 1 --2- 172.21.15.52:0/1755954139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f126c0b4a60 0x7f126c15a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T17:06:58.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.363+0000 7f24b5400640 1 --2- 172.21.15.52:0/4276205162 >> v2:172.21.15.102:3300/0 conn(0x7f24b80cc7b0 0x7f24b80ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.363+0000 7f67c8800640 1 -- 172.21.15.52:0/1395910311 shutdown_connections 2024-03-12T17:06:58.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.363+0000 7f67c8800640 1 --2- 172.21.15.52:0/1395910311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67a8005090 0x7f67a800b980 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.363+0000 7f67c8800640 1 -- 172.21.15.52:0/1395910311 >> 172.21.15.52:0/1395910311 conn(0x7f67bc054ee0 msgr2=0x7f67bc0552f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:58.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.363+0000 7f1271c00640 1 --2- 172.21.15.52:0/1755954139 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f126c15df40 0x7f126c15f3a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f125800c3e0 tx=0x7f125800c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:58.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.364+0000 7f67c8800640 1 -- 172.21.15.52:0/1395910311 shutdown_connections 2024-03-12T17:06:58.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.364+0000 7f67c8800640 1 -- 172.21.15.52:0/1395910311 wait complete. 2024-03-12T17:06:58.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.364+0000 7f24b5e00640 1 -- 172.21.15.52:0/4276205162 >> v2:172.21.15.102:3300/0 conn(0x7f24b80cc7b0 msgr2=0x7f24b80ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:58.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.364+0000 7f24b5e00640 1 --2- 172.21.15.52:0/4276205162 >> v2:172.21.15.102:3300/0 conn(0x7f24b80cc7b0 0x7f24b80ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.364+0000 7f24b5e00640 1 -- 172.21.15.52:0/4276205162 >> v1:172.21.15.102:6789/0 conn(0x7f24b80b27e0 legacy=0x7f24b80b2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:58.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.364+0000 7f24b5e00640 1 -- 172.21.15.52:0/4276205162 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b80b22c0 con 0x7f24b814aa40 2024-03-12T17:06:58.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.364+0000 7f15b5600640 1 -- 172.21.15.52:0/3087183359 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f15b002cd50 con 0x7f15b80b1bc0 2024-03-12T17:06:58.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.364+0000 7f15c0000640 1 -- 172.21.15.52:0/3087183359 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1594005250 con 0x7f15b80b1bc0 2024-03-12T17:06:58.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.365+0000 7f1268c00640 1 -- 172.21.15.52:0/1755954139 <== mon.2 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12580074e0 con 0x7f126c15df40 2024-03-12T17:06:58.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.365+0000 7f1268c00640 1 -- 172.21.15.52:0/1755954139 <== mon.2 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1258007660 con 0x7f126c15df40 2024-03-12T17:06:58.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.365+0000 7f1268c00640 1 -- 172.21.15.52:0/1755954139 <== mon.2 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1258004a20 con 0x7f126c15df40 2024-03-12T17:06:58.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.365+0000 7f15b5600640 1 --2- 172.21.15.52:0/3087183359 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f159806ac80 0x7f159806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.365+0000 7f15b5600640 1 -- 172.21.15.52:0/3087183359 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f15b00abfc0 con 0x7f15b80b1bc0 2024-03-12T17:06:58.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.365+0000 7f1273c00640 1 -- 172.21.15.52:0/1755954139 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f126c15d460 con 0x7f126c15df40 2024-03-12T17:06:58.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.365+0000 7f1273c00640 1 -- 172.21.15.52:0/1755954139 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f126c15da20 con 0x7f126c15df40 2024-03-12T17:06:58.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.366+0000 7f15bd600640 1 --2- 172.21.15.52:0/3087183359 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f159806ac80 0x7f159806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.367+0000 7f67c8800640 1 Processor -- start 2024-03-12T17:06:58.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.368+0000 7f67c8800640 1 -- start start 2024-03-12T17:06:58.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.368+0000 7f15bd600640 1 --2- 172.21.15.52:0/3087183359 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f159806ac80 0x7f159806d140 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f15ac006fd0 tx=0x7f15ac008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:58.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.368+0000 7f0778400640 1 Processor -- start 2024-03-12T17:06:58.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.368+0000 7f24b5e00640 1 --2- 172.21.15.52:0/4276205162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24b814aa40 0x7f24b80cd7a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f24b80b3ea0 tx=0x7f24a802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4feda555b64de3cc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:58.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.369+0000 7f67c8800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67a8005090 0x7f67bc04e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.369+0000 7f67c8800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67bc0e43e0 0x7f67bc04a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.369+0000 7f67c8800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67bc04a9b0 0x7f67bc04d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.369+0000 7f67c8800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc04ae30 con 0x7f67bc04a9b0 2024-03-12T17:06:58.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.369+0000 7f67c8800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc04ed90 con 0x7f67a8005090 2024-03-12T17:06:58.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.369+0000 7f67c8800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc039f50 con 0x7f67bc0e43e0 2024-03-12T17:06:58.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.369+0000 7f1273c00640 1 -- 172.21.15.52:0/1755954139 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f126c073390 con 0x7f126c15df40 2024-03-12T17:06:58.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.370+0000 7fa657000640 1 Processor -- start 2024-03-12T17:06:58.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.370+0000 7f67c6800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67bc04a9b0 0x7f67bc04d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.370+0000 7f67c6800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67bc04a9b0 0x7f67bc04d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60962/0 (socket says 172.21.15.52:60962) 2024-03-12T17:06:58.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.370+0000 7f67c6800640 1 -- 172.21.15.52:0/1868372084 learned_addr learned my addr 172.21.15.52:0/1868372084 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:58.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.370+0000 7fa657000640 1 -- start start 2024-03-12T17:06:58.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.371+0000 7f67c6800640 1 -- 172.21.15.52:0/1868372084 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67bc0e43e0 msgr2=0x7f67bc04a470 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-12T17:06:58.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.371+0000 7f67c6800640 1 --2- 172.21.15.52:0/1868372084 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67bc0e43e0 0x7f67bc04a470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.369+0000 7f0778400640 1 -- start start 2024-03-12T17:06:58.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.371+0000 7f0778400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f077014aa40 0x7f07700cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.371+0000 7f0778400640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077006da70 con 0x7f07700b37f0 2024-03-12T17:06:58.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.371+0000 7f0778400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07700b1d00 con 0x7f07700cc7b0 2024-03-12T17:06:58.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.371+0000 7f0778400640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07700b1fe0 con 0x7f077014aa40 2024-03-12T17:06:58.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.371+0000 7f67c6800640 1 -- 172.21.15.52:0/1868372084 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67a8005090 msgr2=0x7f67bc04e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.371+0000 7f67c6800640 1 --2- 172.21.15.52:0/1868372084 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67a8005090 0x7f67bc04e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.371+0000 7f67c6800640 1 -- 172.21.15.52:0/1868372084 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67bc038eb0 con 0x7f67bc04a9b0 2024-03-12T17:06:58.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.371+0000 7f67c6800640 1 --2- 172.21.15.52:0/1868372084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67bc04a9b0 0x7f67bc04d000 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f67c002c880 tx=0x7f67c0068280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:58.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.371+0000 7f0776400640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f07700b37f0 0x7f077006c280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:54960/0 (socket says 172.21.15.52:54960) 2024-03-12T17:06:58.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.371+0000 7f0776400640 1 -- 172.21.15.52:0/3813215604 learned_addr learned my addr 172.21.15.52:0/3813215604 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:06:58.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.372+0000 7f0775a00640 1 --2- 172.21.15.52:0/3813215604 >> v2:172.21.15.52:3300/0 conn(0x7f077014aa40 0x7f07700cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.372+0000 7fa657000640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa65015ccb0 0x7fa6500d0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.372+0000 7fa657000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa6500d2290 0x7fa6500cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.372+0000 7fa657000640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa65015d090 con 0x7fa6500b4390 2024-03-12T17:06:58.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.372+0000 7fa657000640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6500ce130 con 0x7fa65015ccb0 2024-03-12T17:06:58.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.372+0000 7fa657000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6500d11f0 con 0x7fa6500d2290 2024-03-12T17:06:58.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.372+0000 7f1268c00640 1 -- 172.21.15.52:0/1755954139 <== mon.2 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1258004b80 con 0x7f126c15df40 2024-03-12T17:06:58.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.372+0000 7fa64fe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa65015ccb0 0x7fa6500d0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.372+0000 7fa655000640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fa6500b4390 0x7fa6500b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:54968/0 (socket says 172.21.15.52:54968) 2024-03-12T17:06:58.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.372+0000 7fa64fe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa65015ccb0 0x7fa6500d0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60986/0 (socket says 172.21.15.52:60986) 2024-03-12T17:06:58.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.372+0000 7fa655000640 1 -- 172.21.15.52:0/3937820777 learned_addr learned my addr 172.21.15.52:0/3937820777 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:06:58.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.372+0000 7f38b6800640 1 Processor -- start 2024-03-12T17:06:58.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.372+0000 7fa64f400640 1 --2- 172.21.15.52:0/3937820777 >> v2:172.21.15.102:3300/0 conn(0x7fa6500d2290 0x7fa6500cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.373+0000 7f0775a00640 1 -- 172.21.15.52:0/3813215604 >> v1:172.21.15.102:6789/0 conn(0x7f07700cc7b0 legacy=0x7f07700ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.373+0000 7f0775a00640 1 -- 172.21.15.52:0/3813215604 >> v1:172.21.15.52:6789/0 conn(0x7f07700b37f0 legacy=0x7f077006c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.373+0000 7fa64fe00640 1 -- 172.21.15.52:0/3937820777 >> v2:172.21.15.102:3300/0 conn(0x7fa6500d2290 msgr2=0x7fa6500cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:58.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.373+0000 7fa64fe00640 1 --2- 172.21.15.52:0/3937820777 >> v2:172.21.15.102:3300/0 conn(0x7fa6500d2290 0x7fa6500cdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.373+0000 7fa64fe00640 1 -- 172.21.15.52:0/3937820777 >> v1:172.21.15.52:6789/0 conn(0x7fa6500b4390 legacy=0x7fa6500b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.373+0000 7fa64fe00640 1 -- 172.21.15.52:0/3937820777 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6500d14d0 con 0x7fa65015ccb0 2024-03-12T17:06:58.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.373+0000 7f0775a00640 1 -- 172.21.15.52:0/3813215604 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07700b22c0 con 0x7f077014aa40 2024-03-12T17:06:58.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.373+0000 7f076fe00640 1 -- 172.21.15.52:0/3813215604 <== mon.1 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 56053289 0 0) 0x7f07700b1d00 con 0x7f07700cc7b0 2024-03-12T17:06:58.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.373+0000 7f38b6800640 1 -- start start 2024-03-12T17:06:58.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.374+0000 7f1268c00640 1 --2- 172.21.15.52:0/1755954139 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f124c06ac00 0x7f124c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.374+0000 7f1268c00640 1 -- 172.21.15.52:0/1755954139 <== mon.2 v2:172.21.15.102:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1258089480 con 0x7f126c15df40 2024-03-12T17:06:58.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.374+0000 7fa64fe00640 1 --2- 172.21.15.52:0/3937820777 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa65015ccb0 0x7fa6500d0af0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa6500d0010 tx=0x7fa63c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705513bf01b42dae server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:58.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.375+0000 7f0775a00640 1 --2- 172.21.15.52:0/3813215604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f077014aa40 0x7f07700cd7a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f075c0097a0 tx=0x7f075c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f31ed10f0e9e1c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:58.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.375+0000 7f1271200640 1 --2- 172.21.15.52:0/1755954139 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f124c06ac00 0x7f124c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.376+0000 7f38b6800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f38b00b3f30 0x7f38b00cd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.376+0000 7fa64ea00640 1 -- 172.21.15.52:0/3937820777 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c030840 con 0x7fa65015ccb0 2024-03-12T17:06:58.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.376+0000 7f67ba000640 1 -- 172.21.15.52:0/1868372084 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c004e550 con 0x7f67bc04a9b0 2024-03-12T17:06:58.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.376+0000 7f38b6800640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b00cb6a0 con 0x7f38b0159c20 2024-03-12T17:06:58.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.376+0000 7f38b6800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b015a0a0 con 0x7f38b00b2250 2024-03-12T17:06:58.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.376+0000 7f38b6800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b00cb0e0 con 0x7f38b00b3f30 2024-03-12T17:06:58.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.376+0000 7f24b4a00640 1 -- 172.21.15.52:0/4276205162 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a8031870 con 0x7f24b814aa40 2024-03-12T17:06:58.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.376+0000 7f67ba000640 1 -- 172.21.15.52:0/1868372084 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c0060700 con 0x7f67bc04a9b0 2024-03-12T17:06:58.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.376+0000 7f67ba000640 1 -- 172.21.15.52:0/1868372084 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c0060a70 con 0x7f67bc04a9b0 2024-03-12T17:06:58.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.376+0000 7f67c8800640 1 -- 172.21.15.52:0/1868372084 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67bc039190 con 0x7f67bc04a9b0 2024-03-12T17:06:58.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.376+0000 7f076fe00640 1 -- 172.21.15.52:0/3813215604 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c031830 con 0x7f077014aa40 2024-03-12T17:06:58.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.376+0000 7f076fe00640 1 -- 172.21.15.52:0/3813215604 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f075c031990 con 0x7f077014aa40 2024-03-12T17:06:58.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.376+0000 7f076fe00640 1 -- 172.21.15.52:0/3813215604 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c031d20 con 0x7f077014aa40 2024-03-12T17:06:58.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.376+0000 7fa64ea00640 1 -- 172.21.15.52:0/3937820777 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa63c0309a0 con 0x7fa65015ccb0 2024-03-12T17:06:58.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.376+0000 7fa64ea00640 1 -- 172.21.15.52:0/3937820777 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c030d70 con 0x7fa65015ccb0 2024-03-12T17:06:58.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.376+0000 7f1271200640 1 --2- 172.21.15.52:0/1755954139 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f124c06ac00 0x7f124c06d0c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1264036000 tx=0x7f1264030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:58.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.376+0000 7f67c8800640 1 -- 172.21.15.52:0/1868372084 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67bc039750 con 0x7f67bc04a9b0 2024-03-12T17:06:58.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.376+0000 7f38aea00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f38b0159c20 0x7f38b00ccb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:54982/0 (socket says 172.21.15.52:54982) 2024-03-12T17:06:58.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.376+0000 7f38afe00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f38b00b3f30 0x7f38b00cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.377+0000 7f38afe00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f38b00b3f30 0x7f38b00cd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.52:55764/0 (socket says 172.21.15.52:55764) 2024-03-12T17:06:58.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.377+0000 7f38afe00640 1 -- 172.21.15.52:0/1054639472 learned_addr learned my addr 172.21.15.52:0/1054639472 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:58.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.377+0000 7fa657000640 1 -- 172.21.15.52:0/3937820777 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa65015ccb0 msgr2=0x7fa6500d0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.377+0000 7fa657000640 1 --2- 172.21.15.52:0/3937820777 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa65015ccb0 0x7fa6500d0af0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa6500d0010 tx=0x7fa63c02b980 comp rx=0 tx=0).stop 2024-03-12T17:06:58.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.376+0000 7f5f00400640 1 Processor -- start 2024-03-12T17:06:58.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.376+0000 7f5f00400640 1 -- start start 2024-03-12T17:06:58.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.376+0000 7f24b4a00640 1 -- 172.21.15.52:0/4276205162 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a80319d0 con 0x7f24b814aa40 2024-03-12T17:06:58.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.376+0000 7f24b4a00640 1 -- 172.21.15.52:0/4276205162 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a8031d60 con 0x7f24b814aa40 2024-03-12T17:06:58.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.377+0000 7fa657000640 1 -- 172.21.15.52:0/3937820777 shutdown_connections 2024-03-12T17:06:58.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.377+0000 7fa657000640 1 --2- 172.21.15.52:0/3937820777 >> v2:172.21.15.102:3300/0 conn(0x7fa6500d2290 0x7fa6500cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.377+0000 7fa657000640 1 --2- 172.21.15.52:0/3937820777 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa65015ccb0 0x7fa6500d0af0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.377+0000 7fa657000640 1 -- 172.21.15.52:0/3937820777 >> 172.21.15.52:0/3937820777 conn(0x7fa6500717b0 msgr2=0x7fa650071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:58.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.377+0000 7f15b5600640 1 -- 172.21.15.52:0/3087183359 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f15b0076fd0 con 0x7f15b80b1bc0 2024-03-12T17:06:58.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.378+0000 7fa657000640 1 -- 172.21.15.52:0/3937820777 shutdown_connections 2024-03-12T17:06:58.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.378+0000 7f38afe00640 1 -- 172.21.15.52:0/1054639472 >> v1:172.21.15.102:6789/0 conn(0x7f38b00b2250 legacy=0x7f38b00b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.378+0000 7fa657000640 1 -- 172.21.15.52:0/3937820777 wait complete. 2024-03-12T17:06:58.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.378+0000 7f67c8800640 1 -- 172.21.15.52:0/1868372084 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67bc008df0 con 0x7f67bc04a9b0 2024-03-12T17:06:58.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.378+0000 7f0778400640 1 -- 172.21.15.52:0/3813215604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f077014aa40 msgr2=0x7f07700cd7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.378+0000 7f0778400640 1 --2- 172.21.15.52:0/3813215604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f077014aa40 0x7f07700cd7a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f075c0097a0 tx=0x7f075c02b8d0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.378+0000 7f5f00400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ef80cb250 0x7f5ef8098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.379+0000 7f5f00400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5ef8098f80 0x7f5ef80b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.379+0000 7f5f00400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ef80cb6b0 con 0x7f5ef815a950 2024-03-12T17:06:58.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.379+0000 7f5f00400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ef8099b40 con 0x7f5ef8098f80 2024-03-12T17:06:58.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.379+0000 7f5f00400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef80993f0 con 0x7f5ef80cb250 2024-03-12T17:06:58.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.379+0000 7f0778400640 1 -- 172.21.15.52:0/3813215604 shutdown_connections 2024-03-12T17:06:58.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.379+0000 7f0778400640 1 --2- 172.21.15.52:0/3813215604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f077014aa40 0x7f07700cd7a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.379+0000 7f0778400640 1 -- 172.21.15.52:0/3813215604 >> 172.21.15.52:0/3813215604 conn(0x7f077006a3d0 msgr2=0x7f07700b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:58.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.378+0000 7f24bca00640 1 -- 172.21.15.52:0/4276205162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24b814aa40 msgr2=0x7f24b80cd7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.378+0000 7f24bca00640 1 --2- 172.21.15.52:0/4276205162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24b814aa40 0x7f24b80cd7a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f24b80b3ea0 tx=0x7f24a802b910 comp rx=0 tx=0).stop 2024-03-12T17:06:58.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.379+0000 7f5efda00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f5ef815a950 0x7f5ef80cf140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.52:35842/0 (socket says 172.21.15.52:35842) 2024-03-12T17:06:58.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.378+0000 7f38afe00640 1 -- 172.21.15.52:0/1054639472 >> v1:172.21.15.52:6789/0 conn(0x7f38b0159c20 legacy=0x7f38b00ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.378+0000 7f38afe00640 1 -- 172.21.15.52:0/1054639472 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b00cb3c0 con 0x7f38b00b3f30 2024-03-12T17:06:58.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.379+0000 7f5efe400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5ef8098f80 0x7f5ef80b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.379+0000 7f5efd000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ef80cb250 0x7f5ef8098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.379+0000 7f5efe400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5ef8098f80 0x7f5ef80b13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:38702/0 (socket says 172.21.15.52:38702) 2024-03-12T17:06:58.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.379+0000 7f5efd000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ef80cb250 0x7f5ef8098a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.52:55772/0 (socket says 172.21.15.52:55772) 2024-03-12T17:06:58.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.380+0000 7f5efda00640 1 -- 172.21.15.52:0/3882772499 learned_addr learned my addr 172.21.15.52:0/3882772499 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:06:58.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.380+0000 7f38afe00640 1 --2- 172.21.15.52:0/1054639472 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f38b00b3f30 0x7f38b00cd210 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009e80 tx=0x7f38a802be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=212eecf49ad389d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:58.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.380+0000 7fa657000640 1 Processor -- start 2024-03-12T17:06:58.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.380+0000 7f5efd000640 1 -- 172.21.15.52:0/3882772499 >> v2:172.21.15.102:3300/0 conn(0x7f5ef8098f80 msgr2=0x7f5ef80b13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:06:58.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.380+0000 7f5efd000640 1 --2- 172.21.15.52:0/3882772499 >> v2:172.21.15.102:3300/0 conn(0x7f5ef8098f80 0x7f5ef80b13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.380+0000 7f5efd000640 1 -- 172.21.15.52:0/3882772499 >> v1:172.21.15.102:6789/0 conn(0x7f5ef815a950 legacy=0x7f5ef80cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.380+0000 7f5efd000640 1 -- 172.21.15.52:0/3882772499 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef815ad30 con 0x7f5ef80cb250 2024-03-12T17:06:58.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.381+0000 7f5efe400640 1 --2- 172.21.15.52:0/3882772499 >> v2:172.21.15.102:3300/0 conn(0x7f5ef8098f80 0x7f5ef80b13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:06:58.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.381+0000 7f0778400640 1 -- 172.21.15.52:0/3813215604 shutdown_connections 2024-03-12T17:06:58.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.381+0000 7f0778400640 1 -- 172.21.15.52:0/3813215604 wait complete. 2024-03-12T17:06:58.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.381+0000 7f67ba000640 1 -- 172.21.15.52:0/1868372084 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f67c0060bd0 con 0x7f67bc04a9b0 2024-03-12T17:06:58.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.382+0000 7f5efd000640 1 --2- 172.21.15.52:0/3882772499 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ef80cb250 0x7f5ef8098a40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0097a0 tx=0x7f5eec02b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a025cafba1d3735 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:58.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.381+0000 7fa657000640 1 -- start start 2024-03-12T17:06:58.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.382+0000 7fa657000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6500b4390 0x7fa65014f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.382+0000 7fa657000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa6500d2290 0x7fa6500caf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.382+0000 7fa657000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa65015ccb0 0x7fa6500b75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.382+0000 7fa657000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65015f1e0 con 0x7fa6500b4390 2024-03-12T17:06:58.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.380+0000 7f24bca00640 1 -- 172.21.15.52:0/4276205162 shutdown_connections 2024-03-12T17:06:58.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.380+0000 7f24bca00640 1 --2- 172.21.15.52:0/4276205162 >> v2:172.21.15.102:3300/0 conn(0x7f24b80cc7b0 0x7f24b80ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.382+0000 7fa657000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65015e8d0 con 0x7fa65015ccb0 2024-03-12T17:06:58.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.382+0000 7fa657000640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa65015e100 con 0x7fa6500d2290 2024-03-12T17:06:58.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.380+0000 7f24bca00640 1 --2- 172.21.15.52:0/4276205162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24b814aa40 0x7f24b80cd7a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.380+0000 7f24bca00640 1 -- 172.21.15.52:0/4276205162 >> 172.21.15.52:0/4276205162 conn(0x7f24b806a3d0 msgr2=0x7f24b80b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:58.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.380+0000 7f24bca00640 1 -- 172.21.15.52:0/4276205162 shutdown_connections 2024-03-12T17:06:58.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.380+0000 7f24bca00640 1 -- 172.21.15.52:0/4276205162 wait complete. 2024-03-12T17:06:58.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.383+0000 7fa64fe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6500b4390 0x7fa65014f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.383+0000 7fa64fe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6500b4390 0x7fa65014f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60990/0 (socket says 172.21.15.52:60990) 2024-03-12T17:06:58.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.383+0000 7fa64fe00640 1 -- 172.21.15.52:0/4046110931 learned_addr learned my addr 172.21.15.52:0/4046110931 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:58.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.383+0000 7fa64f400640 1 --2- 172.21.15.52:0/4046110931 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa6500d2290 0x7fa6500caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.383+0000 7fa655000640 1 --2- 172.21.15.52:0/4046110931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa65015ccb0 0x7fa6500b75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.383+0000 7f24bca00640 1 Processor -- start 2024-03-12T17:06:58.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.383+0000 7f67ba000640 1 --2- 172.21.15.52:0/1868372084 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f67a406ac30 0x7f67a406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.383+0000 7f67ba000640 1 -- 172.21.15.52:0/1868372084 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f67c0124290 con 0x7f67bc04a9b0 2024-03-12T17:06:58.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.384+0000 7f0778400640 1 Processor -- start 2024-03-12T17:06:58.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.384+0000 7f24bca00640 1 -- start start 2024-03-12T17:06:58.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.384+0000 7f0778400640 1 -- start start 2024-03-12T17:06:58.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.384+0000 7fa64f400640 1 -- 172.21.15.52:0/4046110931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa65015ccb0 msgr2=0x7fa6500b75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.384+0000 7fa64f400640 1 --2- 172.21.15.52:0/4046110931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa65015ccb0 0x7fa6500b75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.384+0000 7fa64f400640 1 -- 172.21.15.52:0/4046110931 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6500b4390 msgr2=0x7fa65014f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.384+0000 7fa64f400640 1 --2- 172.21.15.52:0/4046110931 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6500b4390 0x7fa65014f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.384+0000 7f67c5e00640 1 --2- 172.21.15.52:0/1868372084 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f67a406ac30 0x7f67a406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.384+0000 7fa64f400640 1 -- 172.21.15.52:0/4046110931 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63c009400 con 0x7fa6500d2290 2024-03-12T17:06:58.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.384+0000 7fa64fe00640 1 --2- 172.21.15.52:0/4046110931 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6500b4390 0x7fa65014f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T17:06:58.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.385+0000 7fa64f400640 1 --2- 172.21.15.52:0/4046110931 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa6500d2290 0x7fa6500caf50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa64000e850 tx=0x7fa64000ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:58.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.385+0000 7f0778400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07700b37f0 0x7f07701606a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.385+0000 7f67c5e00640 1 --2- 172.21.15.52:0/1868372084 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f67a406ac30 0x7f67a406d0f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f67b4007fd0 tx=0x7f67b4007980 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:58.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.385+0000 7f0778400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07700cc7b0 0x7f0770162050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.385+0000 7f0778400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f077014aa40 0x7f077013f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.385+0000 7f0778400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0770163700 con 0x7f07700cc7b0 2024-03-12T17:06:58.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.385+0000 7f0778400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0770162af0 con 0x7f07700b37f0 2024-03-12T17:06:58.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.385+0000 7f0778400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0770162dd0 con 0x7f077014aa40 2024-03-12T17:06:58.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.386+0000 7f0775000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07700cc7b0 0x7f0770162050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.386+0000 7f0775000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07700cc7b0 0x7f0770162050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60994/0 (socket says 172.21.15.52:60994) 2024-03-12T17:06:58.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.386+0000 7f0775000640 1 -- 172.21.15.52:0/3753564539 learned_addr learned my addr 172.21.15.52:0/3753564539 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:58.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.386+0000 7f24bca00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24b80b27e0 0x7f24b8160870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.386+0000 7f24bca00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f24b80cc7b0 0x7f24b8143d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.386+0000 7f0775a00640 1 --2- 172.21.15.52:0/3753564539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07700b37f0 0x7f07701606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.386+0000 7f0776400640 1 --2- 172.21.15.52:0/3753564539 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f077014aa40 0x7f077013f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.386+0000 7f0775000640 1 -- 172.21.15.52:0/3753564539 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f077014aa40 msgr2=0x7f077013f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.386+0000 7f24bca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24b814aa40 0x7f24b81f4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.386+0000 7f0775000640 1 --2- 172.21.15.52:0/3753564539 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f077014aa40 0x7f077013f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.386+0000 7f0775000640 1 -- 172.21.15.52:0/3753564539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07700b37f0 msgr2=0x7f07701606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.386+0000 7f0775000640 1 --2- 172.21.15.52:0/3753564539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07700b37f0 0x7f07701606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.386+0000 7f0775000640 1 -- 172.21.15.52:0/3753564539 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f075c009400 con 0x7f07700cc7b0 2024-03-12T17:06:58.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.386+0000 7f0775a00640 1 --2- 172.21.15.52:0/3753564539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07700b37f0 0x7f07701606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:06:58.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.386+0000 7f24b5e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24b80b27e0 0x7f24b8160870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.386+0000 7f24b5e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24b80b27e0 0x7f24b8160870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:32772/0 (socket says 172.21.15.52:32772) 2024-03-12T17:06:58.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.386+0000 7f0776400640 1 --2- 172.21.15.52:0/3753564539 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f077014aa40 0x7f077013f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:06:58.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.387+0000 7f24bca00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b81feea0 con 0x7f24b80b27e0 2024-03-12T17:06:58.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.387+0000 7f24b5e00640 1 -- 172.21.15.52:0/3230373000 learned_addr learned my addr 172.21.15.52:0/3230373000 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:58.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.387+0000 7f24bca00640 1 -- 172.21.15.52:0/3230373000 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b8160db0 con 0x7f24b814aa40 2024-03-12T17:06:58.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.387+0000 7f24b5400640 1 --2- 172.21.15.52:0/3230373000 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f24b80cc7b0 0x7f24b8143d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.387+0000 7f24bca00640 1 -- 172.21.15.52:0/3230373000 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b8164500 con 0x7f24b80cc7b0 2024-03-12T17:06:58.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.387+0000 7f0775000640 1 --2- 172.21.15.52:0/3753564539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07700cc7b0 0x7f0770162050 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f076800c4b0 tx=0x7f076800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:58.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.387+0000 7f24b6800640 1 --2- 172.21.15.52:0/3230373000 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24b814aa40 0x7f24b81f4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.388+0000 7f24b5e00640 1 -- 172.21.15.52:0/3230373000 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f24b80cc7b0 msgr2=0x7f24b8143d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.388+0000 7f24b5e00640 1 --2- 172.21.15.52:0/3230373000 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f24b80cc7b0 0x7f24b8143d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.388+0000 7f24b5e00640 1 -- 172.21.15.52:0/3230373000 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24b814aa40 msgr2=0x7f24b81f4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.388+0000 7f24b5e00640 1 --2- 172.21.15.52:0/3230373000 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24b814aa40 0x7f24b81f4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.388+0000 7f24b5e00640 1 -- 172.21.15.52:0/3230373000 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a80094d0 con 0x7f24b80b27e0 2024-03-12T17:06:58.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.388+0000 7f24b6800640 1 --2- 172.21.15.52:0/3230373000 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24b814aa40 0x7f24b81f4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:06:58.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.388+0000 7f24b5e00640 1 --2- 172.21.15.52:0/3230373000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24b80b27e0 0x7f24b8160870 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f24a8000c00 tx=0x7f24a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:58.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.390+0000 7f24ae000640 1 -- 172.21.15.52:0/3230373000 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a803abe0 con 0x7f24b80b27e0 2024-03-12T17:06:58.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.390+0000 7f24ae000640 1 -- 172.21.15.52:0/3230373000 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a803ad40 con 0x7f24b80b27e0 2024-03-12T17:06:58.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.390+0000 7f076d600640 1 -- 172.21.15.52:0/3753564539 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0768005480 con 0x7f07700cc7b0 2024-03-12T17:06:58.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.390+0000 7f076d600640 1 -- 172.21.15.52:0/3753564539 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07680035f0 con 0x7f07700cc7b0 2024-03-12T17:06:58.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.391+0000 7f24ae000640 1 -- 172.21.15.52:0/3230373000 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a8044630 con 0x7f24b80b27e0 2024-03-12T17:06:58.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.391+0000 7f076d600640 1 -- 172.21.15.52:0/3753564539 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07680039e0 con 0x7f07700cc7b0 2024-03-12T17:06:58.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.391+0000 7f0778400640 1 -- 172.21.15.52:0/3753564539 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07701630b0 con 0x7f07700cc7b0 2024-03-12T17:06:58.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.391+0000 7f24bca00640 1 -- 172.21.15.52:0/3230373000 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b81ff6d0 con 0x7f24b80b27e0 2024-03-12T17:06:58.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.391+0000 7f24bca00640 1 -- 172.21.15.52:0/3230373000 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b8164220 con 0x7f24b80b27e0 2024-03-12T17:06:58.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.391+0000 7f67ba000640 1 -- 172.21.15.52:0/1868372084 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f67c00ef180 con 0x7f67bc04a9b0 2024-03-12T17:06:58.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.391+0000 7f0778400640 1 -- 172.21.15.52:0/3753564539 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0770160be0 con 0x7f07700cc7b0 2024-03-12T17:06:58.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.392+0000 7f24bca00640 1 -- 172.21.15.52:0/3230373000 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24b806ac20 con 0x7f24b80b27e0 2024-03-12T17:06:58.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.393+0000 7f5ef7e00640 1 -- 172.21.15.52:0/3882772499 <== mon.2 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eec031780 con 0x7f5ef80cb250 2024-03-12T17:06:58.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.393+0000 7fa647e00640 1 -- 172.21.15.52:0/4046110931 <== mon.2 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6400095b0 con 0x7fa6500d2290 2024-03-12T17:06:58.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.393+0000 7fa657000640 1 -- 172.21.15.52:0/4046110931 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa65015eb20 con 0x7fa6500d2290 2024-03-12T17:06:58.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.393+0000 7fa657000640 1 -- 172.21.15.52:0/4046110931 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa65014fad0 con 0x7fa6500d2290 2024-03-12T17:06:58.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.393+0000 7f0778400640 1 -- 172.21.15.52:0/3753564539 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f077006ac20 con 0x7f07700cc7b0 2024-03-12T17:06:58.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.393+0000 7f1268c00640 1 -- 172.21.15.52:0/1755954139 <== mon.2 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f12580542d0 con 0x7f126c15df40 2024-03-12T17:06:58.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.393+0000 7fa647e00640 1 -- 172.21.15.52:0/4046110931 <== mon.2 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa64000c940 con 0x7fa6500d2290 2024-03-12T17:06:58.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.394+0000 7fa647e00640 1 -- 172.21.15.52:0/4046110931 <== mon.2 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64000ccb0 con 0x7fa6500d2290 2024-03-12T17:06:58.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.394+0000 7f5ef7e00640 1 -- 172.21.15.52:0/3882772499 <== mon.2 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eec0318e0 con 0x7f5ef80cb250 2024-03-12T17:06:58.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.394+0000 7f5ef7e00640 1 -- 172.21.15.52:0/3882772499 <== mon.2 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eec031c70 con 0x7f5ef80cb250 2024-03-12T17:06:58.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.395+0000 7f5f00400640 1 -- 172.21.15.52:0/3882772499 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ef80cb250 msgr2=0x7f5ef8098a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.395+0000 7f5f00400640 1 --2- 172.21.15.52:0/3882772499 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ef80cb250 0x7f5ef8098a40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0097a0 tx=0x7f5eec02b820 comp rx=0 tx=0).stop 2024-03-12T17:06:58.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.394+0000 7f24ae000640 1 -- 172.21.15.52:0/3230373000 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f24a804d450 con 0x7f24b80b27e0 2024-03-12T17:06:58.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.395+0000 7f38ae000640 1 -- 172.21.15.52:0/1054639472 <== mon.2 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a802c960 con 0x7f38b00b3f30 2024-03-12T17:06:58.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.397+0000 7f24ae000640 1 --2- 172.21.15.52:0/3230373000 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f249806acd0 0x7f249806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.397+0000 7f24ae000640 1 -- 172.21.15.52:0/3230373000 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f24a802d040 con 0x7f24b80b27e0 2024-03-12T17:06:58.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.395+0000 7f076d600640 1 -- 172.21.15.52:0/3753564539 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0768003b40 con 0x7f07700cc7b0 2024-03-12T17:06:58.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.397+0000 7f076d600640 1 --2- 172.21.15.52:0/3753564539 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f075006acd0 0x7f075006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.397+0000 7f076d600640 1 -- 172.21.15.52:0/3753564539 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0768088cd0 con 0x7f07700cc7b0 2024-03-12T17:06:58.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.395+0000 7fa657000640 1 -- 172.21.15.52:0/4046110931 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa628005250 con 0x7fa6500d2290 2024-03-12T17:06:58.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.396+0000 7f5f00400640 1 -- 172.21.15.52:0/3882772499 shutdown_connections 2024-03-12T17:06:58.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.396+0000 7f5f00400640 1 --2- 172.21.15.52:0/3882772499 >> v2:172.21.15.102:3300/0 conn(0x7f5ef8098f80 0x7f5ef80b13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.396+0000 7f5f00400640 1 --2- 172.21.15.52:0/3882772499 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ef80cb250 0x7f5ef8098a40 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.396+0000 7f5f00400640 1 -- 172.21.15.52:0/3882772499 >> 172.21.15.52:0/3882772499 conn(0x7f5ef80b21a0 msgr2=0x7f5ef80b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:58.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.398+0000 7f5f00400640 1 -- 172.21.15.52:0/3882772499 shutdown_connections 2024-03-12T17:06:58.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.395+0000 7f38ae000640 1 -- 172.21.15.52:0/1054639472 <== mon.2 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a802cac0 con 0x7f38b00b3f30 2024-03-12T17:06:58.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.396+0000 7f38ae000640 1 -- 172.21.15.52:0/1054639472 <== mon.2 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a802ce50 con 0x7f38b00b3f30 2024-03-12T17:06:58.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.397+0000 7f38b6800640 1 -- 172.21.15.52:0/1054639472 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f38b00b3f30 msgr2=0x7f38b00cd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.397+0000 7f38b6800640 1 --2- 172.21.15.52:0/1054639472 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f38b00b3f30 0x7f38b00cd210 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009e80 tx=0x7f38a802be40 comp rx=0 tx=0).stop 2024-03-12T17:06:58.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.398+0000 7f38b6800640 1 -- 172.21.15.52:0/1054639472 shutdown_connections 2024-03-12T17:06:58.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.398+0000 7f38b6800640 1 --2- 172.21.15.52:0/1054639472 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f38b00b3f30 0x7f38b00cd210 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.398+0000 7f5f00400640 1 -- 172.21.15.52:0/3882772499 wait complete. 2024-03-12T17:06:58.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.398+0000 7f24b5400640 1 --2- 172.21.15.52:0/3230373000 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f249806acd0 0x7f249806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.398+0000 7f38b6800640 1 -- 172.21.15.52:0/1054639472 >> 172.21.15.52:0/1054639472 conn(0x7f38b00b03a0 msgr2=0x7f38b00b0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:58.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.398+0000 7fa647e00640 1 -- 172.21.15.52:0/4046110931 <== mon.2 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa640013470 con 0x7fa6500d2290 2024-03-12T17:06:58.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.398+0000 7f0775a00640 1 --2- 172.21.15.52:0/3753564539 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f075006acd0 0x7f075006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.399+0000 7f38b6800640 1 -- 172.21.15.52:0/1054639472 shutdown_connections 2024-03-12T17:06:58.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.399+0000 7f38b6800640 1 -- 172.21.15.52:0/1054639472 wait complete. 2024-03-12T17:06:58.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.400+0000 7fa647e00640 1 --2- 172.21.15.52:0/4046110931 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa63006ac00 0x7fa63006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.400+0000 7fa647e00640 1 -- 172.21.15.52:0/4046110931 <== mon.2 v2:172.21.15.102:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa640089aa0 con 0x7fa6500d2290 2024-03-12T17:06:58.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.400+0000 7fa64fe00640 1 --2- 172.21.15.52:0/4046110931 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa63006ac00 0x7fa63006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.400+0000 7f24b5400640 1 --2- 172.21.15.52:0/3230373000 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f249806acd0 0x7f249806d190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f24b81f4760 tx=0x7f24a4009190 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:58.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.400+0000 7f0775a00640 1 --2- 172.21.15.52:0/3753564539 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f075006acd0 0x7f075006d190 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f075c036000 tx=0x7f075c03c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:58.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.401+0000 7f5f00400640 1 Processor -- start 2024-03-12T17:06:58.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.401+0000 7f5f00400640 1 -- start start 2024-03-12T17:06:58.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.401+0000 7f38b6800640 1 Processor -- start 2024-03-12T17:06:58.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.402+0000 7f38b6800640 1 -- start start 2024-03-12T17:06:58.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.402+0000 7fa64fe00640 1 --2- 172.21.15.52:0/4046110931 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa63006ac00 0x7fa63006d0c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa63c02b9c0 tx=0x7fa63c008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:58.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.402+0000 7f5f00400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ef8098f80 0x7f5ef81590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.402+0000 7f5f00400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ef80cb250 0x7f5ef815d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.403+0000 7f5f00400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ef815a950 0x7f5ef80c9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.403+0000 7f5f00400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef80c8b60 con 0x7f5ef815a950 2024-03-12T17:06:58.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.403+0000 7f5f00400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef80c8e40 con 0x7f5ef80cb250 2024-03-12T17:06:58.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.403+0000 7f5f00400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef80c9120 con 0x7f5ef8098f80 2024-03-12T17:06:58.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.403+0000 7f5efe400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ef815a950 0x7f5ef80c9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.403+0000 7f38b6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38b00b2250 0x7f38b0158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.403+0000 7f38b6800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f38b00b3f30 0x7f38b014ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.403+0000 7f5efe400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ef815a950 0x7f5ef80c9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:32778/0 (socket says 172.21.15.52:32778) 2024-03-12T17:06:58.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.403+0000 7f5efe400640 1 -- 172.21.15.52:0/106612832 learned_addr learned my addr 172.21.15.52:0/106612832 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:58.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.403+0000 7f5efda00640 1 --2- 172.21.15.52:0/106612832 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ef8098f80 0x7f5ef81590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.403+0000 7f5efd000640 1 --2- 172.21.15.52:0/106612832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ef80cb250 0x7f5ef815d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.404+0000 7f38b6800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38b0159c20 0x7f38b014d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.404+0000 7f38b6800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b0156ee0 con 0x7f38b0159c20 2024-03-12T17:06:58.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.404+0000 7f38b6800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b01571c0 con 0x7f38b00b2250 2024-03-12T17:06:58.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.404+0000 7f38b6800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b01574a0 con 0x7f38b00b3f30 2024-03-12T17:06:58.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.404+0000 7f5efe400640 1 -- 172.21.15.52:0/106612832 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ef8098f80 msgr2=0x7f5ef81590b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.404+0000 7f5efe400640 1 --2- 172.21.15.52:0/106612832 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ef8098f80 0x7f5ef81590b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.404+0000 7f5efe400640 1 -- 172.21.15.52:0/106612832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ef80cb250 msgr2=0x7f5ef815d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.404+0000 7f38aea00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f38b00b3f30 0x7f38b014ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.404+0000 7f38aea00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f38b00b3f30 0x7f38b014ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.52:55818/0 (socket says 172.21.15.52:55818) 2024-03-12T17:06:58.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.404+0000 7f38aea00640 1 -- 172.21.15.52:0/1135909508 learned_addr learned my addr 172.21.15.52:0/1135909508 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:06:58.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.404+0000 7f38af400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38b00b2250 0x7f38b0158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.404+0000 7f38afe00640 1 --2- 172.21.15.52:0/1135909508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38b0159c20 0x7f38b014d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.404+0000 7f5efe400640 1 --2- 172.21.15.52:0/106612832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ef80cb250 0x7f5ef815d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.404+0000 7f5efe400640 1 -- 172.21.15.52:0/106612832 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eec009400 con 0x7f5ef815a950 2024-03-12T17:06:58.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.405+0000 7f5efe400640 1 --2- 172.21.15.52:0/106612832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ef815a950 0x7f5ef80c9c00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0007e70 tx=0x7f5ef000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:58.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.405+0000 7f24ae000640 1 -- 172.21.15.52:0/3230373000 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f24a8076980 con 0x7f24b80b27e0 2024-03-12T17:06:58.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.406+0000 7f38afe00640 1 -- 172.21.15.52:0/1135909508 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f38b00b3f30 msgr2=0x7f38b014ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.406+0000 7f38afe00640 1 --2- 172.21.15.52:0/1135909508 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f38b00b3f30 0x7f38b014ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.406+0000 7f38afe00640 1 -- 172.21.15.52:0/1135909508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38b00b2250 msgr2=0x7f38b0158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.406+0000 7f38afe00640 1 --2- 172.21.15.52:0/1135909508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38b00b2250 0x7f38b0158540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.406+0000 7f38afe00640 1 -- 172.21.15.52:0/1135909508 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a8009b30 con 0x7f38b0159c20 2024-03-12T17:06:58.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.406+0000 7f38afe00640 1 --2- 172.21.15.52:0/1135909508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38b0159c20 0x7f38b014d720 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f38a8032000 tx=0x7f38a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:58.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.406+0000 7f38af400640 1 --2- 172.21.15.52:0/1135909508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38b00b2250 0x7f38b0158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T17:06:58.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.406+0000 7f38aea00640 1 --2- 172.21.15.52:0/1135909508 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f38b00b3f30 0x7f38b014ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T17:06:58.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.414+0000 7f076d600640 1 -- 172.21.15.52:0/3753564539 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f076808e020 con 0x7f07700cc7b0 2024-03-12T17:06:58.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.413+0000 7f5ef5600640 1 -- 172.21.15.52:0/106612832 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0003b90 con 0x7f5ef815a950 2024-03-12T17:06:58.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.414+0000 7f5ef5600640 1 -- 172.21.15.52:0/106612832 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef000f040 con 0x7f5ef815a950 2024-03-12T17:06:58.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.414+0000 7f5ef5600640 1 -- 172.21.15.52:0/106612832 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef000de80 con 0x7f5ef815a950 2024-03-12T17:06:58.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.414+0000 7f5f00400640 1 -- 172.21.15.52:0/106612832 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef80c9400 con 0x7f5ef815a950 2024-03-12T17:06:58.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.414+0000 7f5f00400640 1 -- 172.21.15.52:0/106612832 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef81595f0 con 0x7f5ef815a950 2024-03-12T17:06:58.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.414+0000 7fa647e00640 1 -- 172.21.15.52:0/4046110931 <== mon.2 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fa640054a30 con 0x7fa6500d2290 2024-03-12T17:06:58.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.415+0000 7f38a7400640 1 -- 172.21.15.52:0/1135909508 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a803e450 con 0x7f38b0159c20 2024-03-12T17:06:58.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.415+0000 7f38a7400640 1 -- 172.21.15.52:0/1135909508 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a803e5b0 con 0x7f38b0159c20 2024-03-12T17:06:58.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.415+0000 7f38a7400640 1 -- 172.21.15.52:0/1135909508 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a803e940 con 0x7f38b0159c20 2024-03-12T17:06:58.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.416+0000 7f5f00400640 1 -- 172.21.15.52:0/106612832 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef80c9920 con 0x7f5ef815a950 2024-03-12T17:06:58.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.416+0000 7f38b6800640 1 -- 172.21.15.52:0/1135909508 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b0157780 con 0x7f38b0159c20 2024-03-12T17:06:58.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.416+0000 7f38b6800640 1 -- 172.21.15.52:0/1135909508 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b0156490 con 0x7f38b0159c20 2024-03-12T17:06:58.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.418+0000 7f38b6800640 1 -- 172.21.15.52:0/1135909508 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38b014b950 con 0x7f38b0159c20 2024-03-12T17:06:58.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.418+0000 7f5ef5600640 1 -- 172.21.15.52:0/106612832 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5ef0004030 con 0x7f5ef815a950 2024-03-12T17:06:58.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.419+0000 7f5ef5600640 1 --2- 172.21.15.52:0/106612832 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f5ed806ac50 0x7f5ed806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.419+0000 7f5ef5600640 1 -- 172.21.15.52:0/106612832 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5ef00897c0 con 0x7f5ef815a950 2024-03-12T17:06:58.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.422+0000 7f5efda00640 1 --2- 172.21.15.52:0/106612832 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f5ed806ac50 0x7f5ed806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.421+0000 7f38a7400640 1 -- 172.21.15.52:0/1135909508 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f38a803eaa0 con 0x7f38b0159c20 2024-03-12T17:06:58.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.423+0000 7f38a7400640 1 --2- 172.21.15.52:0/1135909508 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f389006acd0 0x7f389006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:06:58.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.423+0000 7f38a7400640 1 -- 172.21.15.52:0/1135909508 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f38a80ac240 con 0x7f38b0159c20 2024-03-12T17:06:58.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.423+0000 7f5efda00640 1 --2- 172.21.15.52:0/106612832 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f5ed806ac50 0x7f5ed806d110 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4005880 tx=0x7f5ee4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:58.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.424+0000 7f38af400640 1 --2- 172.21.15.52:0/1135909508 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f389006acd0 0x7f389006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:06:58.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.425+0000 7f38af400640 1 --2- 172.21.15.52:0/1135909508 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f389006acd0 0x7f389006d190 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f38a0001ce0 tx=0x7f38a0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:06:58.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.429+0000 7f5ef5600640 1 -- 172.21.15.52:0/106612832 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f5ef80c9920 con 0x7f5ef815a950 2024-03-12T17:06:58.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.438+0000 7f38a7400640 1 -- 172.21.15.52:0/1135909508 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f38b014b950 con 0x7f38b0159c20 2024-03-12T17:06:58.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.809+0000 7f15c0000640 1 -- 172.21.15.52:0/3087183359 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1594005500 con 0x7f15b80b1bc0 2024-03-12T17:06:58.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.810+0000 7f15b5600640 1 -- 172.21.15.52:0/3087183359 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f15b0043020 con 0x7f15b80b1bc0 2024-03-12T17:06:58.804 INFO:teuthology.orchestra.run.smithi052.stdout:55834574856 2024-03-12T17:06:58.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.811+0000 7f67c8800640 1 -- 172.21.15.52:0/1868372084 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f67bc0057b0 con 0x7f67bc04a9b0 2024-03-12T17:06:58.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.812+0000 7f67ba000640 1 -- 172.21.15.52:0/1868372084 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f67c0073e40 con 0x7f67bc04a9b0 2024-03-12T17:06:58.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.815+0000 7f15c0000640 1 -- 172.21.15.52:0/3087183359 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f159806ac80 msgr2=0x7f159806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.815+0000 7f15c0000640 1 --2- 172.21.15.52:0/3087183359 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f159806ac80 0x7f159806d140 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f15ac006fd0 tx=0x7f15ac008040 comp rx=0 tx=0).stop 2024-03-12T17:06:58.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.815+0000 7f15c0000640 1 -- 172.21.15.52:0/3087183359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15b80b1bc0 msgr2=0x7f15b80cce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.815+0000 7f15c0000640 1 --2- 172.21.15.52:0/3087183359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15b80b1bc0 0x7f15b80cce40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f15b0000f80 tx=0x7f15b0004a40 comp rx=0 tx=0).stop 2024-03-12T17:06:58.809 INFO:teuthology.orchestra.run.smithi052.stdout:55834574856 2024-03-12T17:06:58.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.815+0000 7f15c0000640 1 -- 172.21.15.52:0/3087183359 shutdown_connections 2024-03-12T17:06:58.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.815+0000 7f15c0000640 1 --2- 172.21.15.52:0/3087183359 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f159806ac80 0x7f159806d140 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.815+0000 7f15c0000640 1 --2- 172.21.15.52:0/3087183359 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f15b80cc0e0 0x7f15b80b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.815+0000 7f15c0000640 1 --2- 172.21.15.52:0/3087183359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15b80b1bc0 0x7f15b80cce40 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.815+0000 7f15c0000640 1 --2- 172.21.15.52:0/3087183359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15b80afd20 0x7f15b80c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.815+0000 7f15c0000640 1 -- 172.21.15.52:0/3087183359 >> 172.21.15.52:0/3087183359 conn(0x7f15b80b2140 msgr2=0x7f15b80ae9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:58.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.815+0000 7f15c0000640 1 -- 172.21.15.52:0/3087183359 shutdown_connections 2024-03-12T17:06:58.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.816+0000 7f15c0000640 1 -- 172.21.15.52:0/3087183359 wait complete. 2024-03-12T17:06:58.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.816+0000 7f67c8800640 1 -- 172.21.15.52:0/1868372084 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f67a406ac30 msgr2=0x7f67a406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.816+0000 7f67c8800640 1 --2- 172.21.15.52:0/1868372084 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f67a406ac30 0x7f67a406d0f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f67b4007fd0 tx=0x7f67b4007980 comp rx=0 tx=0).stop 2024-03-12T17:06:58.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.816+0000 7f67c8800640 1 -- 172.21.15.52:0/1868372084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67bc04a9b0 msgr2=0x7f67bc04d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.816+0000 7f67c8800640 1 --2- 172.21.15.52:0/1868372084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67bc04a9b0 0x7f67bc04d000 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f67c002c880 tx=0x7f67c0068280 comp rx=0 tx=0).stop 2024-03-12T17:06:58.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.816+0000 7f67c8800640 1 -- 172.21.15.52:0/1868372084 shutdown_connections 2024-03-12T17:06:58.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.816+0000 7f67c8800640 1 --2- 172.21.15.52:0/1868372084 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f67a406ac30 0x7f67a406d0f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.816+0000 7f67c8800640 1 --2- 172.21.15.52:0/1868372084 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67bc0e43e0 0x7f67bc04a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.816+0000 7f67c8800640 1 --2- 172.21.15.52:0/1868372084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67bc04a9b0 0x7f67bc04d000 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.816+0000 7f67c8800640 1 --2- 172.21.15.52:0/1868372084 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67a8005090 0x7f67bc04e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.817+0000 7f67c8800640 1 -- 172.21.15.52:0/1868372084 >> 172.21.15.52:0/1868372084 conn(0x7f67bc054ee0 msgr2=0x7f67bc052fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:58.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.817+0000 7f67c8800640 1 -- 172.21.15.52:0/1868372084 shutdown_connections 2024-03-12T17:06:58.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.817+0000 7f67c8800640 1 -- 172.21.15.52:0/1868372084 wait complete. 2024-03-12T17:06:58.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.818+0000 7f1273c00640 1 -- 172.21.15.52:0/1755954139 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f126c15c150 con 0x7f126c15df40 2024-03-12T17:06:58.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.819+0000 7f1268c00640 1 -- 172.21.15.52:0/1755954139 <== mon.2 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f12580578f0 con 0x7f126c15df40 2024-03-12T17:06:58.813 INFO:teuthology.orchestra.run.smithi052.stdout:55834574856 2024-03-12T17:06:58.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.824+0000 7f1273c00640 1 -- 172.21.15.52:0/1755954139 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f124c06ac00 msgr2=0x7f124c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.824+0000 7f1273c00640 1 --2- 172.21.15.52:0/1755954139 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f124c06ac00 0x7f124c06d0c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1264036000 tx=0x7f1264030000 comp rx=0 tx=0).stop 2024-03-12T17:06:58.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.824+0000 7f1273c00640 1 -- 172.21.15.52:0/1755954139 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f126c15df40 msgr2=0x7f126c15f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.824+0000 7f1273c00640 1 --2- 172.21.15.52:0/1755954139 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f126c15df40 0x7f126c15f3a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f125800c3e0 tx=0x7f125800c8a0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.824+0000 7f1273c00640 1 -- 172.21.15.52:0/1755954139 shutdown_connections 2024-03-12T17:06:58.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.824+0000 7f1273c00640 1 --2- 172.21.15.52:0/1755954139 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f124c06ac00 0x7f124c06d0c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.824+0000 7f1273c00640 1 --2- 172.21.15.52:0/1755954139 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f126c15df40 0x7f126c15f3a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.824+0000 7f1273c00640 1 --2- 172.21.15.52:0/1755954139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f126c0b5a30 0x7f126c15ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.824+0000 7f1273c00640 1 --2- 172.21.15.52:0/1755954139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f126c0b4a60 0x7f126c15a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.824+0000 7f1273c00640 1 -- 172.21.15.52:0/1755954139 >> 172.21.15.52:0/1755954139 conn(0x7f126c0cf350 msgr2=0x7f126c0d2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:58.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.825+0000 7f1273c00640 1 -- 172.21.15.52:0/1755954139 shutdown_connections 2024-03-12T17:06:58.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.826+0000 7f1273c00640 1 -- 172.21.15.52:0/1755954139 wait complete. 2024-03-12T17:06:58.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.839+0000 7f24bca00640 1 -- 172.21.15.52:0/3230373000 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f24b8073390 con 0x7f24b80b27e0 2024-03-12T17:06:58.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.840+0000 7f24ae000640 1 -- 172.21.15.52:0/3230373000 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f24a8035090 con 0x7f24b80b27e0 2024-03-12T17:06:58.837 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.0 2024-03-12T17:06:58.837 INFO:teuthology.orchestra.run.smithi052.stdout:55834574857 2024-03-12T17:06:58.838 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.3 2024-03-12T17:06:58.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.846+0000 7f24bca00640 1 -- 172.21.15.52:0/3230373000 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f249806acd0 msgr2=0x7f249806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.846+0000 7f24bca00640 1 --2- 172.21.15.52:0/3230373000 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f249806acd0 0x7f249806d190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f24b81f4760 tx=0x7f24a4009190 comp rx=0 tx=0).stop 2024-03-12T17:06:58.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.847+0000 7f24bca00640 1 -- 172.21.15.52:0/3230373000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24b80b27e0 msgr2=0x7f24b8160870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.847+0000 7f24bca00640 1 --2- 172.21.15.52:0/3230373000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24b80b27e0 0x7f24b8160870 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f24a8000c00 tx=0x7f24a80046c0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.847+0000 7f24bca00640 1 -- 172.21.15.52:0/3230373000 shutdown_connections 2024-03-12T17:06:58.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.847+0000 7f24bca00640 1 --2- 172.21.15.52:0/3230373000 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f249806acd0 0x7f249806d190 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.847+0000 7f24bca00640 1 --2- 172.21.15.52:0/3230373000 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f24b80cc7b0 0x7f24b8143d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.847+0000 7f24bca00640 1 --2- 172.21.15.52:0/3230373000 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24b814aa40 0x7f24b81f4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.847+0000 7f24bca00640 1 --2- 172.21.15.52:0/3230373000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24b80b27e0 0x7f24b8160870 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.847+0000 7f24bca00640 1 -- 172.21.15.52:0/3230373000 >> 172.21.15.52:0/3230373000 conn(0x7f24b806a3d0 msgr2=0x7f24b8070990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:58.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.847+0000 7f24bca00640 1 -- 172.21.15.52:0/3230373000 shutdown_connections 2024-03-12T17:06:58.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.847+0000 7f24bca00640 1 -- 172.21.15.52:0/3230373000 wait complete. 2024-03-12T17:06:58.845 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.1 2024-03-12T17:06:58.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.853+0000 7fa657000640 1 -- 172.21.15.52:0/4046110931 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fa628005500 con 0x7fa6500d2290 2024-03-12T17:06:58.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.853+0000 7f0778400640 1 -- 172.21.15.52:0/3753564539 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0770143b80 con 0x7f07700cc7b0 2024-03-12T17:06:58.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.854+0000 7fa647e00640 1 -- 172.21.15.52:0/4046110931 <== mon.2 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa640021020 con 0x7fa6500d2290 2024-03-12T17:06:58.847 INFO:teuthology.orchestra.run.smithi052.stdout:55834574856 2024-03-12T17:06:58.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.854+0000 7f076d600640 1 -- 172.21.15.52:0/3753564539 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0768017020 con 0x7f07700cc7b0 2024-03-12T17:06:58.848 INFO:teuthology.orchestra.run.smithi052.stdout:55834574856 2024-03-12T17:06:58.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.857+0000 7fa657000640 1 -- 172.21.15.52:0/4046110931 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa63006ac00 msgr2=0x7fa63006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.857+0000 7fa657000640 1 --2- 172.21.15.52:0/4046110931 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa63006ac00 0x7fa63006d0c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa63c02b9c0 tx=0x7fa63c008030 comp rx=0 tx=0).stop 2024-03-12T17:06:58.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.857+0000 7fa657000640 1 -- 172.21.15.52:0/4046110931 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa6500d2290 msgr2=0x7fa6500caf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.857+0000 7fa657000640 1 --2- 172.21.15.52:0/4046110931 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa6500d2290 0x7fa6500caf50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa64000e850 tx=0x7fa64000ed10 comp rx=0 tx=0).stop 2024-03-12T17:06:58.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.857+0000 7fa657000640 1 -- 172.21.15.52:0/4046110931 shutdown_connections 2024-03-12T17:06:58.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.857+0000 7fa657000640 1 --2- 172.21.15.52:0/4046110931 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa63006ac00 0x7fa63006d0c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.857+0000 7fa657000640 1 --2- 172.21.15.52:0/4046110931 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa6500d2290 0x7fa6500caf50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.857+0000 7fa657000640 1 --2- 172.21.15.52:0/4046110931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa65015ccb0 0x7fa6500b75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.857+0000 7fa657000640 1 --2- 172.21.15.52:0/4046110931 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6500b4390 0x7fa65014f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.857+0000 7fa657000640 1 -- 172.21.15.52:0/4046110931 >> 172.21.15.52:0/4046110931 conn(0x7fa6500717b0 msgr2=0x7fa6500b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:58.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.857+0000 7fa657000640 1 -- 172.21.15.52:0/4046110931 shutdown_connections 2024-03-12T17:06:58.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.857+0000 7fa657000640 1 -- 172.21.15.52:0/4046110931 wait complete. 2024-03-12T17:06:58.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.857+0000 7f0778400640 1 -- 172.21.15.52:0/3753564539 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f075006acd0 msgr2=0x7f075006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.857+0000 7f0778400640 1 --2- 172.21.15.52:0/3753564539 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f075006acd0 0x7f075006d190 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f075c036000 tx=0x7f075c03c040 comp rx=0 tx=0).stop 2024-03-12T17:06:58.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.857+0000 7f0778400640 1 -- 172.21.15.52:0/3753564539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07700cc7b0 msgr2=0x7f0770162050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.858+0000 7f0778400640 1 --2- 172.21.15.52:0/3753564539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07700cc7b0 0x7f0770162050 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f076800c4b0 tx=0x7f076800c970 comp rx=0 tx=0).stop 2024-03-12T17:06:58.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.858+0000 7f0778400640 1 -- 172.21.15.52:0/3753564539 shutdown_connections 2024-03-12T17:06:58.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.858+0000 7f0778400640 1 --2- 172.21.15.52:0/3753564539 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f075006acd0 0x7f075006d190 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.858+0000 7f0778400640 1 --2- 172.21.15.52:0/3753564539 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f077014aa40 0x7f077013f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.858+0000 7f0778400640 1 --2- 172.21.15.52:0/3753564539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07700cc7b0 0x7f0770162050 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.858+0000 7f0778400640 1 --2- 172.21.15.52:0/3753564539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07700b37f0 0x7f07701606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.858+0000 7f0778400640 1 -- 172.21.15.52:0/3753564539 >> 172.21.15.52:0/3753564539 conn(0x7f077006a3d0 msgr2=0x7f0770148a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:58.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.858+0000 7f0778400640 1 -- 172.21.15.52:0/3753564539 shutdown_connections 2024-03-12T17:06:58.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.858+0000 7f0778400640 1 -- 172.21.15.52:0/3753564539 wait complete. 2024-03-12T17:06:58.859 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.7 2024-03-12T17:06:58.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.871+0000 7f5f00400640 1 -- 172.21.15.52:0/106612832 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f5ef80c9920 con 0x7f5ef815a950 2024-03-12T17:06:58.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.871+0000 7f5ef5600640 1 -- 172.21.15.52:0/106612832 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5ef80c9920 con 0x7f5ef815a950 2024-03-12T17:06:58.865 INFO:teuthology.orchestra.run.smithi052.stdout:55834574859 2024-03-12T17:06:58.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.874+0000 7f38b6800640 1 -- 172.21.15.52:0/1135909508 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f38b014b950 con 0x7f38b0159c20 2024-03-12T17:06:58.868 INFO:teuthology.orchestra.run.smithi052.stdout:55834574857 2024-03-12T17:06:58.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.875+0000 7f5f00400640 1 -- 172.21.15.52:0/106612832 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f5ed806ac50 msgr2=0x7f5ed806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.875+0000 7f5f00400640 1 --2- 172.21.15.52:0/106612832 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f5ed806ac50 0x7f5ed806d110 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4005880 tx=0x7f5ee4009040 comp rx=0 tx=0).stop 2024-03-12T17:06:58.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.875+0000 7f5f00400640 1 -- 172.21.15.52:0/106612832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ef815a950 msgr2=0x7f5ef80c9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.875+0000 7f5f00400640 1 --2- 172.21.15.52:0/106612832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ef815a950 0x7f5ef80c9c00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0007e70 tx=0x7f5ef000d4a0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.875+0000 7f38a7400640 1 -- 172.21.15.52:0/1135909508 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f38b014b950 con 0x7f38b0159c20 2024-03-12T17:06:58.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.875+0000 7f5f00400640 1 -- 172.21.15.52:0/106612832 shutdown_connections 2024-03-12T17:06:58.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.875+0000 7f5f00400640 1 --2- 172.21.15.52:0/106612832 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f5ed806ac50 0x7f5ed806d110 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.875+0000 7f5f00400640 1 --2- 172.21.15.52:0/106612832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ef815a950 0x7f5ef80c9c00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.875+0000 7f5f00400640 1 --2- 172.21.15.52:0/106612832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ef80cb250 0x7f5ef815d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.875+0000 7f5f00400640 1 --2- 172.21.15.52:0/106612832 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ef8098f80 0x7f5ef81590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.875+0000 7f5f00400640 1 -- 172.21.15.52:0/106612832 >> 172.21.15.52:0/106612832 conn(0x7f5ef80b21a0 msgr2=0x7f5ef80cd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:58.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.875+0000 7f5f00400640 1 -- 172.21.15.52:0/106612832 shutdown_connections 2024-03-12T17:06:58.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.875+0000 7f5f00400640 1 -- 172.21.15.52:0/106612832 wait complete. 2024-03-12T17:06:58.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.877+0000 7f38b6800640 1 -- 172.21.15.52:0/1135909508 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f389006acd0 msgr2=0x7f389006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.877+0000 7f38b6800640 1 --2- 172.21.15.52:0/1135909508 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f389006acd0 0x7f389006d190 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f38a0001ce0 tx=0x7f38a0009040 comp rx=0 tx=0).stop 2024-03-12T17:06:58.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.877+0000 7f38b6800640 1 -- 172.21.15.52:0/1135909508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38b0159c20 msgr2=0x7f38b014d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:06:58.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.877+0000 7f38b6800640 1 --2- 172.21.15.52:0/1135909508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38b0159c20 0x7f38b014d720 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f38a8032000 tx=0x7f38a8004a40 comp rx=0 tx=0).stop 2024-03-12T17:06:58.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.878+0000 7f38b6800640 1 -- 172.21.15.52:0/1135909508 shutdown_connections 2024-03-12T17:06:58.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.878+0000 7f38b6800640 1 --2- 172.21.15.52:0/1135909508 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f389006acd0 0x7f389006d190 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.878+0000 7f38b6800640 1 --2- 172.21.15.52:0/1135909508 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f38b00b3f30 0x7f38b014ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.878+0000 7f38b6800640 1 --2- 172.21.15.52:0/1135909508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38b0159c20 0x7f38b014d720 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.878+0000 7f38b6800640 1 --2- 172.21.15.52:0/1135909508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38b00b2250 0x7f38b0158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:06:58.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.878+0000 7f38b6800640 1 -- 172.21.15.52:0/1135909508 >> 172.21.15.52:0/1135909508 conn(0x7f38b00b03a0 msgr2=0x7f38b00709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:06:58.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.878+0000 7f38b6800640 1 -- 172.21.15.52:0/1135909508 shutdown_connections 2024-03-12T17:06:58.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:06:58.878+0000 7f38b6800640 1 -- 172.21.15.52:0/1135909508 wait complete. 2024-03-12T17:06:58.875 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.5 2024-03-12T17:06:58.875 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.2 2024-03-12T17:06:58.886 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574859 for osd.4 2024-03-12T17:06:58.889 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.6 2024-03-12T17:06:59.838 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-12T17:06:59.839 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-12T17:06:59.846 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-12T17:06:59.860 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-12T17:06:59.876 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-12T17:06:59.876 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-12T17:06:59.887 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-12T17:06:59.889 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-12T17:07:00.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.126+0000 7f3556000640 1 Processor -- start 2024-03-12T17:07:00.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.126+0000 7f3556000640 1 -- start start 2024-03-12T17:07:00.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.127+0000 7f29c7200640 1 Processor -- start 2024-03-12T17:07:00.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.127+0000 7f29c7200640 1 -- start start 2024-03-12T17:07:00.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.128+0000 7f3556000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f355015c830 0x7f35500d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.128+0000 7f3556000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f35500d1380 0x7f35500b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.128+0000 7f3556000640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35500cda60 con 0x7f35500cd1c0 2024-03-12T17:07:00.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.128+0000 7f3556000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35500cdd40 con 0x7f35500d1380 2024-03-12T17:07:00.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.128+0000 7f3556000640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35500ce020 con 0x7f355015c830 2024-03-12T17:07:00.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.129+0000 7f3554c00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f355015c830 0x7f35500d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.129+0000 7f354fe00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f35500cd1c0 0x7f35500d0c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:54990/0 (socket says 172.21.15.52:54990) 2024-03-12T17:07:00.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.129+0000 7f354fe00640 1 -- 172.21.15.52:0/1064207292 learned_addr learned my addr 172.21.15.52:0/1064207292 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:00.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.129+0000 7f3555600640 1 --2- 172.21.15.52:0/1064207292 >> v2:172.21.15.102:3300/0 conn(0x7f35500d1380 0x7f35500b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.130+0000 7f29c7200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f29c00c57d0 0x7f29c00d2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.130+0000 7f29c7200640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c00b1e40 con 0x7f29c00b2430 2024-03-12T17:07:00.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.130+0000 7f29c7200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c00ce610 con 0x7f29c00cf0e0 2024-03-12T17:07:00.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.130+0000 7f29c7200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c00d1730 con 0x7f29c00c57d0 2024-03-12T17:07:00.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.131+0000 7f354f400640 1 -- 172.21.15.52:0/1064207292 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 949337497 0 0) 0x7f35500cda60 con 0x7f35500cd1c0 2024-03-12T17:07:00.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.131+0000 7f354f400640 1 -- 172.21.15.52:0/1064207292 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3534003520 con 0x7f35500cd1c0 2024-03-12T17:07:00.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.131+0000 7f29bfe00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f29c00cf0e0 0x7f29c00cf4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.52:35854/0 (socket says 172.21.15.52:35854) 2024-03-12T17:07:00.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.131+0000 7f29bfe00640 1 -- 172.21.15.52:0/1665698975 learned_addr learned my addr 172.21.15.52:0/1665698975 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:00.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.132+0000 7f29bea00640 1 -- 172.21.15.52:0/1665698975 <== mon.1 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 132442041 0 0) 0x7f29b8001490 con 0x7f29c00cf0e0 2024-03-12T17:07:00.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.132+0000 7f29bea00640 1 -- 172.21.15.52:0/1665698975 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29a4003520 con 0x7f29c00cf0e0 2024-03-12T17:07:00.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.132+0000 7f29bf400640 1 --2- 172.21.15.52:0/1665698975 >> v2:172.21.15.52:3300/0 conn(0x7f29c00c57d0 0x7f29c00d2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.132+0000 7f3554c00640 1 -- 172.21.15.52:0/1064207292 >> v2:172.21.15.102:3300/0 conn(0x7f35500d1380 msgr2=0x7f35500b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:00.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.132+0000 7f3554c00640 1 --2- 172.21.15.52:0/1064207292 >> v2:172.21.15.102:3300/0 conn(0x7f35500d1380 0x7f35500b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.132+0000 7f3554c00640 1 -- 172.21.15.52:0/1064207292 >> v1:172.21.15.52:6789/0 conn(0x7f35500cd1c0 legacy=0x7f35500d0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.132+0000 7f3554c00640 1 -- 172.21.15.52:0/1064207292 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35500ce300 con 0x7f355015c830 2024-03-12T17:07:00.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.134+0000 7f29bf400640 1 -- 172.21.15.52:0/1665698975 >> v1:172.21.15.102:6789/0 conn(0x7f29c00cf0e0 legacy=0x7f29c00cf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.134+0000 7f29bf400640 1 -- 172.21.15.52:0/1665698975 >> v1:172.21.15.52:6789/0 conn(0x7f29c00b2430 legacy=0x7f29c00b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.135+0000 7f3554c00640 1 --2- 172.21.15.52:0/1064207292 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f355015c830 0x7f35500d0640 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3548009d90 tx=0x7f354802be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ceb4df864eb64b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:00.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.134+0000 7f29bf400640 1 -- 172.21.15.52:0/1665698975 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c00d3550 con 0x7f29c00c57d0 2024-03-12T17:07:00.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.136+0000 7f354f400640 1 -- 172.21.15.52:0/1064207292 <== mon.2 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354802cbc0 con 0x7f355015c830 2024-03-12T17:07:00.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.136+0000 7f354f400640 1 -- 172.21.15.52:0/1064207292 <== mon.2 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354802cd20 con 0x7f355015c830 2024-03-12T17:07:00.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.136+0000 7f354f400640 1 -- 172.21.15.52:0/1064207292 <== mon.2 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3548034660 con 0x7f355015c830 2024-03-12T17:07:00.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.136+0000 7f29bf400640 1 --2- 172.21.15.52:0/1665698975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29c00c57d0 0x7f29c00d2e00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f29ac009720 tx=0x7f29ac02b570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4b58f9124f6800f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:00.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.137+0000 7f29bea00640 1 -- 172.21.15.52:0/1665698975 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b8001b80 con 0x7f29c00c57d0 2024-03-12T17:07:00.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.137+0000 7f29bea00640 1 -- 172.21.15.52:0/1665698975 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29ac02e050 con 0x7f29c00c57d0 2024-03-12T17:07:00.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.137+0000 7f29bea00640 1 -- 172.21.15.52:0/1665698975 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac030730 con 0x7f29c00c57d0 2024-03-12T17:07:00.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.137+0000 7f3556000640 1 -- 172.21.15.52:0/1064207292 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f355015c830 msgr2=0x7f35500d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.137+0000 7f3556000640 1 --2- 172.21.15.52:0/1064207292 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f355015c830 0x7f35500d0640 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3548009d90 tx=0x7f354802be10 comp rx=0 tx=0).stop 2024-03-12T17:07:00.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.138+0000 7f3556000640 1 -- 172.21.15.52:0/1064207292 shutdown_connections 2024-03-12T17:07:00.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.138+0000 7f3556000640 1 --2- 172.21.15.52:0/1064207292 >> v2:172.21.15.102:3300/0 conn(0x7f35500d1380 0x7f35500b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.138+0000 7f29c7200640 1 -- 172.21.15.52:0/1665698975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29c00c57d0 msgr2=0x7f29c00d2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.138+0000 7f29c7200640 1 --2- 172.21.15.52:0/1665698975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29c00c57d0 0x7f29c00d2e00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f29ac009720 tx=0x7f29ac02b570 comp rx=0 tx=0).stop 2024-03-12T17:07:00.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.138+0000 7f3556000640 1 --2- 172.21.15.52:0/1064207292 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f355015c830 0x7f35500d0640 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.138+0000 7f3556000640 1 -- 172.21.15.52:0/1064207292 >> 172.21.15.52:0/1064207292 conn(0x7f3550089fd0 msgr2=0x7f35500d26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:00.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.138+0000 7f3556000640 1 -- 172.21.15.52:0/1064207292 shutdown_connections 2024-03-12T17:07:00.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.139+0000 7f29c7200640 1 -- 172.21.15.52:0/1665698975 shutdown_connections 2024-03-12T17:07:00.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.139+0000 7f3556000640 1 -- 172.21.15.52:0/1064207292 wait complete. 2024-03-12T17:07:00.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.139+0000 7f29c7200640 1 --2- 172.21.15.52:0/1665698975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29c00c57d0 0x7f29c00d2e00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.139+0000 7f29c7200640 1 -- 172.21.15.52:0/1665698975 >> 172.21.15.52:0/1665698975 conn(0x7f29c0089fd0 msgr2=0x7f29c00d26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:00.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.139+0000 7f29c7200640 1 -- 172.21.15.52:0/1665698975 shutdown_connections 2024-03-12T17:07:00.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.139+0000 7f29c7200640 1 -- 172.21.15.52:0/1665698975 wait complete. 2024-03-12T17:07:00.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.141+0000 7f3556000640 1 Processor -- start 2024-03-12T17:07:00.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.142+0000 7f3556000640 1 -- start start 2024-03-12T17:07:00.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.142+0000 7f29c7200640 1 Processor -- start 2024-03-12T17:07:00.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.143+0000 7f29c7200640 1 -- start start 2024-03-12T17:07:00.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.144+0000 7f3556000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35500cd1c0 0x7f35500c5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.145+0000 7f29c7200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29c00b2430 0x7f29c00c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.145+0000 7f3556000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35500d1380 0x7f355014deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.145+0000 7f29c7200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29c00c57d0 0x7f29c00c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.145+0000 7f3554c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35500cd1c0 0x7f35500c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.145+0000 7f3554c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35500cd1c0 0x7f35500c5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:32818/0 (socket says 172.21.15.52:32818) 2024-03-12T17:07:00.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.145+0000 7f29c7200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f29c00cf0e0 0x7f29c00cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.146+0000 7f3556000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f355015c830 0x7f355014e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.146+0000 7f29c7200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c00cc950 con 0x7f29c00b2430 2024-03-12T17:07:00.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.146+0000 7f29c7200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c00c6920 con 0x7f29c00c57d0 2024-03-12T17:07:00.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.146+0000 7f29c7200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c00c6c00 con 0x7f29c00cf0e0 2024-03-12T17:07:00.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.146+0000 7f354fe00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35500d1380 0x7f355014deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.146+0000 7f29c5200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f29c00cf0e0 0x7f29c00cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.146+0000 7f29c5200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f29c00cf0e0 0x7f29c00cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.52:55842/0 (socket says 172.21.15.52:55842) 2024-03-12T17:07:00.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.146+0000 7f29c5200640 1 -- 172.21.15.52:0/801340088 learned_addr learned my addr 172.21.15.52:0/801340088 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:00.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.146+0000 7f354fe00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35500d1380 0x7f355014deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.52:55832/0 (socket says 172.21.15.52:55832) 2024-03-12T17:07:00.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.147+0000 7f354fe00640 1 -- 172.21.15.52:0/2572493406 learned_addr learned my addr 172.21.15.52:0/2572493406 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:00.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.147+0000 7f29bfe00640 1 --2- 172.21.15.52:0/801340088 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29c00b2430 0x7f29c00c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.147+0000 7f29bf400640 1 --2- 172.21.15.52:0/801340088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29c00c57d0 0x7f29c00c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.147+0000 7f3556000640 1 -- 172.21.15.52:0/2572493406 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35500c57b0 con 0x7f35500cd1c0 2024-03-12T17:07:00.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.147+0000 7f3556000640 1 -- 172.21.15.52:0/2572493406 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35500c9c50 con 0x7f355015c830 2024-03-12T17:07:00.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.147+0000 7f3555600640 1 --2- 172.21.15.52:0/2572493406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f355015c830 0x7f355014e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.147+0000 7f29c5200640 1 -- 172.21.15.52:0/801340088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29c00c57d0 msgr2=0x7f29c00c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.148+0000 7f3556000640 1 -- 172.21.15.52:0/2572493406 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35500caf80 con 0x7f35500d1380 2024-03-12T17:07:00.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.147+0000 7f29c5200640 1 --2- 172.21.15.52:0/801340088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29c00c57d0 0x7f29c00c6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.147+0000 7f29c5200640 1 -- 172.21.15.52:0/801340088 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29c00b2430 msgr2=0x7f29c00c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.147+0000 7f29c5200640 1 --2- 172.21.15.52:0/801340088 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29c00b2430 0x7f29c00c79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.147+0000 7f29c5200640 1 -- 172.21.15.52:0/801340088 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ac009400 con 0x7f29c00cf0e0 2024-03-12T17:07:00.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.148+0000 7f29bf400640 1 --2- 172.21.15.52:0/801340088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29c00c57d0 0x7f29c00c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:07:00.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.148+0000 7f29c5200640 1 --2- 172.21.15.52:0/801340088 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f29c00cf0e0 0x7f29c00cc410 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f29b000c480 tx=0x7f29b000c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:00.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.149+0000 7f29bfe00640 1 --2- 172.21.15.52:0/801340088 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29c00b2430 0x7f29c00c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:00.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.149+0000 7f29b7e00640 1 -- 172.21.15.52:0/801340088 <== mon.2 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b00035f0 con 0x7f29c00cf0e0 2024-03-12T17:07:00.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.149+0000 7f3555600640 1 -- 172.21.15.52:0/2572493406 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35500d1380 msgr2=0x7f355014deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.149+0000 7f3555600640 1 --2- 172.21.15.52:0/2572493406 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35500d1380 0x7f355014deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.149+0000 7f3555600640 1 -- 172.21.15.52:0/2572493406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35500cd1c0 msgr2=0x7f35500c5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.149+0000 7f3555600640 1 --2- 172.21.15.52:0/2572493406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35500cd1c0 0x7f35500c5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.149+0000 7f3555600640 1 -- 172.21.15.52:0/2572493406 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3548009a70 con 0x7f355015c830 2024-03-12T17:07:00.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.150+0000 7f3555600640 1 --2- 172.21.15.52:0/2572493406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f355015c830 0x7f355014e3f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f354000e7c0 tx=0x7f354000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:00.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.149+0000 7f29b7e00640 1 -- 172.21.15.52:0/801340088 <== mon.2 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b0003750 con 0x7f29c00cf0e0 2024-03-12T17:07:00.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.149+0000 7f29b7e00640 1 -- 172.21.15.52:0/801340088 <== mon.2 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b0003b40 con 0x7f29c00cf0e0 2024-03-12T17:07:00.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.151+0000 7f29c7200640 1 -- 172.21.15.52:0/801340088 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c00c6ee0 con 0x7f29c00cf0e0 2024-03-12T17:07:00.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.151+0000 7f29c7200640 1 -- 172.21.15.52:0/801340088 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c00c74a0 con 0x7f29c00cf0e0 2024-03-12T17:07:00.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.151+0000 7f354cc00640 1 -- 172.21.15.52:0/2572493406 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354000eed0 con 0x7f355015c830 2024-03-12T17:07:00.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.151+0000 7f354cc00640 1 -- 172.21.15.52:0/2572493406 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354000c8e0 con 0x7f355015c830 2024-03-12T17:07:00.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.151+0000 7f354cc00640 1 -- 172.21.15.52:0/2572493406 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354000cc90 con 0x7f355015c830 2024-03-12T17:07:00.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.151+0000 7f354fe00640 1 --2- 172.21.15.52:0/2572493406 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35500d1380 0x7f355014deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:00.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.151+0000 7f3556000640 1 -- 172.21.15.52:0/2572493406 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f355006b060 con 0x7f355015c830 2024-03-12T17:07:00.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.153+0000 7f29b5600640 1 -- 172.21.15.52:0/801340088 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c0073440 con 0x7f29c00cf0e0 2024-03-12T17:07:00.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.153+0000 7f3556000640 1 -- 172.21.15.52:0/2572493406 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35500cac10 con 0x7f355015c830 2024-03-12T17:07:00.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.157+0000 7f3556000640 1 -- 172.21.15.52:0/2572493406 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f355004e030 con 0x7f355015c830 2024-03-12T17:07:00.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.157+0000 7f354cc00640 1 -- 172.21.15.52:0/2572493406 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3540013470 con 0x7f355015c830 2024-03-12T17:07:00.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.156+0000 7f29b7e00640 1 -- 172.21.15.52:0/801340088 <== mon.2 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f29b0003ca0 con 0x7f29c00cf0e0 2024-03-12T17:07:00.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.158+0000 7f354cc00640 1 --2- 172.21.15.52:0/2572493406 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f353006acd0 0x7f353006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.158+0000 7f29b7e00640 1 --2- 172.21.15.52:0/801340088 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f29a006acd0 0x7f29a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.159+0000 7f29b7e00640 1 -- 172.21.15.52:0/801340088 <== mon.2 v2:172.21.15.102:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f29b0088ba0 con 0x7f29c00cf0e0 2024-03-12T17:07:00.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.159+0000 7f354cc00640 1 -- 172.21.15.52:0/2572493406 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f35400896a0 con 0x7f355015c830 2024-03-12T17:07:00.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.159+0000 7f3554c00640 1 --2- 172.21.15.52:0/2572493406 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f353006acd0 0x7f353006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.160+0000 7f29bfe00640 1 --2- 172.21.15.52:0/801340088 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f29a006acd0 0x7f29a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.161+0000 7f3554c00640 1 --2- 172.21.15.52:0/2572493406 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f353006acd0 0x7f353006d190 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3548036000 tx=0x7f3548030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:00.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.161+0000 7f29bfe00640 1 --2- 172.21.15.52:0/801340088 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f29a006acd0 0x7f29a006d190 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f29b8001cb0 tx=0x7f29b8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:00.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.171+0000 7f29b7e00640 1 -- 172.21.15.52:0/801340088 <== mon.2 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f29b0053bb0 con 0x7f29c00cf0e0 2024-03-12T17:07:00.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.174+0000 7f354cc00640 1 -- 172.21.15.52:0/2572493406 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f35400546b0 con 0x7f355015c830 2024-03-12T17:07:00.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.190+0000 7f3c5b800640 1 Processor -- start 2024-03-12T17:07:00.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.190+0000 7f3c5b800640 1 -- start start 2024-03-12T17:07:00.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.192+0000 7f3c5b800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c5415cda0 0x7f3c540cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.192+0000 7f3c5b800640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f3c540b4a60 0x7f3c540b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.192+0000 7f3c5b800640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c540d1830 con 0x7f3c540b5a30 2024-03-12T17:07:00.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.192+0000 7f3c5b800640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c540b1f20 con 0x7f3c540b4a60 2024-03-12T17:07:00.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.192+0000 7f3c5b800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c540d0c20 con 0x7f3c5415cda0 2024-03-12T17:07:00.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.193+0000 7f3c58e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c5415cda0 0x7f3c540cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.193+0000 7f3c58e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c5415cda0 0x7f3c540cdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.52:55846/0 (socket says 172.21.15.52:55846) 2024-03-12T17:07:00.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.193+0000 7f3c58e00640 1 -- 172.21.15.52:0/1237845173 learned_addr learned my addr 172.21.15.52:0/1237845173 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:00.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.193+0000 7f3c59800640 1 --2- 172.21.15.52:0/1237845173 >> v2:172.21.15.52:3300/0 conn(0x7f3c540b4a60 0x7f3c540b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.194+0000 7f546a800640 1 Processor -- start 2024-03-12T17:07:00.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.194+0000 7f546a800640 1 -- start start 2024-03-12T17:07:00.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.195+0000 7f3c58e00640 1 -- 172.21.15.52:0/1237845173 >> v2:172.21.15.52:3300/0 conn(0x7f3c540b4a60 msgr2=0x7f3c540b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:00.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.195+0000 7f3c58e00640 1 --2- 172.21.15.52:0/1237845173 >> v2:172.21.15.52:3300/0 conn(0x7f3c540b4a60 0x7f3c540b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.195+0000 7f3c58e00640 1 -- 172.21.15.52:0/1237845173 >> v1:172.21.15.52:6789/0 conn(0x7f3c540b5a30 legacy=0x7f3c540b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.195+0000 7f3c53400640 1 -- 172.21.15.52:0/1237845173 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1157987957 0 0) 0x7f3c540d1830 con 0x7f3c540b5a30 2024-03-12T17:07:00.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.195+0000 7f3c58e00640 1 -- 172.21.15.52:0/1237845173 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c540d0f00 con 0x7f3c5415cda0 2024-03-12T17:07:00.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.195+0000 7f3c59800640 1 --2- 172.21.15.52:0/1237845173 >> v2:172.21.15.52:3300/0 conn(0x7f3c540b4a60 0x7f3c540b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T17:07:00.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.197+0000 7f546a800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f54640cc960 0x7f54640cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.198+0000 7fb139000640 1 Processor -- start 2024-03-12T17:07:00.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.198+0000 7f546a800640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54640ce310 con 0x7f5464141980 2024-03-12T17:07:00.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.198+0000 7f3c58e00640 1 --2- 172.21.15.52:0/1237845173 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c5415cda0 0x7f3c540cdb40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3c48009870 tx=0x7f3c4802b9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c31d17ebc9a9c533 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:00.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.198+0000 7f5468a00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f5464141980 0x7f54640ccd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55022/0 (socket says 172.21.15.52:55022) 2024-03-12T17:07:00.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.198+0000 7f5468a00640 1 -- 172.21.15.52:0/3475434608 learned_addr learned my addr 172.21.15.52:0/3475434608 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:00.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.198+0000 7f546a800640 1 -- 172.21.15.52:0/3475434608 --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f546406dbf0 con 0x7f54640b2aa0 2024-03-12T17:07:00.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.198+0000 7f5469400640 1 --2- 172.21.15.52:0/3475434608 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f54640cc960 0x7f54640cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.199+0000 7f546a800640 1 -- 172.21.15.52:0/3475434608 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5464141820 con 0x7f54640cc960 2024-03-12T17:07:00.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.198+0000 7fb139000640 1 -- start start 2024-03-12T17:07:00.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.199+0000 7f5463e00640 1 -- 172.21.15.52:0/3475434608 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1927271796 0 0) 0x7f54640ce310 con 0x7f5464141980 2024-03-12T17:07:00.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.199+0000 7f5463e00640 1 -- 172.21.15.52:0/3475434608 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f544c005730 con 0x7f5464141980 2024-03-12T17:07:00.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.200+0000 7f5463e00640 1 -- 172.21.15.52:0/3475434608 <== mon.1 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 911483650 0 0) 0x7f546406dbf0 con 0x7f54640b2aa0 2024-03-12T17:07:00.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.200+0000 7f5463e00640 1 -- 172.21.15.52:0/3475434608 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54640ce310 con 0x7f54640b2aa0 2024-03-12T17:07:00.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.200+0000 7f5469400640 1 -- 172.21.15.52:0/3475434608 >> v1:172.21.15.102:6789/0 conn(0x7f54640b2aa0 legacy=0x7f54641622c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.200+0000 7f5469400640 1 -- 172.21.15.52:0/3475434608 >> v1:172.21.15.52:6789/0 conn(0x7f5464141980 legacy=0x7f54640ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.200+0000 7f5469400640 1 -- 172.21.15.52:0/3475434608 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f546406de90 con 0x7f54640cc960 2024-03-12T17:07:00.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.200+0000 7fb139000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb1340cada0 0x7fb134159870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.201+0000 7f3c53400640 1 -- 172.21.15.52:0/1237845173 <== mon.2 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c48030860 con 0x7f3c5415cda0 2024-03-12T17:07:00.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.201+0000 7fb132a00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fb134158990 0x7fb134070020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.52:35878/0 (socket says 172.21.15.52:35878) 2024-03-12T17:07:00.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.201+0000 7f3c53400640 1 -- 172.21.15.52:0/1237845173 <== mon.2 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c480309c0 con 0x7f3c5415cda0 2024-03-12T17:07:00.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.201+0000 7f3c53400640 1 -- 172.21.15.52:0/1237845173 <== mon.2 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c48030d90 con 0x7f3c5415cda0 2024-03-12T17:07:00.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.201+0000 7fb133e00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fb134070730 0x7fb134163750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55038/0 (socket says 172.21.15.52:55038) 2024-03-12T17:07:00.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.201+0000 7fb139000640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb13406cf50 con 0x7fb134070730 2024-03-12T17:07:00.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.201+0000 7fb139000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb13415c7b0 con 0x7fb134158990 2024-03-12T17:07:00.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.201+0000 7fb139000640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb134159db0 con 0x7fb1340cada0 2024-03-12T17:07:00.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.201+0000 7fb132a00640 1 -- 172.21.15.52:0/574988871 learned_addr learned my addr 172.21.15.52:0/574988871 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:00.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.202+0000 7fb132000640 1 -- 172.21.15.52:0/574988871 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 553005092 0 0) 0x7fb13406cf50 con 0x7fb134070730 2024-03-12T17:07:00.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.202+0000 7fb132000640 1 -- 172.21.15.52:0/574988871 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb118003520 con 0x7fb134070730 2024-03-12T17:07:00.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.202+0000 7f3c5b800640 1 -- 172.21.15.52:0/1237845173 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c5415cda0 msgr2=0x7f3c540cdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.202+0000 7f3c5b800640 1 --2- 172.21.15.52:0/1237845173 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c5415cda0 0x7f3c540cdb40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3c48009870 tx=0x7f3c4802b9a0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.202+0000 7fb132000640 1 -- 172.21.15.52:0/574988871 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2707464064 0 0) 0x7fb118003520 con 0x7fb134070730 2024-03-12T17:07:00.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.202+0000 7fb132000640 1 -- 172.21.15.52:0/574988871 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb1340cada0 msgr2=0x7fb134159870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-12T17:07:00.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.202+0000 7fb132000640 1 --2- 172.21.15.52:0/574988871 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb1340cada0 0x7fb134159870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.202+0000 7fb132000640 1 -- 172.21.15.52:0/574988871 >> v1:172.21.15.102:6789/0 conn(0x7fb134158990 legacy=0x7fb134070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.202+0000 7f3c5b800640 1 -- 172.21.15.52:0/1237845173 shutdown_connections 2024-03-12T17:07:00.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.202+0000 7f3c5b800640 1 --2- 172.21.15.52:0/1237845173 >> v2:172.21.15.52:3300/0 conn(0x7f3c540b4a60 0x7f3c540b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.202+0000 7fb132000640 1 -- 172.21.15.52:0/574988871 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb13415ca90 con 0x7fb134070730 2024-03-12T17:07:00.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.202+0000 7f3c5b800640 1 --2- 172.21.15.52:0/1237845173 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c5415cda0 0x7f3c540cdb40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.202+0000 7f3c5b800640 1 -- 172.21.15.52:0/1237845173 >> 172.21.15.52:0/1237845173 conn(0x7f3c540cf350 msgr2=0x7f3c540cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:00.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.203+0000 7f3c5b800640 1 -- 172.21.15.52:0/1237845173 shutdown_connections 2024-03-12T17:07:00.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.203+0000 7fb132000640 1 -- 172.21.15.52:0/574988871 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7fb120001670 con 0x7fb134070730 2024-03-12T17:07:00.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.203+0000 7fb132000640 1 -- 172.21.15.52:0/574988871 >> v1:172.21.15.52:6789/0 conn(0x7fb134070730 legacy=0x7fb134163750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.203+0000 7f3c5b800640 1 -- 172.21.15.52:0/1237845173 wait complete. 2024-03-12T17:07:00.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.203+0000 7f5469400640 1 --2- 172.21.15.52:0/3475434608 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f54640cc960 0x7f54640cddd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f545c009dc0 tx=0x7f545c02bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1287f3f66d23fd6a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:00.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.204+0000 7f5463e00640 1 -- 172.21.15.52:0/3475434608 <== mon.2 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545c02cce0 con 0x7f54640cc960 2024-03-12T17:07:00.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.204+0000 7f5463e00640 1 -- 172.21.15.52:0/3475434608 <== mon.2 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f545c02ce40 con 0x7f54640cc960 2024-03-12T17:07:00.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.204+0000 7f5463e00640 1 -- 172.21.15.52:0/3475434608 <== mon.2 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545c034660 con 0x7f54640cc960 2024-03-12T17:07:00.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.203+0000 7fb132000640 1 --2- 172.21.15.52:0/574988871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1180039a0 0x7fb118003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.203+0000 7fb132000640 1 --2- 172.21.15.52:0/574988871 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb1340cada0 0x7fb1180047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.203+0000 7fb132000640 1 --2- 172.21.15.52:0/574988871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb118004e20 0x7fb11800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.203+0000 7fb132000640 1 -- 172.21.15.52:0/574988871 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1180054f0 con 0x7fb1180039a0 2024-03-12T17:07:00.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.203+0000 7fb132000640 1 -- 172.21.15.52:0/574988871 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb118005660 con 0x7fb118004e20 2024-03-12T17:07:00.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.203+0000 7fb132000640 1 -- 172.21.15.52:0/574988871 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb11800bf10 con 0x7fb1340cada0 2024-03-12T17:07:00.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.204+0000 7fb133400640 1 --2- 172.21.15.52:0/574988871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1180039a0 0x7fb118003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.204+0000 7fb133e00640 1 --2- 172.21.15.52:0/574988871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb118004e20 0x7fb11800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.204+0000 7fb132a00640 1 --2- 172.21.15.52:0/574988871 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb1340cada0 0x7fb1180047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.205+0000 7f546a800640 1 -- 172.21.15.52:0/3475434608 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f54640cc960 msgr2=0x7f54640cddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.205+0000 7f546a800640 1 --2- 172.21.15.52:0/3475434608 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f54640cc960 0x7f54640cddd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f545c009dc0 tx=0x7f545c02bf30 comp rx=0 tx=0).stop 2024-03-12T17:07:00.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.205+0000 7f546a800640 1 -- 172.21.15.52:0/3475434608 shutdown_connections 2024-03-12T17:07:00.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.205+0000 7f546a800640 1 --2- 172.21.15.52:0/3475434608 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f54640cc960 0x7f54640cddd0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.206+0000 7f546a800640 1 -- 172.21.15.52:0/3475434608 >> 172.21.15.52:0/3475434608 conn(0x7f54640700d0 msgr2=0x7f54640704b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:00.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.205+0000 7fb133400640 1 -- 172.21.15.52:0/574988871 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb1340cada0 msgr2=0x7fb1180047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:00.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.205+0000 7fb133400640 1 --2- 172.21.15.52:0/574988871 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb1340cada0 0x7fb1180047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.205+0000 7fb133400640 1 -- 172.21.15.52:0/574988871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb118004e20 msgr2=0x7fb11800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:00.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.205+0000 7fb133400640 1 --2- 172.21.15.52:0/574988871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb118004e20 0x7fb11800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.205+0000 7fb133400640 1 -- 172.21.15.52:0/574988871 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb118003520 con 0x7fb1180039a0 2024-03-12T17:07:00.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.206+0000 7f3c5b800640 1 Processor -- start 2024-03-12T17:07:00.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.207+0000 7f546a800640 1 -- 172.21.15.52:0/3475434608 shutdown_connections 2024-03-12T17:07:00.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.207+0000 7fb133400640 1 --2- 172.21.15.52:0/574988871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1180039a0 0x7fb118003dc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb12c009e60 tx=0x7fb1180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2885176ad174f125 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:00.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.207+0000 7f3c5b800640 1 -- start start 2024-03-12T17:07:00.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.208+0000 7f546a800640 1 -- 172.21.15.52:0/3475434608 wait complete. 2024-03-12T17:07:00.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.208+0000 7fb132000640 1 -- 172.21.15.52:0/574988871 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c02cdb0 con 0x7fb1180039a0 2024-03-12T17:07:00.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.208+0000 7fb132000640 1 -- 172.21.15.52:0/574988871 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb12c034440 con 0x7fb1180039a0 2024-03-12T17:07:00.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.208+0000 7fb132000640 1 -- 172.21.15.52:0/574988871 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c0348b0 con 0x7fb1180039a0 2024-03-12T17:07:00.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.208+0000 7f3c5b800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3c540b4a60 0x7f3c5415a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.208+0000 7f3c5b800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c540b5a30 0x7f3c540c8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.208+0000 7f3c5b800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c5415cda0 0x7f3c540c8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.208+0000 7f3c5b800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c541f2d30 con 0x7f3c540b4a60 2024-03-12T17:07:00.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.208+0000 7f3c5b800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c541f1df0 con 0x7f3c540b5a30 2024-03-12T17:07:00.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.208+0000 7f3c5b800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c541f20b0 con 0x7f3c5415cda0 2024-03-12T17:07:00.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.209+0000 7f3c58e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3c540b4a60 0x7f3c5415a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.209+0000 7f3c53e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c540b5a30 0x7f3c540c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.209+0000 7f3c58e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3c540b4a60 0x7f3c5415a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:32852/0 (socket says 172.21.15.52:32852) 2024-03-12T17:07:00.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.209+0000 7f3c58e00640 1 -- 172.21.15.52:0/2086889712 learned_addr learned my addr 172.21.15.52:0/2086889712 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:00.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.209+0000 7fb139000640 1 -- 172.21.15.52:0/574988871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1180039a0 msgr2=0x7fb118003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.209+0000 7fb139000640 1 --2- 172.21.15.52:0/574988871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1180039a0 0x7fb118003dc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb12c009e60 tx=0x7fb1180057f0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.209+0000 7fb133e00640 1 -- 172.21.15.52:0/574988871 reap_dead start 2024-03-12T17:07:00.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.209+0000 7f8bd6a00640 1 Processor -- start 2024-03-12T17:07:00.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.209+0000 7fb139000640 1 -- 172.21.15.52:0/574988871 shutdown_connections 2024-03-12T17:07:00.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.209+0000 7fb139000640 1 -- 172.21.15.52:0/574988871 >> 172.21.15.52:0/574988871 conn(0x7fb134069e70 msgr2=0x7fb13406a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:00.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.210+0000 7fb139000640 1 -- 172.21.15.52:0/574988871 shutdown_connections 2024-03-12T17:07:00.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.209+0000 7f3c59800640 1 --2- 172.21.15.52:0/2086889712 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c5415cda0 0x7f3c540c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.210+0000 7fb139000640 1 -- 172.21.15.52:0/574988871 wait complete. 2024-03-12T17:07:00.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.210+0000 7f3c58e00640 1 -- 172.21.15.52:0/2086889712 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c5415cda0 msgr2=0x7f3c540c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.210+0000 7f3c58e00640 1 --2- 172.21.15.52:0/2086889712 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c5415cda0 0x7f3c540c8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.210+0000 7f3c58e00640 1 -- 172.21.15.52:0/2086889712 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c540b5a30 msgr2=0x7f3c540c8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.210+0000 7f3c58e00640 1 --2- 172.21.15.52:0/2086889712 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c540b5a30 0x7f3c540c8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.210+0000 7f3c58e00640 1 -- 172.21.15.52:0/2086889712 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c4c009a70 con 0x7f3c540b4a60 2024-03-12T17:07:00.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.211+0000 7f8bd6a00640 1 -- start start 2024-03-12T17:07:00.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.211+0000 7f3c58e00640 1 --2- 172.21.15.52:0/2086889712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3c540b4a60 0x7f3c5415a750 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3c48000c00 tx=0x7f3c48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:00.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.211+0000 7f3c50c00640 1 -- 172.21.15.52:0/2086889712 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c48039a90 con 0x7f3c540b4a60 2024-03-12T17:07:00.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.212+0000 7f3c50c00640 1 -- 172.21.15.52:0/2086889712 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c48039c30 con 0x7f3c540b4a60 2024-03-12T17:07:00.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.212+0000 7f3c50c00640 1 -- 172.21.15.52:0/2086889712 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c480433f0 con 0x7f3c540b4a60 2024-03-12T17:07:00.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.212+0000 7f3c5b800640 1 -- 172.21.15.52:0/2086889712 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c480094d0 con 0x7f3c540b4a60 2024-03-12T17:07:00.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.212+0000 7f3c5b800640 1 -- 172.21.15.52:0/2086889712 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c541f2670 con 0x7f3c540b4a60 2024-03-12T17:07:00.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.212+0000 7f546a800640 1 Processor -- start 2024-03-12T17:07:00.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.212+0000 7f546a800640 1 -- start start 2024-03-12T17:07:00.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.212+0000 7f8bd6a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8bd015fd10 0x7f8bd00b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.212+0000 7f8bd6a00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f8bd00b8360 0x7f8bd00cd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.212+0000 7f8bd6a00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bd00b7bc0 con 0x7f8bd00cd940 2024-03-12T17:07:00.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.213+0000 7f8bd6a00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bd00cdd50 con 0x7f8bd00b8360 2024-03-12T17:07:00.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.213+0000 7f8bd6a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bd01600f0 con 0x7f8bd015fd10 2024-03-12T17:07:00.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.213+0000 7f8bd4c00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f8bd00b8360 0x7f8bd00cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.213+0000 7f8bd4c00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f8bd00b8360 0x7f8bd00cd400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:32856/0 (socket says 172.21.15.52:32856) 2024-03-12T17:07:00.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.213+0000 7f8bd4c00640 1 -- 172.21.15.52:0/28968140 learned_addr learned my addr 172.21.15.52:0/28968140 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:00.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.213+0000 7f8bd5600640 1 --2- 172.21.15.52:0/28968140 >> v2:172.21.15.102:3300/0 conn(0x7f8bd015fd10 0x7f8bd00b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.213+0000 7fb139000640 1 Processor -- start 2024-03-12T17:07:00.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.214+0000 7ff100a00640 1 Processor -- start 2024-03-12T17:07:00.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.213+0000 7fb139000640 1 -- start start 2024-03-12T17:07:00.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.214+0000 7ff100a00640 1 -- start start 2024-03-12T17:07:00.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.214+0000 7f8bcfe00640 1 -- 172.21.15.52:0/28968140 <== mon.1 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1567489198 0 0) 0x7f8bd00b7bc0 con 0x7f8bd00cd940 2024-03-12T17:07:00.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.214+0000 7f8bcfe00640 1 -- 172.21.15.52:0/28968140 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bb8003520 con 0x7f8bd00cd940 2024-03-12T17:07:00.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.214+0000 7fa2ee600640 1 Processor -- start 2024-03-12T17:07:00.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.215+0000 7f3c5b800640 1 -- 172.21.15.52:0/2086889712 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c54073390 con 0x7f3c540b4a60 2024-03-12T17:07:00.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.215+0000 7f8bcfe00640 1 -- 172.21.15.52:0/28968140 <== mon.1 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1469529100 0 0) 0x7f8bb8003520 con 0x7f8bd00cd940 2024-03-12T17:07:00.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.215+0000 7f8bcfe00640 1 -- 172.21.15.52:0/28968140 >> v2:172.21.15.102:3300/0 conn(0x7f8bd015fd10 msgr2=0x7f8bd00b62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:00.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.215+0000 7f8bcfe00640 1 --2- 172.21.15.52:0/28968140 >> v2:172.21.15.102:3300/0 conn(0x7f8bd015fd10 0x7f8bd00b62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.215+0000 7f8bcfe00640 1 -- 172.21.15.52:0/28968140 >> v2:172.21.15.52:3300/0 conn(0x7f8bd00b8360 msgr2=0x7f8bd00cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:00.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.215+0000 7f8bcfe00640 1 --2- 172.21.15.52:0/28968140 >> v2:172.21.15.52:3300/0 conn(0x7f8bd00b8360 0x7f8bd00cd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.215+0000 7f8bcfe00640 1 -- 172.21.15.52:0/28968140 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd00b6b20 con 0x7f8bd00cd940 2024-03-12T17:07:00.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.215+0000 7f8bcfe00640 1 -- 172.21.15.52:0/28968140 <== mon.1 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f8bc80019d0 con 0x7f8bd00cd940 2024-03-12T17:07:00.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.215+0000 7f8bcfe00640 1 -- 172.21.15.52:0/28968140 >> v1:172.21.15.102:6789/0 conn(0x7f8bd00cd940 legacy=0x7f8bd00b41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.215+0000 7f546a800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54640b2aa0 0x7f546413f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.215+0000 7fb139000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb1180039a0 0x7fb1340ac3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.215+0000 7fb139000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb134156a20 0x7fb134157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.215+0000 7fb139000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb134157e40 0x7fb1342025d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.215+0000 7fb139000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb134203320 con 0x7fb134157e40 2024-03-12T17:07:00.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.215+0000 7fb139000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1341582c0 con 0x7fb134156a20 2024-03-12T17:07:00.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.215+0000 7fb139000640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1340ab360 con 0x7fb1180039a0 2024-03-12T17:07:00.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.215+0000 7f8bcfe00640 1 --2- 172.21.15.52:0/28968140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8bb8003990 0x7f8bb8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.215+0000 7f8bcfe00640 1 --2- 172.21.15.52:0/28968140 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8bb80047c0 0x7f8bb8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.215+0000 7f8bcfe00640 1 --2- 172.21.15.52:0/28968140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bb8005180 0x7f8bb800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.215+0000 7f8bcfe00640 1 -- 172.21.15.52:0/28968140 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd00cdd50 con 0x7f8bb8003990 2024-03-12T17:07:00.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.215+0000 7f8bcfe00640 1 -- 172.21.15.52:0/28968140 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd01600f0 con 0x7f8bb8005180 2024-03-12T17:07:00.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.215+0000 7f8bcfe00640 1 -- 172.21.15.52:0/28968140 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb800c080 con 0x7f8bb80047c0 2024-03-12T17:07:00.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.216+0000 7fb133e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb134157e40 0x7fb1342025d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.216+0000 7fb133e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb134157e40 0x7fb1342025d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:32872/0 (socket says 172.21.15.52:32872) 2024-03-12T17:07:00.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.216+0000 7fb133e00640 1 -- 172.21.15.52:0/1185494491 learned_addr learned my addr 172.21.15.52:0/1185494491 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:00.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.216+0000 7f8bd5600640 1 --2- 172.21.15.52:0/28968140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8bb8003990 0x7f8bb8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.216+0000 7ff100a00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff0fc14aa40 0x7ff0fc0cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.216+0000 7ff100a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff0fc0cc7b0 0x7ff0fc0ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.216+0000 7ff100a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0fc0b27e0 0x7ff0fc0b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.215+0000 7fa2ee600640 1 -- start start 2024-03-12T17:07:00.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.216+0000 7ff100a00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0fc06da70 con 0x7ff0fc14aa40 2024-03-12T17:07:00.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.216+0000 7ff100a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0fc0b1d00 con 0x7ff0fc0cc7b0 2024-03-12T17:07:00.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.216+0000 7ff100a00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc0b1fe0 con 0x7ff0fc0b27e0 2024-03-12T17:07:00.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.216+0000 7f5469400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54640b2aa0 0x7f546413f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.216+0000 7f5469400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54640b2aa0 0x7f546413f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:32886/0 (socket says 172.21.15.52:32886) 2024-03-12T17:07:00.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.216+0000 7f5469400640 1 -- 172.21.15.52:0/3175683879 learned_addr learned my addr 172.21.15.52:0/3175683879 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:00.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.216+0000 7fb132a00640 1 --2- 172.21.15.52:0/1185494491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb134156a20 0x7fb134157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.216+0000 7fb133400640 1 --2- 172.21.15.52:0/1185494491 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb1180039a0 0x7fb1340ac3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.217+0000 7f546a800640 1 --2- 172.21.15.52:0/3175683879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54640cc960 0x7f54641f9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.217+0000 7ff0f9400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff0fc0cc7b0 0x7ff0fc0ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.217+0000 7ff0f9400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff0fc0cc7b0 0x7ff0fc0ccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:38822/0 (socket says 172.21.15.52:38822) 2024-03-12T17:07:00.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.217+0000 7ff0f9400640 1 -- 172.21.15.52:0/2499711228 learned_addr learned my addr 172.21.15.52:0/2499711228 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:00.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.217+0000 7ff0fa800640 1 --2- 172.21.15.52:0/2499711228 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0fc0b27e0 0x7ff0fc0b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.216+0000 7f8bd6000640 1 --2- 172.21.15.52:0/28968140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bb8005180 0x7f8bb800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.217+0000 7f8bd5600640 1 -- 172.21.15.52:0/28968140 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8bb80047c0 msgr2=0x7f8bb8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:00.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.216+0000 7f8bd4c00640 1 --2- 172.21.15.52:0/28968140 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8bb80047c0 0x7f8bb8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.217+0000 7f3c50c00640 1 -- 172.21.15.52:0/2086889712 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3c48043550 con 0x7f3c540b4a60 2024-03-12T17:07:00.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.217+0000 7fb133400640 1 -- 172.21.15.52:0/1185494491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb134156a20 msgr2=0x7fb134157900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.217+0000 7f546a800640 1 --2- 172.21.15.52:0/3175683879 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5464141980 0x7f54641fa120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.217+0000 7fb133400640 1 --2- 172.21.15.52:0/1185494491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb134156a20 0x7fb134157900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.217+0000 7fb133400640 1 -- 172.21.15.52:0/1185494491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb134157e40 msgr2=0x7fb1342025d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.217+0000 7fb133400640 1 --2- 172.21.15.52:0/1185494491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb134157e40 0x7fb1342025d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.217+0000 7fb133400640 1 -- 172.21.15.52:0/1185494491 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1340ab5e0 con 0x7fb1180039a0 2024-03-12T17:07:00.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.217+0000 7fb132a00640 1 --2- 172.21.15.52:0/1185494491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb134156a20 0x7fb134157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:00.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.217+0000 7f8bd5600640 1 --2- 172.21.15.52:0/28968140 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8bb80047c0 0x7f8bb8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.217+0000 7f8bd5600640 1 -- 172.21.15.52:0/28968140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bb8005180 msgr2=0x7f8bb800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:00.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.217+0000 7f8bd5600640 1 --2- 172.21.15.52:0/28968140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bb8005180 0x7f8bb800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.217+0000 7f8bd5600640 1 -- 172.21.15.52:0/28968140 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd00b6b20 con 0x7f8bb8003990 2024-03-12T17:07:00.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.218+0000 7fb133e00640 1 --2- 172.21.15.52:0/1185494491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb134157e40 0x7fb1342025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T17:07:00.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.217+0000 7f5468a00640 1 --2- 172.21.15.52:0/3175683879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54640cc960 0x7f54641f9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.217+0000 7f8bd4c00640 1 --2- 172.21.15.52:0/28968140 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8bb80047c0 0x7f8bb8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:00.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.218+0000 7f8bd6000640 1 --2- 172.21.15.52:0/28968140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bb8005180 0x7f8bb800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:00.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.218+0000 7f8bd6000640 1 -- 172.21.15.52:0/28968140 reap_dead start 2024-03-12T17:07:00.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.218+0000 7f5469e00640 1 --2- 172.21.15.52:0/3175683879 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5464141980 0x7f54641fa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.218+0000 7fb133400640 1 --2- 172.21.15.52:0/1185494491 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb1180039a0 0x7fb1340ac3a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb118005630 tx=0x7fb12c004a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:00.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.219+0000 7f546a800640 1 -- 172.21.15.52:0/3175683879 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54640c9c30 con 0x7f54640b2aa0 2024-03-12T17:07:00.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.219+0000 7fa2ee600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa2e8160ee0 0x7fa2e80cad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.219+0000 7f546a800640 1 -- 172.21.15.52:0/3175683879 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54641401c0 con 0x7f54640cc960 2024-03-12T17:07:00.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.219+0000 7f546a800640 1 -- 172.21.15.52:0/3175683879 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f546413fc60 con 0x7f5464141980 2024-03-12T17:07:00.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.217+0000 7ff0f9400640 1 -- 172.21.15.52:0/2499711228 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0fc0b27e0 msgr2=0x7ff0fc0b2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:00.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.217+0000 7ff0f9400640 1 --2- 172.21.15.52:0/2499711228 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0fc0b27e0 0x7ff0fc0b2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.217+0000 7ff0f9400640 1 -- 172.21.15.52:0/2499711228 >> v2:172.21.15.52:3300/0 conn(0x7ff0fc14aa40 msgr2=0x7ff0fc0cd7a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-12T17:07:00.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.219+0000 7fa2ee600640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa2e80ccbd0 0x7fa2e80b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.219+0000 7f3c50c00640 1 --2- 172.21.15.52:0/2086889712 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f3c3406ac00 0x7f3c3406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.219+0000 7fa2ee600640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e80cb710 con 0x7fa2e80cb2b0 2024-03-12T17:07:00.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.219+0000 7f3c50c00640 1 -- 172.21.15.52:0/2086889712 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3c480ab6a0 con 0x7f3c540b4a60 2024-03-12T17:07:00.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.219+0000 7f8bd5600640 1 --2- 172.21.15.52:0/28968140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8bb8003990 0x7f8bb8003dd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009aa0 tx=0x7f8bb80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4faea78968392294 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:00.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.220+0000 7f3c53e00640 1 --2- 172.21.15.52:0/2086889712 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f3c3406ac00 0x7f3c3406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.220+0000 7f5469400640 1 -- 172.21.15.52:0/3175683879 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5464141980 msgr2=0x7f54641fa120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.220+0000 7f5469400640 1 --2- 172.21.15.52:0/3175683879 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5464141980 0x7f54641fa120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.220+0000 7f5469400640 1 -- 172.21.15.52:0/3175683879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54640cc960 msgr2=0x7f54641f9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.221+0000 7f5469400640 1 --2- 172.21.15.52:0/3175683879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54640cc960 0x7f54641f9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.221+0000 7f5469400640 1 -- 172.21.15.52:0/3175683879 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545c009a70 con 0x7f54640b2aa0 2024-03-12T17:07:00.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.220+0000 7fb12b400640 1 -- 172.21.15.52:0/1185494491 <== mon.2 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c03f420 con 0x7fb1180039a0 2024-03-12T17:07:00.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.220+0000 7fb12b400640 1 -- 172.21.15.52:0/1185494491 <== mon.2 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb12c03f5a0 con 0x7fb1180039a0 2024-03-12T17:07:00.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.220+0000 7fb12b400640 1 -- 172.21.15.52:0/1185494491 <== mon.2 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c03f950 con 0x7fb1180039a0 2024-03-12T17:07:00.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.220+0000 7fb139000640 1 -- 172.21.15.52:0/1185494491 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1340ab8c0 con 0x7fb1180039a0 2024-03-12T17:07:00.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.220+0000 7fb139000640 1 -- 172.21.15.52:0/1185494491 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1340abe80 con 0x7fb1180039a0 2024-03-12T17:07:00.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.219+0000 7ff0f9e00640 1 --2- 172.21.15.52:0/2499711228 >> v2:172.21.15.52:3300/0 conn(0x7ff0fc14aa40 0x7ff0fc0cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.219+0000 7ff0f9400640 1 --2- 172.21.15.52:0/2499711228 >> v2:172.21.15.52:3300/0 conn(0x7ff0fc14aa40 0x7ff0fc0cd7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.219+0000 7ff0f9400640 1 -- 172.21.15.52:0/2499711228 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0fc0b22c0 con 0x7ff0fc0cc7b0 2024-03-12T17:07:00.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.220+0000 7ff0fa800640 1 --2- 172.21.15.52:0/2499711228 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0fc0b27e0 0x7ff0fc0b2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:00.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.220+0000 7fa2ed200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa2e8160ee0 0x7fa2e80cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.220+0000 7fa2ed200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa2e8160ee0 0x7fa2e80cad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:38850/0 (socket says 172.21.15.52:38850) 2024-03-12T17:07:00.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.220+0000 7fa2ed200640 1 -- 172.21.15.52:0/1622274838 learned_addr learned my addr 172.21.15.52:0/1622274838 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:00.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.221+0000 7f5468a00640 1 --2- 172.21.15.52:0/3175683879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54640cc960 0x7f54641f9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T17:07:00.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.221+0000 7fa2edc00640 1 --2- 172.21.15.52:0/1622274838 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa2e80ccbd0 0x7fa2e80b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.221+0000 7fa2ee600640 1 -- 172.21.15.52:0/1622274838 --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2e80cd1b0 con 0x7fa2e8160ee0 2024-03-12T17:07:00.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.221+0000 7fa2ee600640 1 -- 172.21.15.52:0/1622274838 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e80cd490 con 0x7fa2e80ccbd0 2024-03-12T17:07:00.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.221+0000 7f3c53e00640 1 --2- 172.21.15.52:0/2086889712 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f3c3406ac00 0x7f3c3406d0c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3c540d38b0 tx=0x7f3c40009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:00.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.222+0000 7ff0f9400640 1 --2- 172.21.15.52:0/2499711228 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0fc0cc7b0 0x7ff0fc0ccc60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0b3ea0 tx=0x7ff0ec02b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e264bb9c7d230b19 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:00.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.222+0000 7f5469400640 1 --2- 172.21.15.52:0/3175683879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54640b2aa0 0x7f546413f720 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f545c009ee0 tx=0x7f545c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:00.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.222+0000 7fb139000640 1 -- 172.21.15.52:0/1185494491 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1340ac0a0 con 0x7fb1180039a0 2024-03-12T17:07:00.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.222+0000 7fa2e7400640 1 -- 172.21.15.52:0/1622274838 <== mon.1 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3183013193 0 0) 0x7fa2e80cb710 con 0x7fa2e80cb2b0 2024-03-12T17:07:00.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.222+0000 7fa2e7400640 1 -- 172.21.15.52:0/1622274838 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2cc003520 con 0x7fa2e80cb2b0 2024-03-12T17:07:00.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.222+0000 7fa2edc00640 1 -- 172.21.15.52:0/1622274838 >> v2:172.21.15.102:3300/0 conn(0x7fa2e8160ee0 msgr2=0x7fa2e80cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:00.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.222+0000 7fa2edc00640 1 --2- 172.21.15.52:0/1622274838 >> v2:172.21.15.102:3300/0 conn(0x7fa2e8160ee0 0x7fa2e80cad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.222+0000 7fa2edc00640 1 -- 172.21.15.52:0/1622274838 >> v1:172.21.15.102:6789/0 conn(0x7fa2e80cb2b0 legacy=0x7fa2e80cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.222+0000 7fa2edc00640 1 -- 172.21.15.52:0/1622274838 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e80b1ca0 con 0x7fa2e80ccbd0 2024-03-12T17:07:00.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.223+0000 7ff0f8a00640 1 -- 172.21.15.52:0/2499711228 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec031880 con 0x7ff0fc0cc7b0 2024-03-12T17:07:00.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.223+0000 7ff0f8a00640 1 -- 172.21.15.52:0/2499711228 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0ec0319e0 con 0x7ff0fc0cc7b0 2024-03-12T17:07:00.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.223+0000 7ff0f8a00640 1 -- 172.21.15.52:0/2499711228 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec031d70 con 0x7ff0fc0cc7b0 2024-03-12T17:07:00.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.224+0000 7fa2edc00640 1 --2- 172.21.15.52:0/1622274838 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa2e80ccbd0 0x7fa2e80b24a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00097a0 tx=0x7fa2e002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6c4b75c1eb470f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:00.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.224+0000 7ff100a00640 1 -- 172.21.15.52:0/2499711228 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0fc0cc7b0 msgr2=0x7ff0fc0ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.224+0000 7ff100a00640 1 --2- 172.21.15.52:0/2499711228 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0fc0cc7b0 0x7ff0fc0ccc60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0b3ea0 tx=0x7ff0ec02b920 comp rx=0 tx=0).stop 2024-03-12T17:07:00.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.224+0000 7ff100a00640 1 -- 172.21.15.52:0/2499711228 shutdown_connections 2024-03-12T17:07:00.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.224+0000 7ff100a00640 1 --2- 172.21.15.52:0/2499711228 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0fc0b27e0 0x7ff0fc0b2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.224+0000 7ff100a00640 1 --2- 172.21.15.52:0/2499711228 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0fc0cc7b0 0x7ff0fc0ccc60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.224+0000 7ff100a00640 1 --2- 172.21.15.52:0/2499711228 >> v2:172.21.15.52:3300/0 conn(0x7ff0fc14aa40 0x7ff0fc0cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.224+0000 7ff100a00640 1 -- 172.21.15.52:0/2499711228 >> 172.21.15.52:0/2499711228 conn(0x7ff0fc06a3d0 msgr2=0x7ff0fc0b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:00.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.224+0000 7ff100a00640 1 -- 172.21.15.52:0/2499711228 shutdown_connections 2024-03-12T17:07:00.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.224+0000 7ff100a00640 1 -- 172.21.15.52:0/2499711228 wait complete. 2024-03-12T17:07:00.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.225+0000 7f8bcfe00640 1 -- 172.21.15.52:0/28968140 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc402c6b0 con 0x7f8bb8003990 2024-03-12T17:07:00.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.225+0000 7f8bcfe00640 1 -- 172.21.15.52:0/28968140 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc402c810 con 0x7f8bb8003990 2024-03-12T17:07:00.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.225+0000 7f8bcfe00640 1 -- 172.21.15.52:0/28968140 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc402cbc0 con 0x7f8bb8003990 2024-03-12T17:07:00.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.225+0000 7fb12b400640 1 -- 172.21.15.52:0/1185494491 <== mon.2 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb12c03fab0 con 0x7fb1180039a0 2024-03-12T17:07:00.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.225+0000 7f3c50c00640 1 -- 172.21.15.52:0/2086889712 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f3c480765a0 con 0x7f3c540b4a60 2024-03-12T17:07:00.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.225+0000 7f5461600640 1 -- 172.21.15.52:0/3175683879 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545c03d600 con 0x7f54640b2aa0 2024-03-12T17:07:00.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.225+0000 7f5461600640 1 -- 172.21.15.52:0/3175683879 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f545c03d760 con 0x7f54640b2aa0 2024-03-12T17:07:00.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.225+0000 7f5461600640 1 -- 172.21.15.52:0/3175683879 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545c03db10 con 0x7f54640b2aa0 2024-03-12T17:07:00.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.226+0000 7f8bd6a00640 1 -- 172.21.15.52:0/28968140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8bb8003990 msgr2=0x7f8bb8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.226+0000 7f8bd6a00640 1 --2- 172.21.15.52:0/28968140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8bb8003990 0x7f8bb8003dd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009aa0 tx=0x7f8bb80057f0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.226+0000 7f546a800640 1 -- 172.21.15.52:0/3175683879 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54640aa100 con 0x7f54640b2aa0 2024-03-12T17:07:00.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.227+0000 7f546a800640 1 -- 172.21.15.52:0/3175683879 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54640aa680 con 0x7f54640b2aa0 2024-03-12T17:07:00.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.227+0000 7f8bd6a00640 1 -- 172.21.15.52:0/28968140 shutdown_connections 2024-03-12T17:07:00.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.227+0000 7f8bd6a00640 1 --2- 172.21.15.52:0/28968140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8bb8003990 0x7f8bb8003dd0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.227+0000 7f8bd6a00640 1 -- 172.21.15.52:0/28968140 >> 172.21.15.52:0/28968140 conn(0x7f8bd00717e0 msgr2=0x7f8bd0071bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:00.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.227+0000 7ff100a00640 1 Processor -- start 2024-03-12T17:07:00.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.227+0000 7f8bd6a00640 1 -- 172.21.15.52:0/28968140 shutdown_connections 2024-03-12T17:07:00.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.227+0000 7f8bd6a00640 1 -- 172.21.15.52:0/28968140 wait complete. 2024-03-12T17:07:00.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.227+0000 7fb12b400640 1 --2- 172.21.15.52:0/1185494491 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb11406ac50 0x7fb11406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.227+0000 7fb12b400640 1 -- 172.21.15.52:0/1185494491 <== mon.2 v2:172.21.15.102:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb12c0ac1a0 con 0x7fb1180039a0 2024-03-12T17:07:00.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.228+0000 7ff100a00640 1 -- start start 2024-03-12T17:07:00.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.228+0000 7f546a800640 1 -- 172.21.15.52:0/3175683879 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54640a74a0 con 0x7f54640b2aa0 2024-03-12T17:07:00.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.229+0000 7fb132a00640 1 --2- 172.21.15.52:0/1185494491 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb11406ac50 0x7fb11406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.229+0000 7ff100a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0fc0b27e0 0x7ff0fc160940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.229+0000 7ff100a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0fc0cc7b0 0x7ff0fc163360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.229+0000 7ff100a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0fc14aa40 0x7ff0fc1638a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.229+0000 7ff100a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc13d260 con 0x7ff0fc0b27e0 2024-03-12T17:07:00.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.229+0000 7ff100a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc139df0 con 0x7ff0fc0cc7b0 2024-03-12T17:07:00.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.229+0000 7ff100a00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc139b10 con 0x7ff0fc14aa40 2024-03-12T17:07:00.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.229+0000 7ff0f9e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0fc0b27e0 0x7ff0fc160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.229+0000 7ff0f9e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0fc0b27e0 0x7ff0fc160940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:32892/0 (socket says 172.21.15.52:32892) 2024-03-12T17:07:00.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.229+0000 7ff0f9e00640 1 -- 172.21.15.52:0/3207057098 learned_addr learned my addr 172.21.15.52:0/3207057098 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:00.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.230+0000 7ff0f9400640 1 --2- 172.21.15.52:0/3207057098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0fc0cc7b0 0x7ff0fc163360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.230+0000 7ff0fa800640 1 --2- 172.21.15.52:0/3207057098 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0fc14aa40 0x7ff0fc1638a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.230+0000 7ff0f9e00640 1 -- 172.21.15.52:0/3207057098 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0fc14aa40 msgr2=0x7ff0fc1638a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.230+0000 7ff0f9e00640 1 --2- 172.21.15.52:0/3207057098 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0fc14aa40 0x7ff0fc1638a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.230+0000 7ff0f9e00640 1 -- 172.21.15.52:0/3207057098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0fc0cc7b0 msgr2=0x7ff0fc163360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.230+0000 7ff0f9e00640 1 --2- 172.21.15.52:0/3207057098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0fc0cc7b0 0x7ff0fc163360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.230+0000 7ff0f9e00640 1 -- 172.21.15.52:0/3207057098 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ec0094d0 con 0x7ff0fc0b27e0 2024-03-12T17:07:00.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.230+0000 7ff0f9400640 1 --2- 172.21.15.52:0/3207057098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0fc0cc7b0 0x7ff0fc163360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:00.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.230+0000 7f8bd6a00640 1 Processor -- start 2024-03-12T17:07:00.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.230+0000 7f8bd6a00640 1 -- start start 2024-03-12T17:07:00.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.230+0000 7ff0f9e00640 1 --2- 172.21.15.52:0/3207057098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0fc0b27e0 0x7ff0fc160940 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff0f400ccd0 tx=0x7ff0f400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:00.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.231+0000 7f5461600640 1 -- 172.21.15.52:0/3175683879 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f545c03b040 con 0x7f54640b2aa0 2024-03-12T17:07:00.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.232+0000 7fb132a00640 1 --2- 172.21.15.52:0/1185494491 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb11406ac50 0x7fb11406d110 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb124000c00 tx=0x7fb124009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:00.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.232+0000 7f8bd6a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8bb8003990 0x7f8bd01fa360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.232+0000 7f8bd6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bd01f37f0 0x7f8bd01f7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.232+0000 7f8bd6a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8bd01f6260 0x7f8bd015c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.232+0000 7f8bd6a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd00cad90 con 0x7f8bb8003990 2024-03-12T17:07:00.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.232+0000 7f8bd6a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd01f7960 con 0x7f8bd01f37f0 2024-03-12T17:07:00.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.232+0000 7f8bd6a00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd00caa60 con 0x7f8bd01f6260 2024-03-12T17:07:00.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.232+0000 7f5461600640 1 --2- 172.21.15.52:0/3175683879 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f544406acd0 0x7f544406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.232+0000 7f8bd5600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8bb8003990 0x7f8bd01fa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.232+0000 7f8bd5600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8bb8003990 0x7f8bd01fa360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:32898/0 (socket says 172.21.15.52:32898) 2024-03-12T17:07:00.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.233+0000 7f5461600640 1 -- 172.21.15.52:0/3175683879 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f545c0ab6d0 con 0x7f54640b2aa0 2024-03-12T17:07:00.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.232+0000 7f8bd5600640 1 -- 172.21.15.52:0/3984085717 learned_addr learned my addr 172.21.15.52:0/3984085717 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:00.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.233+0000 7f8bd4c00640 1 --2- 172.21.15.52:0/3984085717 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bd01f37f0 0x7f8bd01f7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.233+0000 7f5468a00640 1 --2- 172.21.15.52:0/3175683879 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f544406acd0 0x7f544406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.233+0000 7f8bd5600640 1 -- 172.21.15.52:0/3984085717 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8bd01f6260 msgr2=0x7f8bd015c450 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-12T17:07:00.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.233+0000 7f8bd5600640 1 --2- 172.21.15.52:0/3984085717 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8bd01f6260 0x7f8bd015c450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.233+0000 7f8bd5600640 1 -- 172.21.15.52:0/3984085717 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bd01f37f0 msgr2=0x7f8bd01f7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.233+0000 7f8bd5600640 1 --2- 172.21.15.52:0/3984085717 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bd01f37f0 0x7f8bd01f7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.233+0000 7f8bd5600640 1 -- 172.21.15.52:0/3984085717 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd01f3c70 con 0x7f8bb8003990 2024-03-12T17:07:00.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.233+0000 7f8bd4c00640 1 --2- 172.21.15.52:0/3984085717 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bd01f37f0 0x7f8bd01f7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:00.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.234+0000 7f8bd5600640 1 --2- 172.21.15.52:0/3984085717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8bb8003990 0x7f8bd01fa360 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4037040 tx=0x7f8bc4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:00.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.234+0000 7f5468a00640 1 --2- 172.21.15.52:0/3175683879 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f544406acd0 0x7f544406d190 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f54641facc0 tx=0x7f5458006060 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:00.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.238+0000 7fa2e7400640 1 -- 172.21.15.52:0/1622274838 <== mon.2 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e002d040 con 0x7fa2e80ccbd0 2024-03-12T17:07:00.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.238+0000 7fa2e7400640 1 -- 172.21.15.52:0/1622274838 <== mon.2 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e0031810 con 0x7fa2e80ccbd0 2024-03-12T17:07:00.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.238+0000 7fa2e7400640 1 -- 172.21.15.52:0/1622274838 <== mon.2 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0031ba0 con 0x7fa2e80ccbd0 2024-03-12T17:07:00.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.239+0000 7fa2ee600640 1 -- 172.21.15.52:0/1622274838 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa2e80ccbd0 msgr2=0x7fa2e80b24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.239+0000 7fa2ee600640 1 --2- 172.21.15.52:0/1622274838 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa2e80ccbd0 0x7fa2e80b24a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00097a0 tx=0x7fa2e002b8b0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.239+0000 7fa2ee600640 1 -- 172.21.15.52:0/1622274838 shutdown_connections 2024-03-12T17:07:00.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.239+0000 7fa2ee600640 1 --2- 172.21.15.52:0/1622274838 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa2e80ccbd0 0x7fa2e80b24a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.239+0000 7fa2ee600640 1 --2- 172.21.15.52:0/1622274838 >> v2:172.21.15.102:3300/0 conn(0x7fa2e8160ee0 0x7fa2e80cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.239+0000 7fa2ee600640 1 -- 172.21.15.52:0/1622274838 >> 172.21.15.52:0/1622274838 conn(0x7fa2e8065e20 msgr2=0x7fa2e8069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:00.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.239+0000 7fb12b400640 1 -- 172.21.15.52:0/1185494491 <== mon.2 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fb12c0772b0 con 0x7fb1180039a0 2024-03-12T17:07:00.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.239+0000 7ff0f2000640 1 -- 172.21.15.52:0/3207057098 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4003990 con 0x7ff0fc0b27e0 2024-03-12T17:07:00.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.239+0000 7ff100a00640 1 -- 172.21.15.52:0/3207057098 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0fc136f10 con 0x7ff0fc0b27e0 2024-03-12T17:07:00.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.239+0000 7f8bcd600640 1 -- 172.21.15.52:0/3984085717 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4034c80 con 0x7f8bb8003990 2024-03-12T17:07:00.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.240+0000 7fa2ee600640 1 -- 172.21.15.52:0/1622274838 shutdown_connections 2024-03-12T17:07:00.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.240+0000 7f8bd6a00640 1 -- 172.21.15.52:0/3984085717 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd01f6a50 con 0x7f8bb8003990 2024-03-12T17:07:00.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.240+0000 7ff100a00640 1 -- 172.21.15.52:0/3207057098 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0fc1374d0 con 0x7ff0fc0b27e0 2024-03-12T17:07:00.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.240+0000 7f5461600640 1 -- 172.21.15.52:0/3175683879 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f54640a74a0 con 0x7f54640b2aa0 2024-03-12T17:07:00.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.240+0000 7f8bd6a00640 1 -- 172.21.15.52:0/3984085717 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd01f7010 con 0x7f8bb8003990 2024-03-12T17:07:00.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.240+0000 7fa2ee600640 1 -- 172.21.15.52:0/1622274838 wait complete. 2024-03-12T17:07:00.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.240+0000 7f8bcd600640 1 -- 172.21.15.52:0/3984085717 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc4034de0 con 0x7f8bb8003990 2024-03-12T17:07:00.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.240+0000 7f8bcd600640 1 -- 172.21.15.52:0/3984085717 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc403e640 con 0x7f8bb8003990 2024-03-12T17:07:00.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.241+0000 7ff100a00640 1 -- 172.21.15.52:0/3207057098 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0fc06ac20 con 0x7ff0fc0b27e0 2024-03-12T17:07:00.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.241+0000 7ff0f2000640 1 -- 172.21.15.52:0/3207057098 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f4007970 con 0x7ff0fc0b27e0 2024-03-12T17:07:00.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.241+0000 7ff0f2000640 1 -- 172.21.15.52:0/3207057098 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4003cf0 con 0x7ff0fc0b27e0 2024-03-12T17:07:00.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.242+0000 7fa2ee600640 1 Processor -- start 2024-03-12T17:07:00.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.243+0000 7fa2ee600640 1 -- start start 2024-03-12T17:07:00.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.244+0000 7f8bd6a00640 1 -- 172.21.15.52:0/3984085717 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bac005250 con 0x7f8bb8003990 2024-03-12T17:07:00.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.244+0000 7f8bcd600640 1 -- 172.21.15.52:0/3984085717 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8bc403e7a0 con 0x7f8bb8003990 2024-03-12T17:07:00.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.245+0000 7ff0f2000640 1 -- 172.21.15.52:0/3207057098 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff0f4004020 con 0x7ff0fc0b27e0 2024-03-12T17:07:00.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.245+0000 7fa2ee600640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa2e80cb2b0 0x7fa2e814d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.245+0000 7fa2ee600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2e80ccbd0 0x7fa2e8162310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.245+0000 7fa2ee600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2e8160ee0 0x7fa2e80b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.246+0000 7fa2ee600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e8163870 con 0x7fa2e8160ee0 2024-03-12T17:07:00.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.246+0000 7ff0f2000640 1 --2- 172.21.15.52:0/3207057098 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff0dc06ac30 0x7ff0dc06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.246+0000 7fa2e7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2e80ccbd0 0x7fa2e8162310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.246+0000 7fa2edc00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2e8160ee0 0x7fa2e80b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.246+0000 7ff0f2000640 1 -- 172.21.15.52:0/3207057098 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff0f4058bb0 con 0x7ff0fc0b27e0 2024-03-12T17:07:00.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.246+0000 7fa2edc00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2e8160ee0 0x7fa2e80b2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:32912/0 (socket says 172.21.15.52:32912) 2024-03-12T17:07:00.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.246+0000 7fa2edc00640 1 -- 172.21.15.52:0/2838555706 learned_addr learned my addr 172.21.15.52:0/2838555706 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:00.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.246+0000 7fa2ed200640 1 --2- 172.21.15.52:0/2838555706 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa2e80cb2b0 0x7fa2e814d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.246+0000 7fa2ee600640 1 -- 172.21.15.52:0/2838555706 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e8162cf0 con 0x7fa2e80ccbd0 2024-03-12T17:07:00.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.246+0000 7fa2ee600640 1 -- 172.21.15.52:0/2838555706 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e8162fd0 con 0x7fa2e80cb2b0 2024-03-12T17:07:00.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.247+0000 7fa2edc00640 1 -- 172.21.15.52:0/2838555706 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa2e80cb2b0 msgr2=0x7fa2e814d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.247+0000 7fa2edc00640 1 --2- 172.21.15.52:0/2838555706 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa2e80cb2b0 0x7fa2e814d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.247+0000 7fa2edc00640 1 -- 172.21.15.52:0/2838555706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2e80ccbd0 msgr2=0x7fa2e8162310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.247+0000 7ff0f9400640 1 --2- 172.21.15.52:0/3207057098 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff0dc06ac30 0x7ff0dc06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.247+0000 7fa2edc00640 1 --2- 172.21.15.52:0/2838555706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2e80ccbd0 0x7fa2e8162310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.247+0000 7fa2edc00640 1 -- 172.21.15.52:0/2838555706 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e0009400 con 0x7fa2e8160ee0 2024-03-12T17:07:00.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.248+0000 7fa2edc00640 1 --2- 172.21.15.52:0/2838555706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2e8160ee0 0x7fa2e80b2f40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa2e002b8f0 tx=0x7fa2e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:00.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.246+0000 7f8bcd600640 1 --2- 172.21.15.52:0/3984085717 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f8bb006ac00 0x7f8bb006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.246+0000 7f8bcd600640 1 -- 172.21.15.52:0/3984085717 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8bc40ab470 con 0x7f8bb8003990 2024-03-12T17:07:00.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.247+0000 7f8bd4c00640 1 --2- 172.21.15.52:0/3984085717 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f8bb006ac00 0x7f8bb006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.248+0000 7f8bd4c00640 1 --2- 172.21.15.52:0/3984085717 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f8bb006ac00 0x7f8bb006d0c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0071390 tx=0x7f8bbc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:00.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.249+0000 7ff0f9400640 1 --2- 172.21.15.52:0/3207057098 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff0dc06ac30 0x7ff0dc06d0f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec036000 tx=0x7ff0ec03c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:00.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.255+0000 7f8bcd600640 1 -- 172.21.15.52:0/3984085717 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f8bc40762c0 con 0x7f8bb8003990 2024-03-12T17:07:00.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.265+0000 7fa2e4c00640 1 -- 172.21.15.52:0/2838555706 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e003ace0 con 0x7fa2e8160ee0 2024-03-12T17:07:00.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.265+0000 7fa2e4c00640 1 -- 172.21.15.52:0/2838555706 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e003ae60 con 0x7fa2e8160ee0 2024-03-12T17:07:00.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.265+0000 7fa2e4c00640 1 -- 172.21.15.52:0/2838555706 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e00443f0 con 0x7fa2e8160ee0 2024-03-12T17:07:00.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.266+0000 7ff0f2000640 1 -- 172.21.15.52:0/3207057098 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7ff0f4054f80 con 0x7ff0fc0b27e0 2024-03-12T17:07:00.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.265+0000 7fa2ee600640 1 -- 172.21.15.52:0/2838555706 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e81632b0 con 0x7fa2e8160ee0 2024-03-12T17:07:00.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.266+0000 7fa2ee600640 1 -- 172.21.15.52:0/2838555706 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e814d8f0 con 0x7fa2e8160ee0 2024-03-12T17:07:00.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.270+0000 7fa2e4c00640 1 -- 172.21.15.52:0/2838555706 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa2e0044550 con 0x7fa2e8160ee0 2024-03-12T17:07:00.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.271+0000 7fa2ee600640 1 -- 172.21.15.52:0/2838555706 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e804e030 con 0x7fa2e8160ee0 2024-03-12T17:07:00.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.271+0000 7fa2e4c00640 1 --2- 172.21.15.52:0/2838555706 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa2c406acd0 0x7fa2c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.271+0000 7fa2e4c00640 1 -- 172.21.15.52:0/2838555706 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa2e00ab730 con 0x7fa2e8160ee0 2024-03-12T17:07:00.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.276+0000 7fa2ed200640 1 --2- 172.21.15.52:0/2838555706 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa2c406acd0 0x7fa2c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.277+0000 7fa2ed200640 1 --2- 172.21.15.52:0/2838555706 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa2c406acd0 0x7fa2c406d190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8011340 tx=0x7fa2d800b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:00.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.282+0000 7fa2e4c00640 1 -- 172.21.15.52:0/2838555706 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fa2e0076860 con 0x7fa2e8160ee0 2024-03-12T17:07:00.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.601+0000 7f3556000640 1 -- 172.21.15.52:0/2572493406 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f35500cb0c0 con 0x7f355015c830 2024-03-12T17:07:00.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.602+0000 7f354cc00640 1 -- 172.21.15.52:0/2572493406 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3540057cd0 con 0x7f355015c830 2024-03-12T17:07:00.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.603+0000 7f29b5600640 1 -- 172.21.15.52:0/801340088 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f29c00c6d40 con 0x7f29c00cf0e0 2024-03-12T17:07:00.597 INFO:teuthology.orchestra.run.smithi052.stdout:55834574858 2024-03-12T17:07:00.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.604+0000 7f29b7e00640 1 -- 172.21.15.52:0/801340088 <== mon.2 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f29c00c6d40 con 0x7f29c00cf0e0 2024-03-12T17:07:00.600 INFO:teuthology.orchestra.run.smithi052.stdout:55834574858 2024-03-12T17:07:00.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.606+0000 7f3556000640 1 -- 172.21.15.52:0/2572493406 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f353006acd0 msgr2=0x7f353006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.606+0000 7f3556000640 1 --2- 172.21.15.52:0/2572493406 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f353006acd0 0x7f353006d190 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3548036000 tx=0x7f3548030000 comp rx=0 tx=0).stop 2024-03-12T17:07:00.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.606+0000 7f3556000640 1 -- 172.21.15.52:0/2572493406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f355015c830 msgr2=0x7f355014e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.606+0000 7f3556000640 1 --2- 172.21.15.52:0/2572493406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f355015c830 0x7f355014e3f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f354000e7c0 tx=0x7f354000ec80 comp rx=0 tx=0).stop 2024-03-12T17:07:00.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.606+0000 7f3556000640 1 -- 172.21.15.52:0/2572493406 shutdown_connections 2024-03-12T17:07:00.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.606+0000 7f3556000640 1 --2- 172.21.15.52:0/2572493406 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f353006acd0 0x7f353006d190 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.606+0000 7f3556000640 1 --2- 172.21.15.52:0/2572493406 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35500d1380 0x7f355014deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.607+0000 7f3556000640 1 --2- 172.21.15.52:0/2572493406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f355015c830 0x7f355014e3f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.607+0000 7f3556000640 1 --2- 172.21.15.52:0/2572493406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35500cd1c0 0x7f35500c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.607+0000 7f3556000640 1 -- 172.21.15.52:0/2572493406 >> 172.21.15.52:0/2572493406 conn(0x7f3550089fd0 msgr2=0x7f355014f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:00.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.607+0000 7f3556000640 1 -- 172.21.15.52:0/2572493406 shutdown_connections 2024-03-12T17:07:00.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.607+0000 7f3556000640 1 -- 172.21.15.52:0/2572493406 wait complete. 2024-03-12T17:07:00.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.607+0000 7f29b5600640 1 -- 172.21.15.52:0/801340088 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f29a006acd0 msgr2=0x7f29a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.607+0000 7f29b5600640 1 --2- 172.21.15.52:0/801340088 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f29a006acd0 0x7f29a006d190 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f29b8001cb0 tx=0x7f29b8009040 comp rx=0 tx=0).stop 2024-03-12T17:07:00.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.607+0000 7f29b5600640 1 -- 172.21.15.52:0/801340088 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f29c00cf0e0 msgr2=0x7f29c00cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.607+0000 7f29b5600640 1 --2- 172.21.15.52:0/801340088 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f29c00cf0e0 0x7f29c00cc410 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f29b000c480 tx=0x7f29b000c940 comp rx=0 tx=0).stop 2024-03-12T17:07:00.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.608+0000 7f29b5600640 1 -- 172.21.15.52:0/801340088 shutdown_connections 2024-03-12T17:07:00.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.608+0000 7f29b5600640 1 --2- 172.21.15.52:0/801340088 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f29a006acd0 0x7f29a006d190 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.608+0000 7f29b5600640 1 --2- 172.21.15.52:0/801340088 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f29c00cf0e0 0x7f29c00cc410 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.608+0000 7f29b5600640 1 --2- 172.21.15.52:0/801340088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29c00c57d0 0x7f29c00c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.608+0000 7f29b5600640 1 --2- 172.21.15.52:0/801340088 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29c00b2430 0x7f29c00c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.608+0000 7f29b5600640 1 -- 172.21.15.52:0/801340088 >> 172.21.15.52:0/801340088 conn(0x7f29c0089fd0 msgr2=0x7f29c00af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:00.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.609+0000 7f29b5600640 1 -- 172.21.15.52:0/801340088 shutdown_connections 2024-03-12T17:07:00.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.609+0000 7f29b5600640 1 -- 172.21.15.52:0/801340088 wait complete. 2024-03-12T17:07:00.630 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.3 2024-03-12T17:07:00.630 DEBUG:teuthology.parallel:result is None 2024-03-12T17:07:00.631 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.0 2024-03-12T17:07:00.632 DEBUG:teuthology.parallel:result is None 2024-03-12T17:07:00.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.662+0000 7ff100a00640 1 -- 172.21.15.52:0/3207057098 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7ff0fc149580 con 0x7ff0fc0b27e0 2024-03-12T17:07:00.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.662+0000 7ff0f2000640 1 -- 172.21.15.52:0/3207057098 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff0f401d070 con 0x7ff0fc0b27e0 2024-03-12T17:07:00.656 INFO:teuthology.orchestra.run.smithi052.stdout:55834574861 2024-03-12T17:07:00.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.664+0000 7ff100a00640 1 -- 172.21.15.52:0/3207057098 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff0dc06ac30 msgr2=0x7ff0dc06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.664+0000 7ff100a00640 1 --2- 172.21.15.52:0/3207057098 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff0dc06ac30 0x7ff0dc06d0f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec036000 tx=0x7ff0ec03c040 comp rx=0 tx=0).stop 2024-03-12T17:07:00.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.664+0000 7ff100a00640 1 -- 172.21.15.52:0/3207057098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0fc0b27e0 msgr2=0x7ff0fc160940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.664+0000 7ff100a00640 1 --2- 172.21.15.52:0/3207057098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0fc0b27e0 0x7ff0fc160940 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff0f400ccd0 tx=0x7ff0f400a5c0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.665+0000 7ff100a00640 1 -- 172.21.15.52:0/3207057098 shutdown_connections 2024-03-12T17:07:00.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.665+0000 7ff100a00640 1 --2- 172.21.15.52:0/3207057098 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff0dc06ac30 0x7ff0dc06d0f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.665+0000 7ff100a00640 1 --2- 172.21.15.52:0/3207057098 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0fc14aa40 0x7ff0fc1638a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.665+0000 7ff100a00640 1 --2- 172.21.15.52:0/3207057098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0fc0cc7b0 0x7ff0fc163360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.665+0000 7ff100a00640 1 --2- 172.21.15.52:0/3207057098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0fc0b27e0 0x7ff0fc160940 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.665+0000 7ff100a00640 1 -- 172.21.15.52:0/3207057098 >> 172.21.15.52:0/3207057098 conn(0x7ff0fc06a3d0 msgr2=0x7ff0fc149cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:00.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.665+0000 7ff100a00640 1 -- 172.21.15.52:0/3207057098 shutdown_connections 2024-03-12T17:07:00.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.665+0000 7ff100a00640 1 -- 172.21.15.52:0/3207057098 wait complete. 2024-03-12T17:07:00.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.668+0000 7f546a800640 1 -- 172.21.15.52:0/3175683879 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f54640a74a0 con 0x7f54640b2aa0 2024-03-12T17:07:00.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.668+0000 7f5461600640 1 -- 172.21.15.52:0/3175683879 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f54640a74a0 con 0x7f54640b2aa0 2024-03-12T17:07:00.662 INFO:teuthology.orchestra.run.smithi052.stdout:55834574859 2024-03-12T17:07:00.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.669+0000 7fa2ee600640 1 -- 172.21.15.52:0/2838555706 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fa2e815adb0 con 0x7fa2e8160ee0 2024-03-12T17:07:00.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.669+0000 7fb139000640 1 -- 172.21.15.52:0/1185494491 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb1340cdab0 con 0x7fb1180039a0 2024-03-12T17:07:00.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.669+0000 7fa2e4c00640 1 -- 172.21.15.52:0/2838555706 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa2e0035070 con 0x7fa2e8160ee0 2024-03-12T17:07:00.664 INFO:teuthology.orchestra.run.smithi052.stdout:55834574859 2024-03-12T17:07:00.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.670+0000 7fb12b400640 1 -- 172.21.15.52:0/1185494491 <== mon.2 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb12c07a8d0 con 0x7fb1180039a0 2024-03-12T17:07:00.664 INFO:teuthology.orchestra.run.smithi052.stdout:55834574858 2024-03-12T17:07:00.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.671+0000 7f3c5b800640 1 -- 172.21.15.52:0/2086889712 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3c540b23a0 con 0x7f3c540b4a60 2024-03-12T17:07:00.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.671+0000 7f3c50c00640 1 -- 172.21.15.52:0/2086889712 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3c48079bc0 con 0x7f3c540b4a60 2024-03-12T17:07:00.665 INFO:teuthology.orchestra.run.smithi052.stdout:55834574858 2024-03-12T17:07:00.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.672+0000 7f546a800640 1 -- 172.21.15.52:0/3175683879 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f544406acd0 msgr2=0x7f544406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.672+0000 7f546a800640 1 --2- 172.21.15.52:0/3175683879 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f544406acd0 0x7f544406d190 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f54641facc0 tx=0x7f5458006060 comp rx=0 tx=0).stop 2024-03-12T17:07:00.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.672+0000 7f546a800640 1 -- 172.21.15.52:0/3175683879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54640b2aa0 msgr2=0x7f546413f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.672+0000 7f546a800640 1 --2- 172.21.15.52:0/3175683879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54640b2aa0 0x7f546413f720 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f545c009ee0 tx=0x7f545c004a40 comp rx=0 tx=0).stop 2024-03-12T17:07:00.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.672+0000 7f546a800640 1 -- 172.21.15.52:0/3175683879 shutdown_connections 2024-03-12T17:07:00.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.672+0000 7f546a800640 1 --2- 172.21.15.52:0/3175683879 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f544406acd0 0x7f544406d190 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.672+0000 7f546a800640 1 --2- 172.21.15.52:0/3175683879 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5464141980 0x7f54641fa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.672+0000 7f546a800640 1 --2- 172.21.15.52:0/3175683879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54640cc960 0x7f54641f9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.672+0000 7f546a800640 1 --2- 172.21.15.52:0/3175683879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54640b2aa0 0x7f546413f720 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.672+0000 7f546a800640 1 -- 172.21.15.52:0/3175683879 >> 172.21.15.52:0/3175683879 conn(0x7f54640700d0 msgr2=0x7f54640b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:00.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.672+0000 7f546a800640 1 -- 172.21.15.52:0/3175683879 shutdown_connections 2024-03-12T17:07:00.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.673+0000 7f546a800640 1 -- 172.21.15.52:0/3175683879 wait complete. 2024-03-12T17:07:00.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.673+0000 7fa2ee600640 1 -- 172.21.15.52:0/2838555706 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa2c406acd0 msgr2=0x7fa2c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.673+0000 7fa2ee600640 1 --2- 172.21.15.52:0/2838555706 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa2c406acd0 0x7fa2c406d190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8011340 tx=0x7fa2d800b040 comp rx=0 tx=0).stop 2024-03-12T17:07:00.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.673+0000 7fa2ee600640 1 -- 172.21.15.52:0/2838555706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2e8160ee0 msgr2=0x7fa2e80b2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.673+0000 7fa2ee600640 1 --2- 172.21.15.52:0/2838555706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2e8160ee0 0x7fa2e80b2f40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa2e002b8f0 tx=0x7fa2e00046c0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.673+0000 7fa2ee600640 1 -- 172.21.15.52:0/2838555706 shutdown_connections 2024-03-12T17:07:00.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.673+0000 7fa2ee600640 1 --2- 172.21.15.52:0/2838555706 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa2c406acd0 0x7fa2c406d190 secure :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8011340 tx=0x7fa2d800b040 comp rx=0 tx=0).stop 2024-03-12T17:07:00.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.673+0000 7fa2ee600640 1 --2- 172.21.15.52:0/2838555706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2e8160ee0 0x7fa2e80b2f40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.673+0000 7fa2ee600640 1 --2- 172.21.15.52:0/2838555706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2e80ccbd0 0x7fa2e8162310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.673+0000 7fa2ee600640 1 --2- 172.21.15.52:0/2838555706 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa2e80cb2b0 0x7fa2e814d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.673+0000 7fa2ee600640 1 -- 172.21.15.52:0/2838555706 >> 172.21.15.52:0/2838555706 conn(0x7fa2e8065e20 msgr2=0x7fa2e8069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:00.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.673+0000 7fa2ee600640 1 -- 172.21.15.52:0/2838555706 shutdown_connections 2024-03-12T17:07:00.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.673+0000 7fa2ee600640 1 -- 172.21.15.52:0/2838555706 wait complete. 2024-03-12T17:07:00.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.674+0000 7fb139000640 1 -- 172.21.15.52:0/1185494491 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb11406ac50 msgr2=0x7fb11406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.674+0000 7fb139000640 1 --2- 172.21.15.52:0/1185494491 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb11406ac50 0x7fb11406d110 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb124000c00 tx=0x7fb124009040 comp rx=0 tx=0).stop 2024-03-12T17:07:00.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.674+0000 7fb139000640 1 -- 172.21.15.52:0/1185494491 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb1180039a0 msgr2=0x7fb1340ac3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.674+0000 7fb139000640 1 --2- 172.21.15.52:0/1185494491 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb1180039a0 0x7fb1340ac3a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb118005630 tx=0x7fb12c004a80 comp rx=0 tx=0).stop 2024-03-12T17:07:00.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.674+0000 7f3c5b800640 1 -- 172.21.15.52:0/2086889712 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f3c3406ac00 msgr2=0x7f3c3406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.674+0000 7f3c5b800640 1 --2- 172.21.15.52:0/2086889712 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f3c3406ac00 0x7f3c3406d0c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3c540d38b0 tx=0x7f3c40009040 comp rx=0 tx=0).stop 2024-03-12T17:07:00.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.674+0000 7f3c5b800640 1 -- 172.21.15.52:0/2086889712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3c540b4a60 msgr2=0x7f3c5415a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.674+0000 7f3c5b800640 1 --2- 172.21.15.52:0/2086889712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3c540b4a60 0x7f3c5415a750 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3c48000c00 tx=0x7f3c48004730 comp rx=0 tx=0).stop 2024-03-12T17:07:00.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.674+0000 7f3c5b800640 1 -- 172.21.15.52:0/2086889712 shutdown_connections 2024-03-12T17:07:00.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.674+0000 7f3c5b800640 1 --2- 172.21.15.52:0/2086889712 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f3c3406ac00 0x7f3c3406d0c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.674+0000 7f3c5b800640 1 --2- 172.21.15.52:0/2086889712 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3c5415cda0 0x7f3c540c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.674+0000 7f3c5b800640 1 --2- 172.21.15.52:0/2086889712 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c540b5a30 0x7f3c540c8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.674+0000 7f3c5b800640 1 --2- 172.21.15.52:0/2086889712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3c540b4a60 0x7f3c5415a750 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.675+0000 7f3c5b800640 1 -- 172.21.15.52:0/2086889712 >> 172.21.15.52:0/2086889712 conn(0x7f3c540cf350 msgr2=0x7f3c5406b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:00.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.674+0000 7fb139000640 1 -- 172.21.15.52:0/1185494491 shutdown_connections 2024-03-12T17:07:00.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.674+0000 7fb139000640 1 --2- 172.21.15.52:0/1185494491 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb11406ac50 0x7fb11406d110 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.674+0000 7fb139000640 1 --2- 172.21.15.52:0/1185494491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb134157e40 0x7fb1342025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.674+0000 7fb139000640 1 --2- 172.21.15.52:0/1185494491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb134156a20 0x7fb134157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.674+0000 7fb139000640 1 --2- 172.21.15.52:0/1185494491 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb1180039a0 0x7fb1340ac3a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.674+0000 7fb139000640 1 -- 172.21.15.52:0/1185494491 >> 172.21.15.52:0/1185494491 conn(0x7fb134069e70 msgr2=0x7fb1340b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:00.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.675+0000 7f3c5b800640 1 -- 172.21.15.52:0/2086889712 shutdown_connections 2024-03-12T17:07:00.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.675+0000 7f3c5b800640 1 -- 172.21.15.52:0/2086889712 wait complete. 2024-03-12T17:07:00.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.675+0000 7fb139000640 1 -- 172.21.15.52:0/1185494491 shutdown_connections 2024-03-12T17:07:00.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.675+0000 7fb139000640 1 -- 172.21.15.52:0/1185494491 wait complete. 2024-03-12T17:07:00.680 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574861 for osd.4 2024-03-12T17:07:00.680 DEBUG:teuthology.parallel:result is None 2024-03-12T17:07:00.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.690+0000 7f8bd6a00640 1 -- 172.21.15.52:0/3984085717 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8bac005500 con 0x7f8bb8003990 2024-03-12T17:07:00.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.691+0000 7f8bcd600640 1 -- 172.21.15.52:0/3984085717 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8bc40798e0 con 0x7f8bb8003990 2024-03-12T17:07:00.685 INFO:teuthology.orchestra.run.smithi052.stdout:55834574858 2024-03-12T17:07:00.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.694+0000 7f8bd6a00640 1 -- 172.21.15.52:0/3984085717 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f8bb006ac00 msgr2=0x7f8bb006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.694+0000 7f8bd6a00640 1 --2- 172.21.15.52:0/3984085717 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f8bb006ac00 0x7f8bb006d0c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0071390 tx=0x7f8bbc008040 comp rx=0 tx=0).stop 2024-03-12T17:07:00.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.694+0000 7f8bd6a00640 1 -- 172.21.15.52:0/3984085717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8bb8003990 msgr2=0x7f8bd01fa360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.694+0000 7f8bd6a00640 1 --2- 172.21.15.52:0/3984085717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8bb8003990 0x7f8bd01fa360 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4037040 tx=0x7f8bc4004730 comp rx=0 tx=0).stop 2024-03-12T17:07:00.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.694+0000 7f8bd6a00640 1 -- 172.21.15.52:0/3984085717 shutdown_connections 2024-03-12T17:07:00.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.694+0000 7f8bd6a00640 1 --2- 172.21.15.52:0/3984085717 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f8bb006ac00 0x7f8bb006d0c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.694+0000 7f8bd6a00640 1 --2- 172.21.15.52:0/3984085717 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8bd01f6260 0x7f8bd015c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.694+0000 7f8bd6a00640 1 --2- 172.21.15.52:0/3984085717 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bd01f37f0 0x7f8bd01f7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.694+0000 7f8bd6a00640 1 --2- 172.21.15.52:0/3984085717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8bb8003990 0x7f8bd01fa360 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.694+0000 7f8bd6a00640 1 -- 172.21.15.52:0/3984085717 >> 172.21.15.52:0/3984085717 conn(0x7f8bd00717e0 msgr2=0x7f8bd00d2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:00.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.694+0000 7f8bd6a00640 1 -- 172.21.15.52:0/3984085717 shutdown_connections 2024-03-12T17:07:00.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.694+0000 7f8bd6a00640 1 -- 172.21.15.52:0/3984085717 wait complete. 2024-03-12T17:07:00.690 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.1 2024-03-12T17:07:00.690 DEBUG:teuthology.parallel:result is None 2024-03-12T17:07:00.694 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.5 2024-03-12T17:07:00.694 DEBUG:teuthology.parallel:result is None 2024-03-12T17:07:00.695 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.7 2024-03-12T17:07:00.695 DEBUG:teuthology.parallel:result is None 2024-03-12T17:07:00.695 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.6 2024-03-12T17:07:00.696 DEBUG:teuthology.parallel:result is None 2024-03-12T17:07:00.705 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.2 2024-03-12T17:07:00.705 DEBUG:teuthology.parallel:result is None 2024-03-12T17:07:00.705 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-12T17:07:00.705 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-12T17:07:00.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.967+0000 7fbc0c200640 1 Processor -- start 2024-03-12T17:07:00.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.967+0000 7fbc0c200640 1 -- start start 2024-03-12T17:07:00.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.968+0000 7fbc0c200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fbc0414aa40 0x7fbc040cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.969+0000 7fbc0c200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fbc040cc7b0 0x7fbc040ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.969+0000 7fbc0c200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbc040b27e0 0x7fbc040b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.969+0000 7fbc0c200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc0406da70 con 0x7fbc040cc7b0 2024-03-12T17:07:00.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.969+0000 7fbc0c200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc040b1d00 con 0x7fbc0414aa40 2024-03-12T17:07:00.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.969+0000 7fbc0c200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc040b1fe0 con 0x7fbc040b27e0 2024-03-12T17:07:00.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.969+0000 7fbc08e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fbc040cc7b0 0x7fbc040ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.969+0000 7fbc08e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fbc040cc7b0 0x7fbc040ccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:32914/0 (socket says 172.21.15.52:32914) 2024-03-12T17:07:00.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.969+0000 7fbc08e00640 1 -- 172.21.15.52:0/552819805 learned_addr learned my addr 172.21.15.52:0/552819805 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:00.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.969+0000 7fbc0a200640 1 --2- 172.21.15.52:0/552819805 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbc040b27e0 0x7fbc040b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.969+0000 7fbc09800640 1 --2- 172.21.15.52:0/552819805 >> v2:172.21.15.102:3300/0 conn(0x7fbc0414aa40 0x7fbc040cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.969+0000 7fbc08e00640 1 -- 172.21.15.52:0/552819805 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbc040b27e0 msgr2=0x7fbc040b2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:00.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.969+0000 7fbc08e00640 1 --2- 172.21.15.52:0/552819805 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbc040b27e0 0x7fbc040b2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.969+0000 7fbc08e00640 1 -- 172.21.15.52:0/552819805 >> v2:172.21.15.102:3300/0 conn(0x7fbc0414aa40 msgr2=0x7fbc040cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:00.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.969+0000 7fbc08e00640 1 --2- 172.21.15.52:0/552819805 >> v2:172.21.15.102:3300/0 conn(0x7fbc0414aa40 0x7fbc040cd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.969+0000 7fbc08e00640 1 -- 172.21.15.52:0/552819805 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc040b22c0 con 0x7fbc040cc7b0 2024-03-12T17:07:00.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.969+0000 7fbc09800640 1 --2- 172.21.15.52:0/552819805 >> v2:172.21.15.102:3300/0 conn(0x7fbc0414aa40 0x7fbc040cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:00.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.971+0000 7fbc08e00640 1 --2- 172.21.15.52:0/552819805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc040cc7b0 0x7fbc040ccc60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbc040b4960 tx=0x7fbbf002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd21a9f925b941f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:00.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.971+0000 7fbc03e00640 1 -- 172.21.15.52:0/552819805 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0031870 con 0x7fbc040cc7b0 2024-03-12T17:07:00.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.971+0000 7fbc03e00640 1 -- 172.21.15.52:0/552819805 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf00319d0 con 0x7fbc040cc7b0 2024-03-12T17:07:00.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.971+0000 7fbc03e00640 1 -- 172.21.15.52:0/552819805 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0031d60 con 0x7fbc040cc7b0 2024-03-12T17:07:00.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.972+0000 7fbc0c200640 1 -- 172.21.15.52:0/552819805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc040cc7b0 msgr2=0x7fbc040ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.972+0000 7fbc0c200640 1 --2- 172.21.15.52:0/552819805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc040cc7b0 0x7fbc040ccc60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbc040b4960 tx=0x7fbbf002b910 comp rx=0 tx=0).stop 2024-03-12T17:07:00.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.972+0000 7fbc0c200640 1 -- 172.21.15.52:0/552819805 shutdown_connections 2024-03-12T17:07:00.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.972+0000 7fbc0c200640 1 --2- 172.21.15.52:0/552819805 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbc040b27e0 0x7fbc040b2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.972+0000 7fbc0c200640 1 --2- 172.21.15.52:0/552819805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc040cc7b0 0x7fbc040ccc60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.972+0000 7fbc0c200640 1 --2- 172.21.15.52:0/552819805 >> v2:172.21.15.102:3300/0 conn(0x7fbc0414aa40 0x7fbc040cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.972+0000 7fbc0c200640 1 -- 172.21.15.52:0/552819805 >> 172.21.15.52:0/552819805 conn(0x7fbc0406a3d0 msgr2=0x7fbc040b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:00.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.972+0000 7fbc0c200640 1 -- 172.21.15.52:0/552819805 shutdown_connections 2024-03-12T17:07:00.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.972+0000 7fbc0c200640 1 -- 172.21.15.52:0/552819805 wait complete. 2024-03-12T17:07:00.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.974+0000 7fbc0c200640 1 Processor -- start 2024-03-12T17:07:00.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.974+0000 7fbc0c200640 1 -- start start 2024-03-12T17:07:00.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.975+0000 7fbc0c200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc040b27e0 0x7fbc041498d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.976+0000 7fbc0c200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbc040cc7b0 0x7fbc04160730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.976+0000 7fbc0c200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc0414aa40 0x7fbc041637e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.976+0000 7fbc0c200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc040ad8c0 con 0x7fbc040b27e0 2024-03-12T17:07:00.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.976+0000 7fbc0c200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc041617d0 con 0x7fbc0414aa40 2024-03-12T17:07:00.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.976+0000 7fbc0c200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc04149e10 con 0x7fbc040cc7b0 2024-03-12T17:07:00.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.976+0000 7fbc09800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc040b27e0 0x7fbc041498d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.976+0000 7fbc09800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc040b27e0 0x7fbc041498d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:32916/0 (socket says 172.21.15.52:32916) 2024-03-12T17:07:00.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.976+0000 7fbc09800640 1 -- 172.21.15.52:0/2827736077 learned_addr learned my addr 172.21.15.52:0/2827736077 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:00.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.976+0000 7fbc08e00640 1 --2- 172.21.15.52:0/2827736077 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbc040cc7b0 0x7fbc04160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.976+0000 7fbc09800640 1 -- 172.21.15.52:0/2827736077 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbc040cc7b0 msgr2=0x7fbc04160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.976+0000 7fbc09800640 1 --2- 172.21.15.52:0/2827736077 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbc040cc7b0 0x7fbc04160730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.976+0000 7fbc09800640 1 -- 172.21.15.52:0/2827736077 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc0414aa40 msgr2=0x7fbc041637e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:00.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.976+0000 7fbc09800640 1 --2- 172.21.15.52:0/2827736077 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc0414aa40 0x7fbc041637e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:00.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.976+0000 7fbc09800640 1 -- 172.21.15.52:0/2827736077 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf00094d0 con 0x7fbc040b27e0 2024-03-12T17:07:00.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.977+0000 7fbc09800640 1 --2- 172.21.15.52:0/2827736077 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc040b27e0 0x7fbc041498d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800c7f0 tx=0x7fbbf800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:00.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.977+0000 7fbc01600640 1 -- 172.21.15.52:0/2827736077 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8007820 con 0x7fbc040b27e0 2024-03-12T17:07:00.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.977+0000 7fbc01600640 1 -- 172.21.15.52:0/2827736077 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf8007980 con 0x7fbc040b27e0 2024-03-12T17:07:00.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.977+0000 7fbc01600640 1 -- 172.21.15.52:0/2827736077 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8007cf0 con 0x7fbc040b27e0 2024-03-12T17:07:00.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.977+0000 7fbc0c200640 1 -- 172.21.15.52:0/2827736077 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc041614f0 con 0x7fbc040b27e0 2024-03-12T17:07:00.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.977+0000 7fbc0c200640 1 -- 172.21.15.52:0/2827736077 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc041f6980 con 0x7fbc040b27e0 2024-03-12T17:07:00.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.978+0000 7fbc0c200640 1 -- 172.21.15.52:0/2827736077 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc041f6500 con 0x7fbc040b27e0 2024-03-12T17:07:00.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.979+0000 7fbc01600640 1 -- 172.21.15.52:0/2827736077 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fbbf80113f0 con 0x7fbc040b27e0 2024-03-12T17:07:00.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.980+0000 7fbc01600640 1 --2- 172.21.15.52:0/2827736077 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fbbe406ac00 0x7fbbe406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:00.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.980+0000 7fbc01600640 1 -- 172.21.15.52:0/2827736077 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbbf80898b0 con 0x7fbc040b27e0 2024-03-12T17:07:00.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.981+0000 7fbc08e00640 1 --2- 172.21.15.52:0/2827736077 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fbbe406ac00 0x7fbbe406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:00.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.982+0000 7fbc08e00640 1 --2- 172.21.15.52:0/2827736077 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fbbe406ac00 0x7fbbe406d0c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0036000 tx=0x7fbbf003c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:00.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:00.985+0000 7fbc01600640 1 -- 172.21.15.52:0/2827736077 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fbc041f6500 con 0x7fbc040b27e0 2024-03-12T17:07:01.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.175+0000 7fbc0c200640 1 -- 172.21.15.52:0/2827736077 --> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbc041f6500 con 0x7fbbe406ac00 2024-03-12T17:07:01.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.201+0000 7fbc01600640 1 -- 172.21.15.52:0/2827736077 <== mgr.4109 v2:172.21.15.102:6832/36622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345027 (secure 0 0 0) 0x7fbc041f6500 con 0x7fbbe406ac00 2024-03-12T17:07:01.199 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:07:01.202 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-03-12T17:07:01.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.208+0000 7fbc0c200640 1 -- 172.21.15.52:0/2827736077 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fbbe406ac00 msgr2=0x7fbbe406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:01.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.208+0000 7fbc0c200640 1 --2- 172.21.15.52:0/2827736077 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fbbe406ac00 0x7fbbe406d0c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0036000 tx=0x7fbbf003c040 comp rx=0 tx=0).stop 2024-03-12T17:07:01.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.208+0000 7fbc0c200640 1 -- 172.21.15.52:0/2827736077 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc040b27e0 msgr2=0x7fbc041498d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:01.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.208+0000 7fbc0c200640 1 --2- 172.21.15.52:0/2827736077 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc040b27e0 0x7fbc041498d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800c7f0 tx=0x7fbbf800ccb0 comp rx=0 tx=0).stop 2024-03-12T17:07:01.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.208+0000 7fbc0c200640 1 -- 172.21.15.52:0/2827736077 shutdown_connections 2024-03-12T17:07:01.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.208+0000 7fbc0c200640 1 --2- 172.21.15.52:0/2827736077 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fbbe406ac00 0x7fbbe406d0c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:01.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.208+0000 7fbc0c200640 1 --2- 172.21.15.52:0/2827736077 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbc040cc7b0 0x7fbc04160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:01.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.208+0000 7fbc0c200640 1 --2- 172.21.15.52:0/2827736077 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc0414aa40 0x7fbc041637e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:01.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.208+0000 7fbc0c200640 1 --2- 172.21.15.52:0/2827736077 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc040b27e0 0x7fbc041498d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:01.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.208+0000 7fbc0c200640 1 -- 172.21.15.52:0/2827736077 >> 172.21.15.52:0/2827736077 conn(0x7fbc0406a3d0 msgr2=0x7fbc0406c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:01.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.209+0000 7fbc0c200640 1 -- 172.21.15.52:0/2827736077 shutdown_connections 2024-03-12T17:07:01.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.209+0000 7fbc0c200640 1 -- 172.21.15.52:0/2827736077 wait complete. 2024-03-12T17:07:01.218 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-03-12T17:07:01.160543+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594961,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231808,"kb_used_data":3304,"kb_used_omap":13,"kb_used_meta":228338,"kb_avail":754742912,"statfs":{"total":773094113280,"available":772856741888,"internally_reserved":0,"allocated":3383296,"data_stored":2025530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":233818200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005926"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.999570+0000","last_change":"2024-03-12T17:06:25.442740+0000","last_active":"2024-03-12T17:06:29.999570+0000","last_peered":"2024-03-12T17:06:29.999570+0000","last_clean":"2024-03-12T17:06:29.999570+0000","last_became_active":"2024-03-12T17:06:25.438608+0000","last_became_peered":"2024-03-12T17:06:25.438608+0000","last_unstale":"2024-03-12T17:06:29.999570+0000","last_undegraded":"2024-03-12T17:06:29.999570+0000","last_fullsized":"2024-03-12T17:06:29.999570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:59:41.676210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.004064+0000","last_change":"2024-03-12T17:06:25.939770+0000","last_active":"2024-03-12T17:06:30.004064+0000","last_peered":"2024-03-12T17:06:30.004064+0000","last_clean":"2024-03-12T17:06:30.004064+0000","last_became_active":"2024-03-12T17:06:25.938969+0000","last_became_peered":"2024-03-12T17:06:25.938969+0000","last_unstale":"2024-03-12T17:06:30.004064+0000","last_undegraded":"2024-03-12T17:06:30.004064+0000","last_fullsized":"2024-03-12T17:06:30.004064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:23:25.984714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.476841+0000","last_change":"2024-03-12T17:06:30.023660+0000","last_active":"2024-03-12T17:06:30.476841+0000","last_peered":"2024-03-12T17:06:30.476841+0000","last_clean":"2024-03-12T17:06:30.476841+0000","last_became_active":"2024-03-12T17:06:30.022948+0000","last_became_peered":"2024-03-12T17:06:30.022948+0000","last_unstale":"2024-03-12T17:06:30.476841+0000","last_undegraded":"2024-03-12T17:06:30.476841+0000","last_fullsized":"2024-03-12T17:06:30.476841+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:04:13.473422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.003821+0000","last_change":"2024-03-12T17:06:25.446629+0000","last_active":"2024-03-12T17:06:30.003821+0000","last_peered":"2024-03-12T17:06:30.003821+0000","last_clean":"2024-03-12T17:06:30.003821+0000","last_became_active":"2024-03-12T17:06:25.442784+0000","last_became_peered":"2024-03-12T17:06:25.442784+0000","last_unstale":"2024-03-12T17:06:30.003821+0000","last_undegraded":"2024-03-12T17:06:30.003821+0000","last_fullsized":"2024-03-12T17:06:30.003821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:59:41.676210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.999425+0000","last_change":"2024-03-12T17:06:25.953462+0000","last_active":"2024-03-12T17:06:29.999425+0000","last_peered":"2024-03-12T17:06:29.999425+0000","last_clean":"2024-03-12T17:06:29.999425+0000","last_became_active":"2024-03-12T17:06:25.953198+0000","last_became_peered":"2024-03-12T17:06:25.953198+0000","last_unstale":"2024-03-12T17:06:29.999425+0000","last_undegraded":"2024-03-12T17:06:29.999425+0000","last_fullsized":"2024-03-12T17:06:29.999425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:52:14.405796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.480606+0000","last_change":"2024-03-12T17:06:30.017287+0000","last_active":"2024-03-12T17:06:30.480606+0000","last_peered":"2024-03-12T17:06:30.480606+0000","last_clean":"2024-03-12T17:06:30.480606+0000","last_became_active":"2024-03-12T17:06:30.016834+0000","last_became_peered":"2024-03-12T17:06:30.016834+0000","last_unstale":"2024-03-12T17:06:30.480606+0000","last_undegraded":"2024-03-12T17:06:30.480606+0000","last_fullsized":"2024-03-12T17:06:30.480606+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:21:26.853333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012212+0000","last_change":"2024-03-12T17:06:25.454124+0000","last_active":"2024-03-12T17:06:30.012212+0000","last_peered":"2024-03-12T17:06:30.012212+0000","last_clean":"2024-03-12T17:06:30.012212+0000","last_became_active":"2024-03-12T17:06:25.453877+0000","last_became_peered":"2024-03-12T17:06:25.453877+0000","last_unstale":"2024-03-12T17:06:30.012212+0000","last_undegraded":"2024-03-12T17:06:30.012212+0000","last_fullsized":"2024-03-12T17:06:30.012212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:45:01.926495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.991068+0000","last_change":"2024-03-12T17:06:25.926772+0000","last_active":"2024-03-12T17:06:29.991068+0000","last_peered":"2024-03-12T17:06:29.991068+0000","last_clean":"2024-03-12T17:06:29.991068+0000","last_became_active":"2024-03-12T17:06:25.926486+0000","last_became_peered":"2024-03-12T17:06:25.926486+0000","last_unstale":"2024-03-12T17:06:29.991068+0000","last_undegraded":"2024-03-12T17:06:29.991068+0000","last_fullsized":"2024-03-12T17:06:29.991068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:28:58.739417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.482311+0000","last_change":"2024-03-12T17:06:30.028104+0000","last_active":"2024-03-12T17:06:30.482311+0000","last_peered":"2024-03-12T17:06:30.482311+0000","last_clean":"2024-03-12T17:06:30.482311+0000","last_became_active":"2024-03-12T17:06:30.027645+0000","last_became_peered":"2024-03-12T17:06:30.027645+0000","last_unstale":"2024-03-12T17:06:30.482311+0000","last_undegraded":"2024-03-12T17:06:30.482311+0000","last_fullsized":"2024-03-12T17:06:30.482311+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:13:55.311599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.011875+0000","last_change":"2024-03-12T17:06:25.455911+0000","last_active":"2024-03-12T17:06:30.011875+0000","last_peered":"2024-03-12T17:06:30.011875+0000","last_clean":"2024-03-12T17:06:30.011875+0000","last_became_active":"2024-03-12T17:06:25.455251+0000","last_became_peered":"2024-03-12T17:06:25.455251+0000","last_unstale":"2024-03-12T17:06:30.011875+0000","last_undegraded":"2024-03-12T17:06:30.011875+0000","last_fullsized":"2024-03-12T17:06:30.011875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:36:25.468644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990029+0000","last_change":"2024-03-12T17:06:25.940304+0000","last_active":"2024-03-12T17:06:29.990029+0000","last_peered":"2024-03-12T17:06:29.990029+0000","last_clean":"2024-03-12T17:06:29.990029+0000","last_became_active":"2024-03-12T17:06:25.940019+0000","last_became_peered":"2024-03-12T17:06:25.940019+0000","last_unstale":"2024-03-12T17:06:29.990029+0000","last_undegraded":"2024-03-12T17:06:29.990029+0000","last_fullsized":"2024-03-12T17:06:29.990029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:44:34.582031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.481007+0000","last_change":"2024-03-12T17:06:30.030105+0000","last_active":"2024-03-12T17:06:30.481007+0000","last_peered":"2024-03-12T17:06:30.481007+0000","last_clean":"2024-03-12T17:06:30.481007+0000","last_became_active":"2024-03-12T17:06:30.029702+0000","last_became_peered":"2024-03-12T17:06:30.029702+0000","last_unstale":"2024-03-12T17:06:30.481007+0000","last_undegraded":"2024-03-12T17:06:30.481007+0000","last_fullsized":"2024-03-12T17:06:30.481007+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:33:44.182800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.986543+0000","last_change":"2024-03-12T17:06:25.942985+0000","last_active":"2024-03-12T17:06:29.986543+0000","last_peered":"2024-03-12T17:06:29.986543+0000","last_clean":"2024-03-12T17:06:29.986543+0000","last_became_active":"2024-03-12T17:06:25.942526+0000","last_became_peered":"2024-03-12T17:06:25.942526+0000","last_unstale":"2024-03-12T17:06:29.986543+0000","last_undegraded":"2024-03-12T17:06:29.986543+0000","last_fullsized":"2024-03-12T17:06:29.986543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:44:25.750833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012322+0000","last_change":"2024-03-12T17:06:25.453557+0000","last_active":"2024-03-12T17:06:30.012322+0000","last_peered":"2024-03-12T17:06:30.012322+0000","last_clean":"2024-03-12T17:06:30.012322+0000","last_became_active":"2024-03-12T17:06:25.451663+0000","last_became_peered":"2024-03-12T17:06:25.451663+0000","last_unstale":"2024-03-12T17:06:30.012322+0000","last_undegraded":"2024-03-12T17:06:30.012322+0000","last_fullsized":"2024-03-12T17:06:30.012322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:36:47.407275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.482348+0000","last_change":"2024-03-12T17:06:30.018487+0000","last_active":"2024-03-12T17:06:30.482348+0000","last_peered":"2024-03-12T17:06:30.482348+0000","last_clean":"2024-03-12T17:06:30.482348+0000","last_became_active":"2024-03-12T17:06:30.017933+0000","last_became_peered":"2024-03-12T17:06:30.017933+0000","last_unstale":"2024-03-12T17:06:30.482348+0000","last_undegraded":"2024-03-12T17:06:30.482348+0000","last_fullsized":"2024-03-12T17:06:30.482348+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:42:49.327905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.995828+0000","last_change":"2024-03-12T17:06:25.952715+0000","last_active":"2024-03-12T17:06:29.995828+0000","last_peered":"2024-03-12T17:06:29.995828+0000","last_clean":"2024-03-12T17:06:29.995828+0000","last_became_active":"2024-03-12T17:06:25.952288+0000","last_became_peered":"2024-03-12T17:06:25.952288+0000","last_unstale":"2024-03-12T17:06:29.995828+0000","last_undegraded":"2024-03-12T17:06:29.995828+0000","last_fullsized":"2024-03-12T17:06:29.995828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:29:49.461651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.015251+0000","last_change":"2024-03-12T17:06:25.448043+0000","last_active":"2024-03-12T17:06:30.015251+0000","last_peered":"2024-03-12T17:06:30.015251+0000","last_clean":"2024-03-12T17:06:30.015251+0000","last_became_active":"2024-03-12T17:06:25.447665+0000","last_became_peered":"2024-03-12T17:06:25.447665+0000","last_unstale":"2024-03-12T17:06:30.015251+0000","last_undegraded":"2024-03-12T17:06:30.015251+0000","last_fullsized":"2024-03-12T17:06:30.015251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:28:25.809578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.481001+0000","last_change":"2024-03-12T17:06:30.020157+0000","last_active":"2024-03-12T17:06:30.481001+0000","last_peered":"2024-03-12T17:06:30.481001+0000","last_clean":"2024-03-12T17:06:30.481001+0000","last_became_active":"2024-03-12T17:06:30.019631+0000","last_became_peered":"2024-03-12T17:06:30.019631+0000","last_unstale":"2024-03-12T17:06:30.481001+0000","last_undegraded":"2024-03-12T17:06:30.481001+0000","last_fullsized":"2024-03-12T17:06:30.481001+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:46:40.007538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.987316+0000","last_change":"2024-03-12T17:06:25.925596+0000","last_active":"2024-03-12T17:06:29.987316+0000","last_peered":"2024-03-12T17:06:29.987316+0000","last_clean":"2024-03-12T17:06:29.987316+0000","last_became_active":"2024-03-12T17:06:25.925225+0000","last_became_peered":"2024-03-12T17:06:25.925225+0000","last_unstale":"2024-03-12T17:06:29.987316+0000","last_undegraded":"2024-03-12T17:06:29.987316+0000","last_fullsized":"2024-03-12T17:06:29.987316+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:44:34.582031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.014268+0000","last_change":"2024-03-12T17:06:25.450336+0000","last_active":"2024-03-12T17:06:30.014268+0000","last_peered":"2024-03-12T17:06:30.014268+0000","last_clean":"2024-03-12T17:06:30.014268+0000","last_became_active":"2024-03-12T17:06:25.450049+0000","last_became_peered":"2024-03-12T17:06:25.450049+0000","last_unstale":"2024-03-12T17:06:30.014268+0000","last_undegraded":"2024-03-12T17:06:30.014268+0000","last_fullsized":"2024-03-12T17:06:30.014268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:00:06.114876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.511116+0000","last_change":"2024-03-12T17:06:30.024652+0000","last_active":"2024-03-12T17:06:30.511116+0000","last_peered":"2024-03-12T17:06:30.511116+0000","last_clean":"2024-03-12T17:06:30.511116+0000","last_became_active":"2024-03-12T17:06:30.024156+0000","last_became_peered":"2024-03-12T17:06:30.024156+0000","last_unstale":"2024-03-12T17:06:30.511116+0000","last_undegraded":"2024-03-12T17:06:30.511116+0000","last_fullsized":"2024-03-12T17:06:30.511116+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:21:26.853333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012195+0000","last_change":"2024-03-12T17:06:25.935633+0000","last_active":"2024-03-12T17:06:30.012195+0000","last_peered":"2024-03-12T17:06:30.012195+0000","last_clean":"2024-03-12T17:06:30.012195+0000","last_became_active":"2024-03-12T17:06:25.935322+0000","last_became_peered":"2024-03-12T17:06:25.935322+0000","last_unstale":"2024-03-12T17:06:30.012195+0000","last_undegraded":"2024-03-12T17:06:30.012195+0000","last_fullsized":"2024-03-12T17:06:30.012195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:34:27.281126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.014503+0000","last_change":"2024-03-12T17:06:25.456881+0000","last_active":"2024-03-12T17:06:30.014503+0000","last_peered":"2024-03-12T17:06:30.014503+0000","last_clean":"2024-03-12T17:06:30.014503+0000","last_became_active":"2024-03-12T17:06:25.456425+0000","last_became_peered":"2024-03-12T17:06:25.456425+0000","last_unstale":"2024-03-12T17:06:30.014503+0000","last_undegraded":"2024-03-12T17:06:30.014503+0000","last_fullsized":"2024-03-12T17:06:30.014503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:33:42.671579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.482388+0000","last_change":"2024-03-12T17:06:30.021976+0000","last_active":"2024-03-12T17:06:30.482388+0000","last_peered":"2024-03-12T17:06:30.482388+0000","last_clean":"2024-03-12T17:06:30.482388+0000","last_became_active":"2024-03-12T17:06:30.021512+0000","last_became_peered":"2024-03-12T17:06:30.021512+0000","last_unstale":"2024-03-12T17:06:30.482388+0000","last_undegraded":"2024-03-12T17:06:30.482388+0000","last_fullsized":"2024-03-12T17:06:30.482388+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:31:47.699683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.999227+0000","last_change":"2024-03-12T17:06:25.937491+0000","last_active":"2024-03-12T17:06:29.999227+0000","last_peered":"2024-03-12T17:06:29.999227+0000","last_clean":"2024-03-12T17:06:29.999227+0000","last_became_active":"2024-03-12T17:06:25.937173+0000","last_became_peered":"2024-03-12T17:06:25.937173+0000","last_unstale":"2024-03-12T17:06:29.999227+0000","last_undegraded":"2024-03-12T17:06:29.999227+0000","last_fullsized":"2024-03-12T17:06:29.999227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:29:49.461651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990464+0000","last_change":"2024-03-12T17:06:25.450121+0000","last_active":"2024-03-12T17:06:29.990464+0000","last_peered":"2024-03-12T17:06:29.990464+0000","last_clean":"2024-03-12T17:06:29.990464+0000","last_became_active":"2024-03-12T17:06:25.448549+0000","last_became_peered":"2024-03-12T17:06:25.448549+0000","last_unstale":"2024-03-12T17:06:29.990464+0000","last_undegraded":"2024-03-12T17:06:29.990464+0000","last_fullsized":"2024-03-12T17:06:29.990464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:36:47.407275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.481700+0000","last_change":"2024-03-12T17:06:30.020438+0000","last_active":"2024-03-12T17:06:30.481700+0000","last_peered":"2024-03-12T17:06:30.481700+0000","last_clean":"2024-03-12T17:06:30.481700+0000","last_became_active":"2024-03-12T17:06:30.020055+0000","last_became_peered":"2024-03-12T17:06:30.020055+0000","last_unstale":"2024-03-12T17:06:30.481700+0000","last_undegraded":"2024-03-12T17:06:30.481700+0000","last_fullsized":"2024-03-12T17:06:30.481700+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:17:09.055890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.004123+0000","last_change":"2024-03-12T17:06:25.947199+0000","last_active":"2024-03-12T17:06:30.004123+0000","last_peered":"2024-03-12T17:06:30.004123+0000","last_clean":"2024-03-12T17:06:30.004123+0000","last_became_active":"2024-03-12T17:06:25.941750+0000","last_became_peered":"2024-03-12T17:06:25.941750+0000","last_unstale":"2024-03-12T17:06:30.004123+0000","last_undegraded":"2024-03-12T17:06:30.004123+0000","last_fullsized":"2024-03-12T17:06:30.004123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:04:04.198678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.021973+0000","last_change":"2024-03-12T17:06:25.452843+0000","last_active":"2024-03-12T17:06:30.021973+0000","last_peered":"2024-03-12T17:06:30.021973+0000","last_clean":"2024-03-12T17:06:30.021973+0000","last_became_active":"2024-03-12T17:06:25.452410+0000","last_became_peered":"2024-03-12T17:06:25.452410+0000","last_unstale":"2024-03-12T17:06:30.021973+0000","last_undegraded":"2024-03-12T17:06:30.021973+0000","last_fullsized":"2024-03-12T17:06:30.021973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:36:25.468644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.476561+0000","last_change":"2024-03-12T17:06:30.028820+0000","last_active":"2024-03-12T17:06:30.476561+0000","last_peered":"2024-03-12T17:06:30.476561+0000","last_clean":"2024-03-12T17:06:30.476561+0000","last_became_active":"2024-03-12T17:06:30.028425+0000","last_became_peered":"2024-03-12T17:06:30.028425+0000","last_unstale":"2024-03-12T17:06:30.476561+0000","last_undegraded":"2024-03-12T17:06:30.476561+0000","last_fullsized":"2024-03-12T17:06:30.476561+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:46:40.007538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.019530+0000","last_change":"2024-03-12T17:06:25.929758+0000","last_active":"2024-03-12T17:06:30.019530+0000","last_peered":"2024-03-12T17:06:30.019530+0000","last_clean":"2024-03-12T17:06:30.019530+0000","last_became_active":"2024-03-12T17:06:25.929481+0000","last_became_peered":"2024-03-12T17:06:25.929481+0000","last_unstale":"2024-03-12T17:06:30.019530+0000","last_undegraded":"2024-03-12T17:06:30.019530+0000","last_fullsized":"2024-03-12T17:06:30.019530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:44:25.750833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012765+0000","last_change":"2024-03-12T17:06:25.456024+0000","last_active":"2024-03-12T17:06:30.012765+0000","last_peered":"2024-03-12T17:06:30.012765+0000","last_clean":"2024-03-12T17:06:30.012765+0000","last_became_active":"2024-03-12T17:06:25.455718+0000","last_became_peered":"2024-03-12T17:06:25.455718+0000","last_unstale":"2024-03-12T17:06:30.012765+0000","last_undegraded":"2024-03-12T17:06:30.012765+0000","last_fullsized":"2024-03-12T17:06:30.012765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:00:06.114876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.508448+0000","last_change":"2024-03-12T17:06:30.020641+0000","last_active":"2024-03-12T17:06:30.508448+0000","last_peered":"2024-03-12T17:06:30.508448+0000","last_clean":"2024-03-12T17:06:30.508448+0000","last_became_active":"2024-03-12T17:06:30.020373+0000","last_became_peered":"2024-03-12T17:06:30.020373+0000","last_unstale":"2024-03-12T17:06:30.508448+0000","last_undegraded":"2024-03-12T17:06:30.508448+0000","last_fullsized":"2024-03-12T17:06:30.508448+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:03:49.781309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990659+0000","last_change":"2024-03-12T17:06:25.943799+0000","last_active":"2024-03-12T17:06:29.990659+0000","last_peered":"2024-03-12T17:06:29.990659+0000","last_clean":"2024-03-12T17:06:29.990659+0000","last_became_active":"2024-03-12T17:06:25.943543+0000","last_became_peered":"2024-03-12T17:06:25.943543+0000","last_unstale":"2024-03-12T17:06:29.990659+0000","last_undegraded":"2024-03-12T17:06:29.990659+0000","last_fullsized":"2024-03-12T17:06:29.990659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:38:51.036295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.019581+0000","last_change":"2024-03-12T17:06:25.456899+0000","last_active":"2024-03-12T17:06:30.019581+0000","last_peered":"2024-03-12T17:06:30.019581+0000","last_clean":"2024-03-12T17:06:30.019581+0000","last_became_active":"2024-03-12T17:06:25.456513+0000","last_became_peered":"2024-03-12T17:06:25.456513+0000","last_unstale":"2024-03-12T17:06:30.019581+0000","last_undegraded":"2024-03-12T17:06:30.019581+0000","last_fullsized":"2024-03-12T17:06:30.019581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:58:32.509940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.508585+0000","last_change":"2024-03-12T17:06:30.018815+0000","last_active":"2024-03-12T17:06:30.508585+0000","last_peered":"2024-03-12T17:06:30.508585+0000","last_clean":"2024-03-12T17:06:30.508585+0000","last_became_active":"2024-03-12T17:06:30.014225+0000","last_became_peered":"2024-03-12T17:06:30.014225+0000","last_unstale":"2024-03-12T17:06:30.508585+0000","last_undegraded":"2024-03-12T17:06:30.508585+0000","last_fullsized":"2024-03-12T17:06:30.508585+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:49:38.782632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012440+0000","last_change":"2024-03-12T17:06:25.954609+0000","last_active":"2024-03-12T17:06:30.012440+0000","last_peered":"2024-03-12T17:06:30.012440+0000","last_clean":"2024-03-12T17:06:30.012440+0000","last_became_active":"2024-03-12T17:06:25.954337+0000","last_became_peered":"2024-03-12T17:06:25.954337+0000","last_unstale":"2024-03-12T17:06:30.012440+0000","last_undegraded":"2024-03-12T17:06:30.012440+0000","last_fullsized":"2024-03-12T17:06:30.012440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:33:43.674985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.992408+0000","last_change":"2024-03-12T17:06:25.442368+0000","last_active":"2024-03-12T17:06:29.992408+0000","last_peered":"2024-03-12T17:06:29.992408+0000","last_clean":"2024-03-12T17:06:29.992408+0000","last_became_active":"2024-03-12T17:06:25.442084+0000","last_became_peered":"2024-03-12T17:06:25.442084+0000","last_unstale":"2024-03-12T17:06:29.992408+0000","last_undegraded":"2024-03-12T17:06:29.992408+0000","last_fullsized":"2024-03-12T17:06:29.992408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:58:32.509940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.509666+0000","last_change":"2024-03-12T17:06:30.031785+0000","last_active":"2024-03-12T17:06:30.509666+0000","last_peered":"2024-03-12T17:06:30.509666+0000","last_clean":"2024-03-12T17:06:30.509666+0000","last_became_active":"2024-03-12T17:06:30.031328+0000","last_became_peered":"2024-03-12T17:06:30.031328+0000","last_unstale":"2024-03-12T17:06:30.509666+0000","last_undegraded":"2024-03-12T17:06:30.509666+0000","last_fullsized":"2024-03-12T17:06:30.509666+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:35:06.129757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012185+0000","last_change":"2024-03-12T17:06:25.931255+0000","last_active":"2024-03-12T17:06:30.012185+0000","last_peered":"2024-03-12T17:06:30.012185+0000","last_clean":"2024-03-12T17:06:30.012185+0000","last_became_active":"2024-03-12T17:06:25.929387+0000","last_became_peered":"2024-03-12T17:06:25.929387+0000","last_unstale":"2024-03-12T17:06:30.012185+0000","last_undegraded":"2024-03-12T17:06:30.012185+0000","last_fullsized":"2024-03-12T17:06:30.012185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:04:04.198678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.980151+0000","last_change":"2024-03-12T17:06:25.443901+0000","last_active":"2024-03-12T17:06:29.980151+0000","last_peered":"2024-03-12T17:06:29.980151+0000","last_clean":"2024-03-12T17:06:29.980151+0000","last_became_active":"2024-03-12T17:06:25.443602+0000","last_became_peered":"2024-03-12T17:06:25.443602+0000","last_unstale":"2024-03-12T17:06:29.980151+0000","last_undegraded":"2024-03-12T17:06:29.980151+0000","last_fullsized":"2024-03-12T17:06:29.980151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:00:32.347614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.511172+0000","last_change":"2024-03-12T17:06:30.025943+0000","last_active":"2024-03-12T17:06:30.511172+0000","last_peered":"2024-03-12T17:06:30.511172+0000","last_clean":"2024-03-12T17:06:30.511172+0000","last_became_active":"2024-03-12T17:06:30.025481+0000","last_became_peered":"2024-03-12T17:06:30.025481+0000","last_unstale":"2024-03-12T17:06:30.511172+0000","last_undegraded":"2024-03-12T17:06:30.511172+0000","last_fullsized":"2024-03-12T17:06:30.511172+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:17:09.055890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.996440+0000","last_change":"2024-03-12T17:06:25.940996+0000","last_active":"2024-03-12T17:06:29.996440+0000","last_peered":"2024-03-12T17:06:29.996440+0000","last_clean":"2024-03-12T17:06:29.996440+0000","last_became_active":"2024-03-12T17:06:25.940740+0000","last_became_peered":"2024-03-12T17:06:25.940740+0000","last_unstale":"2024-03-12T17:06:29.996440+0000","last_undegraded":"2024-03-12T17:06:29.996440+0000","last_fullsized":"2024-03-12T17:06:29.996440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:36:26.472050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.014590+0000","last_change":"2024-03-12T17:06:25.450168+0000","last_active":"2024-03-12T17:06:30.014590+0000","last_peered":"2024-03-12T17:06:30.014590+0000","last_clean":"2024-03-12T17:06:30.014590+0000","last_became_active":"2024-03-12T17:06:25.449620+0000","last_became_peered":"2024-03-12T17:06:25.449620+0000","last_unstale":"2024-03-12T17:06:30.014590+0000","last_undegraded":"2024-03-12T17:06:30.014590+0000","last_fullsized":"2024-03-12T17:06:30.014590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:58:32.509940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.509124+0000","last_change":"2024-03-12T17:06:30.020545+0000","last_active":"2024-03-12T17:06:30.509124+0000","last_peered":"2024-03-12T17:06:30.509124+0000","last_clean":"2024-03-12T17:06:30.509124+0000","last_became_active":"2024-03-12T17:06:30.020246+0000","last_became_peered":"2024-03-12T17:06:30.020246+0000","last_unstale":"2024-03-12T17:06:30.509124+0000","last_undegraded":"2024-03-12T17:06:30.509124+0000","last_fullsized":"2024-03-12T17:06:30.509124+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:08:39.661179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990996+0000","last_change":"2024-03-12T17:06:25.931831+0000","last_active":"2024-03-12T17:06:29.990996+0000","last_peered":"2024-03-12T17:06:29.990996+0000","last_clean":"2024-03-12T17:06:29.990996+0000","last_became_active":"2024-03-12T17:06:25.931525+0000","last_became_peered":"2024-03-12T17:06:25.931525+0000","last_unstale":"2024-03-12T17:06:29.990996+0000","last_undegraded":"2024-03-12T17:06:29.990996+0000","last_fullsized":"2024-03-12T17:06:29.990996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:06:19.464242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.989265+0000","last_change":"2024-03-12T17:06:25.435909+0000","last_active":"2024-03-12T17:06:29.989265+0000","last_peered":"2024-03-12T17:06:29.989265+0000","last_clean":"2024-03-12T17:06:29.989265+0000","last_became_active":"2024-03-12T17:06:25.434838+0000","last_became_peered":"2024-03-12T17:06:25.434838+0000","last_unstale":"2024-03-12T17:06:29.989265+0000","last_undegraded":"2024-03-12T17:06:29.989265+0000","last_fullsized":"2024-03-12T17:06:29.989265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:36:47.407275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.014690+0000","last_change":"2024-03-12T17:06:21.714813+0000","last_active":"2024-03-12T17:06:30.014690+0000","last_peered":"2024-03-12T17:06:30.014690+0000","last_clean":"2024-03-12T17:06:30.014690+0000","last_became_active":"2024-03-12T17:06:21.712488+0000","last_became_peered":"2024-03-12T17:06:21.712488+0000","last_unstale":"2024-03-12T17:06:30.014690+0000","last_undegraded":"2024-03-12T17:06:30.014690+0000","last_fullsized":"2024-03-12T17:06:30.014690+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:19.815688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:19.815688+0000","last_clean_scrub_stamp":"2024-03-12T17:06:19.815688+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:38:54.300793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.510890+0000","last_change":"2024-03-12T17:06:30.017415+0000","last_active":"2024-03-12T17:06:30.510890+0000","last_peered":"2024-03-12T17:06:30.510890+0000","last_clean":"2024-03-12T17:06:30.510890+0000","last_became_active":"2024-03-12T17:06:30.015976+0000","last_became_peered":"2024-03-12T17:06:30.015976+0000","last_unstale":"2024-03-12T17:06:30.510890+0000","last_undegraded":"2024-03-12T17:06:30.510890+0000","last_fullsized":"2024-03-12T17:06:30.510890+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:33:44.182800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012124+0000","last_change":"2024-03-12T17:06:25.959282+0000","last_active":"2024-03-12T17:06:30.012124+0000","last_peered":"2024-03-12T17:06:30.012124+0000","last_clean":"2024-03-12T17:06:30.012124+0000","last_became_active":"2024-03-12T17:06:25.959028+0000","last_became_peered":"2024-03-12T17:06:25.959028+0000","last_unstale":"2024-03-12T17:06:30.012124+0000","last_undegraded":"2024-03-12T17:06:30.012124+0000","last_fullsized":"2024-03-12T17:06:30.012124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:38:01.478195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.980482+0000","last_change":"2024-03-12T17:06:25.442608+0000","last_active":"2024-03-12T17:06:29.980482+0000","last_peered":"2024-03-12T17:06:29.980482+0000","last_clean":"2024-03-12T17:06:29.980482+0000","last_became_active":"2024-03-12T17:06:25.441652+0000","last_became_peered":"2024-03-12T17:06:25.441652+0000","last_unstale":"2024-03-12T17:06:29.980482+0000","last_undegraded":"2024-03-12T17:06:29.980482+0000","last_fullsized":"2024-03-12T17:06:29.980482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:34:26.277720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.510468+0000","last_change":"2024-03-12T17:06:30.020225+0000","last_active":"2024-03-12T17:06:30.510468+0000","last_peered":"2024-03-12T17:06:30.510468+0000","last_clean":"2024-03-12T17:06:30.510468+0000","last_became_active":"2024-03-12T17:06:30.017277+0000","last_became_peered":"2024-03-12T17:06:30.017277+0000","last_unstale":"2024-03-12T17:06:30.510468+0000","last_undegraded":"2024-03-12T17:06:30.510468+0000","last_fullsized":"2024-03-12T17:06:30.510468+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:35:06.129757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.991439+0000","last_change":"2024-03-12T17:06:25.942345+0000","last_active":"2024-03-12T17:06:29.991439+0000","last_peered":"2024-03-12T17:06:29.991439+0000","last_clean":"2024-03-12T17:06:29.991439+0000","last_became_active":"2024-03-12T17:06:25.941771+0000","last_became_peered":"2024-03-12T17:06:25.941771+0000","last_unstale":"2024-03-12T17:06:29.991439+0000","last_undegraded":"2024-03-12T17:06:29.991439+0000","last_fullsized":"2024-03-12T17:06:29.991439+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:32:08.311478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.022019+0000","last_change":"2024-03-12T17:06:25.448435+0000","last_active":"2024-03-12T17:06:30.022019+0000","last_peered":"2024-03-12T17:06:30.022019+0000","last_clean":"2024-03-12T17:06:30.022019+0000","last_became_active":"2024-03-12T17:06:25.448141+0000","last_became_peered":"2024-03-12T17:06:25.448141+0000","last_unstale":"2024-03-12T17:06:30.022019+0000","last_undegraded":"2024-03-12T17:06:30.022019+0000","last_fullsized":"2024-03-12T17:06:30.022019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:36:47.407275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.508634+0000","last_change":"2024-03-12T17:06:30.017805+0000","last_active":"2024-03-12T17:06:30.508634+0000","last_peered":"2024-03-12T17:06:30.508634+0000","last_clean":"2024-03-12T17:06:30.508634+0000","last_became_active":"2024-03-12T17:06:30.017383+0000","last_became_peered":"2024-03-12T17:06:30.017383+0000","last_unstale":"2024-03-12T17:06:30.508634+0000","last_undegraded":"2024-03-12T17:06:30.508634+0000","last_fullsized":"2024-03-12T17:06:30.508634+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:17:09.055890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.013754+0000","last_change":"2024-03-12T17:06:25.958864+0000","last_active":"2024-03-12T17:06:30.013754+0000","last_peered":"2024-03-12T17:06:30.013754+0000","last_clean":"2024-03-12T17:06:30.013754+0000","last_became_active":"2024-03-12T17:06:25.958588+0000","last_became_peered":"2024-03-12T17:06:25.958588+0000","last_unstale":"2024-03-12T17:06:30.013754+0000","last_undegraded":"2024-03-12T17:06:30.013754+0000","last_fullsized":"2024-03-12T17:06:30.013754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:21:02.182215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.980396+0000","last_change":"2024-03-12T17:06:25.453348+0000","last_active":"2024-03-12T17:06:29.980396+0000","last_peered":"2024-03-12T17:06:29.980396+0000","last_clean":"2024-03-12T17:06:29.980396+0000","last_became_active":"2024-03-12T17:06:25.449918+0000","last_became_peered":"2024-03-12T17:06:25.449918+0000","last_unstale":"2024-03-12T17:06:29.980396+0000","last_undegraded":"2024-03-12T17:06:29.980396+0000","last_fullsized":"2024-03-12T17:06:29.980396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:49:52.284732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.479802+0000","last_change":"2024-03-12T17:06:30.023418+0000","last_active":"2024-03-12T17:06:30.479802+0000","last_peered":"2024-03-12T17:06:30.479802+0000","last_clean":"2024-03-12T17:06:30.479802+0000","last_became_active":"2024-03-12T17:06:30.022919+0000","last_became_peered":"2024-03-12T17:06:30.022919+0000","last_unstale":"2024-03-12T17:06:30.479802+0000","last_undegraded":"2024-03-12T17:06:30.479802+0000","last_fullsized":"2024-03-12T17:06:30.479802+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:13:55.311599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.015933+0000","last_change":"2024-03-12T17:06:25.961193+0000","last_active":"2024-03-12T17:06:30.015933+0000","last_peered":"2024-03-12T17:06:30.015933+0000","last_clean":"2024-03-12T17:06:30.015933+0000","last_became_active":"2024-03-12T17:06:25.960906+0000","last_became_peered":"2024-03-12T17:06:25.960906+0000","last_unstale":"2024-03-12T17:06:30.015933+0000","last_undegraded":"2024-03-12T17:06:30.015933+0000","last_fullsized":"2024-03-12T17:06:30.015933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:17:09.067136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.013327+0000","last_change":"2024-03-12T17:06:25.442363+0000","last_active":"2024-03-12T17:06:30.013327+0000","last_peered":"2024-03-12T17:06:30.013327+0000","last_clean":"2024-03-12T17:06:30.013327+0000","last_became_active":"2024-03-12T17:06:25.442053+0000","last_became_peered":"2024-03-12T17:06:25.442053+0000","last_unstale":"2024-03-12T17:06:30.013327+0000","last_undegraded":"2024-03-12T17:06:30.013327+0000","last_fullsized":"2024-03-12T17:06:30.013327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:06:05.702956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.481199+0000","last_change":"2024-03-12T17:06:30.020741+0000","last_active":"2024-03-12T17:06:30.481199+0000","last_peered":"2024-03-12T17:06:30.481199+0000","last_clean":"2024-03-12T17:06:30.481199+0000","last_became_active":"2024-03-12T17:06:30.018043+0000","last_became_peered":"2024-03-12T17:06:30.018043+0000","last_unstale":"2024-03-12T17:06:30.481199+0000","last_undegraded":"2024-03-12T17:06:30.481199+0000","last_fullsized":"2024-03-12T17:06:30.481199+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:11:34.105824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.016104+0000","last_change":"2024-03-12T17:06:25.958260+0000","last_active":"2024-03-12T17:06:30.016104+0000","last_peered":"2024-03-12T17:06:30.016104+0000","last_clean":"2024-03-12T17:06:30.016104+0000","last_became_active":"2024-03-12T17:06:25.955966+0000","last_became_peered":"2024-03-12T17:06:25.955966+0000","last_unstale":"2024-03-12T17:06:30.016104+0000","last_undegraded":"2024-03-12T17:06:30.016104+0000","last_fullsized":"2024-03-12T17:06:30.016104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:33:16.553805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.979851+0000","last_change":"2024-03-12T17:06:25.452222+0000","last_active":"2024-03-12T17:06:29.979851+0000","last_peered":"2024-03-12T17:06:29.979851+0000","last_clean":"2024-03-12T17:06:29.979851+0000","last_became_active":"2024-03-12T17:06:25.451696+0000","last_became_peered":"2024-03-12T17:06:25.451696+0000","last_unstale":"2024-03-12T17:06:29.979851+0000","last_undegraded":"2024-03-12T17:06:29.979851+0000","last_fullsized":"2024-03-12T17:06:29.979851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:28:25.809578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.508879+0000","last_change":"2024-03-12T17:06:30.009551+0000","last_active":"2024-03-12T17:06:30.508879+0000","last_peered":"2024-03-12T17:06:30.508879+0000","last_clean":"2024-03-12T17:06:30.508879+0000","last_became_active":"2024-03-12T17:06:30.009269+0000","last_became_peered":"2024-03-12T17:06:30.009269+0000","last_unstale":"2024-03-12T17:06:30.508879+0000","last_undegraded":"2024-03-12T17:06:30.508879+0000","last_fullsized":"2024-03-12T17:06:30.508879+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:42:49.327905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.984652+0000","last_change":"2024-03-12T17:06:25.949275+0000","last_active":"2024-03-12T17:06:29.984652+0000","last_peered":"2024-03-12T17:06:29.984652+0000","last_clean":"2024-03-12T17:06:29.984652+0000","last_became_active":"2024-03-12T17:06:25.947884+0000","last_became_peered":"2024-03-12T17:06:25.947884+0000","last_unstale":"2024-03-12T17:06:29.984652+0000","last_undegraded":"2024-03-12T17:06:29.984652+0000","last_fullsized":"2024-03-12T17:06:29.984652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:28:59.465721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.984927+0000","last_change":"2024-03-12T17:06:25.450408+0000","last_active":"2024-03-12T17:06:29.984927+0000","last_peered":"2024-03-12T17:06:29.984927+0000","last_clean":"2024-03-12T17:06:29.984927+0000","last_became_active":"2024-03-12T17:06:25.448867+0000","last_became_peered":"2024-03-12T17:06:25.448867+0000","last_unstale":"2024-03-12T17:06:29.984927+0000","last_undegraded":"2024-03-12T17:06:29.984927+0000","last_fullsized":"2024-03-12T17:06:29.984927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:49:52.284732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.477643+0000","last_change":"2024-03-12T17:06:30.022424+0000","last_active":"2024-03-12T17:06:30.477643+0000","last_peered":"2024-03-12T17:06:30.477643+0000","last_clean":"2024-03-12T17:06:30.477643+0000","last_became_active":"2024-03-12T17:06:30.021994+0000","last_became_peered":"2024-03-12T17:06:30.021994+0000","last_unstale":"2024-03-12T17:06:30.477643+0000","last_undegraded":"2024-03-12T17:06:30.477643+0000","last_fullsized":"2024-03-12T17:06:30.477643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:32:37.399700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012128+0000","last_change":"2024-03-12T17:06:25.958282+0000","last_active":"2024-03-12T17:06:30.012128+0000","last_peered":"2024-03-12T17:06:30.012128+0000","last_clean":"2024-03-12T17:06:30.012128+0000","last_became_active":"2024-03-12T17:06:25.955357+0000","last_became_peered":"2024-03-12T17:06:25.955357+0000","last_unstale":"2024-03-12T17:06:30.012128+0000","last_undegraded":"2024-03-12T17:06:30.012128+0000","last_fullsized":"2024-03-12T17:06:30.012128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:15:36.195143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.992283+0000","last_change":"2024-03-12T17:06:25.448887+0000","last_active":"2024-03-12T17:06:29.992283+0000","last_peered":"2024-03-12T17:06:29.992283+0000","last_clean":"2024-03-12T17:06:29.992283+0000","last_became_active":"2024-03-12T17:06:25.448299+0000","last_became_peered":"2024-03-12T17:06:25.448299+0000","last_unstale":"2024-03-12T17:06:29.992283+0000","last_undegraded":"2024-03-12T17:06:29.992283+0000","last_fullsized":"2024-03-12T17:06:29.992283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:00:06.114876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.477924+0000","last_change":"2024-03-12T17:06:30.022221+0000","last_active":"2024-03-12T17:06:30.477924+0000","last_peered":"2024-03-12T17:06:30.477924+0000","last_clean":"2024-03-12T17:06:30.477924+0000","last_became_active":"2024-03-12T17:06:30.021765+0000","last_became_peered":"2024-03-12T17:06:30.021765+0000","last_unstale":"2024-03-12T17:06:30.477924+0000","last_undegraded":"2024-03-12T17:06:30.477924+0000","last_fullsized":"2024-03-12T17:06:30.477924+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:28:34.686064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.996652+0000","last_change":"2024-03-12T17:06:25.944466+0000","last_active":"2024-03-12T17:06:29.996652+0000","last_peered":"2024-03-12T17:06:29.996652+0000","last_clean":"2024-03-12T17:06:29.996652+0000","last_became_active":"2024-03-12T17:06:25.944075+0000","last_became_peered":"2024-03-12T17:06:25.944075+0000","last_unstale":"2024-03-12T17:06:29.996652+0000","last_undegraded":"2024-03-12T17:06:29.996652+0000","last_fullsized":"2024-03-12T17:06:29.996652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T20:48:48.332409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.996480+0000","last_change":"2024-03-12T17:06:25.444436+0000","last_active":"2024-03-12T17:06:29.996480+0000","last_peered":"2024-03-12T17:06:29.996480+0000","last_clean":"2024-03-12T17:06:29.996480+0000","last_became_active":"2024-03-12T17:06:25.443700+0000","last_became_peered":"2024-03-12T17:06:25.443700+0000","last_unstale":"2024-03-12T17:06:29.996480+0000","last_undegraded":"2024-03-12T17:06:29.996480+0000","last_fullsized":"2024-03-12T17:06:29.996480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:55:17.158520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.483163+0000","last_change":"2024-03-12T17:06:30.022439+0000","last_active":"2024-03-12T17:06:30.483163+0000","last_peered":"2024-03-12T17:06:30.483163+0000","last_clean":"2024-03-12T17:06:30.483163+0000","last_became_active":"2024-03-12T17:06:30.017040+0000","last_became_peered":"2024-03-12T17:06:30.017040+0000","last_unstale":"2024-03-12T17:06:30.483163+0000","last_undegraded":"2024-03-12T17:06:30.483163+0000","last_fullsized":"2024-03-12T17:06:30.483163+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:08:39.661179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012020+0000","last_change":"2024-03-12T17:06:25.958251+0000","last_active":"2024-03-12T17:06:30.012020+0000","last_peered":"2024-03-12T17:06:30.012020+0000","last_clean":"2024-03-12T17:06:30.012020+0000","last_became_active":"2024-03-12T17:06:25.956505+0000","last_became_peered":"2024-03-12T17:06:25.956505+0000","last_unstale":"2024-03-12T17:06:30.012020+0000","last_undegraded":"2024-03-12T17:06:30.012020+0000","last_fullsized":"2024-03-12T17:06:30.012020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:32:08.311478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:44.758048+0000","last_change":"2024-03-12T17:06:25.448251+0000","last_active":"2024-03-12T17:06:44.758048+0000","last_peered":"2024-03-12T17:06:44.758048+0000","last_clean":"2024-03-12T17:06:44.758048+0000","last_became_active":"2024-03-12T17:06:25.447768+0000","last_became_peered":"2024-03-12T17:06:25.447768+0000","last_unstale":"2024-03-12T17:06:44.758048+0000","last_undegraded":"2024-03-12T17:06:44.758048+0000","last_fullsized":"2024-03-12T17:06:44.758048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:34:26.277720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3795,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.508684+0000","last_change":"2024-03-12T17:06:30.019328+0000","last_active":"2024-03-12T17:06:30.508684+0000","last_peered":"2024-03-12T17:06:30.508684+0000","last_clean":"2024-03-12T17:06:30.508684+0000","last_became_active":"2024-03-12T17:06:30.014815+0000","last_became_peered":"2024-03-12T17:06:30.014815+0000","last_unstale":"2024-03-12T17:06:30.508684+0000","last_undegraded":"2024-03-12T17:06:30.508684+0000","last_fullsized":"2024-03-12T17:06:30.508684+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:08:39.661179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012873+0000","last_change":"2024-03-12T17:06:25.945208+0000","last_active":"2024-03-12T17:06:30.012873+0000","last_peered":"2024-03-12T17:06:30.012873+0000","last_clean":"2024-03-12T17:06:30.012873+0000","last_became_active":"2024-03-12T17:06:25.944043+0000","last_became_peered":"2024-03-12T17:06:25.944043+0000","last_unstale":"2024-03-12T17:06:30.012873+0000","last_undegraded":"2024-03-12T17:06:30.012873+0000","last_fullsized":"2024-03-12T17:06:30.012873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:36:48.410681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.021900+0000","last_change":"2024-03-12T17:06:25.455080+0000","last_active":"2024-03-12T17:06:30.021900+0000","last_peered":"2024-03-12T17:06:30.021900+0000","last_clean":"2024-03-12T17:06:30.021900+0000","last_became_active":"2024-03-12T17:06:25.454788+0000","last_became_peered":"2024-03-12T17:06:25.454788+0000","last_unstale":"2024-03-12T17:06:30.021900+0000","last_undegraded":"2024-03-12T17:06:30.021900+0000","last_fullsized":"2024-03-12T17:06:30.021900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:33:42.671579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.482470+0000","last_change":"2024-03-12T17:06:30.018328+0000","last_active":"2024-03-12T17:06:30.482470+0000","last_peered":"2024-03-12T17:06:30.482470+0000","last_clean":"2024-03-12T17:06:30.482470+0000","last_became_active":"2024-03-12T17:06:30.017852+0000","last_became_peered":"2024-03-12T17:06:30.017852+0000","last_unstale":"2024-03-12T17:06:30.482470+0000","last_undegraded":"2024-03-12T17:06:30.482470+0000","last_fullsized":"2024-03-12T17:06:30.482470+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T02:49:07.614077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.999490+0000","last_change":"2024-03-12T17:06:25.938178+0000","last_active":"2024-03-12T17:06:29.999490+0000","last_peered":"2024-03-12T17:06:29.999490+0000","last_clean":"2024-03-12T17:06:29.999490+0000","last_became_active":"2024-03-12T17:06:25.937926+0000","last_became_peered":"2024-03-12T17:06:25.937926+0000","last_unstale":"2024-03-12T17:06:29.999490+0000","last_undegraded":"2024-03-12T17:06:29.999490+0000","last_fullsized":"2024-03-12T17:06:29.999490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:57:56.009742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990778+0000","last_change":"2024-03-12T17:06:25.451993+0000","last_active":"2024-03-12T17:06:29.990778+0000","last_peered":"2024-03-12T17:06:29.990778+0000","last_clean":"2024-03-12T17:06:29.990778+0000","last_became_active":"2024-03-12T17:06:25.451794+0000","last_became_peered":"2024-03-12T17:06:25.451794+0000","last_unstale":"2024-03-12T17:06:29.990778+0000","last_undegraded":"2024-03-12T17:06:29.990778+0000","last_fullsized":"2024-03-12T17:06:29.990778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:33:42.671579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.476846+0000","last_change":"2024-03-12T17:06:30.016232+0000","last_active":"2024-03-12T17:06:30.476846+0000","last_peered":"2024-03-12T17:06:30.476846+0000","last_clean":"2024-03-12T17:06:30.476846+0000","last_became_active":"2024-03-12T17:06:30.015825+0000","last_became_peered":"2024-03-12T17:06:30.015825+0000","last_unstale":"2024-03-12T17:06:30.476846+0000","last_undegraded":"2024-03-12T17:06:30.476846+0000","last_fullsized":"2024-03-12T17:06:30.476846+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:28:28.745853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990537+0000","last_change":"2024-03-12T17:06:25.938249+0000","last_active":"2024-03-12T17:06:29.990537+0000","last_peered":"2024-03-12T17:06:29.990537+0000","last_clean":"2024-03-12T17:06:29.990537+0000","last_became_active":"2024-03-12T17:06:25.937744+0000","last_became_peered":"2024-03-12T17:06:25.937744+0000","last_unstale":"2024-03-12T17:06:29.990537+0000","last_undegraded":"2024-03-12T17:06:29.990537+0000","last_fullsized":"2024-03-12T17:06:29.990537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:06:19.464242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990879+0000","last_change":"2024-03-12T17:06:25.452356+0000","last_active":"2024-03-12T17:06:29.990879+0000","last_peered":"2024-03-12T17:06:29.990879+0000","last_clean":"2024-03-12T17:06:29.990879+0000","last_became_active":"2024-03-12T17:06:25.452033+0000","last_became_peered":"2024-03-12T17:06:25.452033+0000","last_unstale":"2024-03-12T17:06:29.990879+0000","last_undegraded":"2024-03-12T17:06:29.990879+0000","last_fullsized":"2024-03-12T17:06:29.990879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:58:32.509940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.477019+0000","last_change":"2024-03-12T17:06:30.023373+0000","last_active":"2024-03-12T17:06:30.477019+0000","last_peered":"2024-03-12T17:06:30.477019+0000","last_clean":"2024-03-12T17:06:30.477019+0000","last_became_active":"2024-03-12T17:06:30.022331+0000","last_became_peered":"2024-03-12T17:06:30.022331+0000","last_unstale":"2024-03-12T17:06:30.477019+0000","last_undegraded":"2024-03-12T17:06:30.477019+0000","last_fullsized":"2024-03-12T17:06:30.477019+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:50:22.021457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.508638+0000","last_change":"2024-03-12T17:06:30.020503+0000","last_active":"2024-03-12T17:06:30.508638+0000","last_peered":"2024-03-12T17:06:30.508638+0000","last_clean":"2024-03-12T17:06:30.508638+0000","last_became_active":"2024-03-12T17:06:30.019897+0000","last_became_peered":"2024-03-12T17:06:30.019897+0000","last_unstale":"2024-03-12T17:06:30.508638+0000","last_undegraded":"2024-03-12T17:06:30.508638+0000","last_fullsized":"2024-03-12T17:06:30.508638+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:13:55.311599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.013985+0000","last_change":"2024-03-12T17:06:25.927847+0000","last_active":"2024-03-12T17:06:30.013985+0000","last_peered":"2024-03-12T17:06:30.013985+0000","last_clean":"2024-03-12T17:06:30.013985+0000","last_became_active":"2024-03-12T17:06:25.927572+0000","last_became_peered":"2024-03-12T17:06:25.927572+0000","last_unstale":"2024-03-12T17:06:30.013985+0000","last_undegraded":"2024-03-12T17:06:30.013985+0000","last_fullsized":"2024-03-12T17:06:30.013985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:29:49.461651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.989420+0000","last_change":"2024-03-12T17:06:25.443473+0000","last_active":"2024-03-12T17:06:29.989420+0000","last_peered":"2024-03-12T17:06:29.989420+0000","last_clean":"2024-03-12T17:06:29.989420+0000","last_became_active":"2024-03-12T17:06:25.443046+0000","last_became_peered":"2024-03-12T17:06:25.443046+0000","last_unstale":"2024-03-12T17:06:29.989420+0000","last_undegraded":"2024-03-12T17:06:29.989420+0000","last_fullsized":"2024-03-12T17:06:29.989420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:49:52.284732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.513045+0000","last_change":"2024-03-12T17:06:30.017610+0000","last_active":"2024-03-12T17:06:30.513045+0000","last_peered":"2024-03-12T17:06:30.513045+0000","last_clean":"2024-03-12T17:06:30.513045+0000","last_became_active":"2024-03-12T17:06:30.017119+0000","last_became_peered":"2024-03-12T17:06:30.017119+0000","last_unstale":"2024-03-12T17:06:30.513045+0000","last_undegraded":"2024-03-12T17:06:30.513045+0000","last_fullsized":"2024-03-12T17:06:30.513045+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:42:49.327905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.986700+0000","last_change":"2024-03-12T17:06:25.935139+0000","last_active":"2024-03-12T17:06:29.986700+0000","last_peered":"2024-03-12T17:06:29.986700+0000","last_clean":"2024-03-12T17:06:29.986700+0000","last_became_active":"2024-03-12T17:06:25.934842+0000","last_became_peered":"2024-03-12T17:06:25.934842+0000","last_unstale":"2024-03-12T17:06:29.986700+0000","last_undegraded":"2024-03-12T17:06:29.986700+0000","last_fullsized":"2024-03-12T17:06:29.986700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:28:59.465721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.989691+0000","last_change":"2024-03-12T17:06:25.447398+0000","last_active":"2024-03-12T17:06:29.989691+0000","last_peered":"2024-03-12T17:06:29.989691+0000","last_clean":"2024-03-12T17:06:29.989691+0000","last_became_active":"2024-03-12T17:06:25.447080+0000","last_became_peered":"2024-03-12T17:06:25.447080+0000","last_unstale":"2024-03-12T17:06:29.989691+0000","last_undegraded":"2024-03-12T17:06:29.989691+0000","last_fullsized":"2024-03-12T17:06:29.989691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:00:32.347614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.986717+0000","last_change":"2024-03-12T17:06:25.447858+0000","last_active":"2024-03-12T17:06:29.986717+0000","last_peered":"2024-03-12T17:06:29.986717+0000","last_clean":"2024-03-12T17:06:29.986717+0000","last_became_active":"2024-03-12T17:06:25.447551+0000","last_became_peered":"2024-03-12T17:06:25.447551+0000","last_unstale":"2024-03-12T17:06:29.986717+0000","last_undegraded":"2024-03-12T17:06:29.986717+0000","last_fullsized":"2024-03-12T17:06:29.986717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:28:25.809578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.996427+0000","last_change":"2024-03-12T17:06:25.943160+0000","last_active":"2024-03-12T17:06:29.996427+0000","last_peered":"2024-03-12T17:06:29.996427+0000","last_clean":"2024-03-12T17:06:29.996427+0000","last_became_active":"2024-03-12T17:06:25.942883+0000","last_became_peered":"2024-03-12T17:06:25.942883+0000","last_unstale":"2024-03-12T17:06:29.996427+0000","last_undegraded":"2024-03-12T17:06:29.996427+0000","last_fullsized":"2024-03-12T17:06:29.996427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:04:04.198678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.509831+0000","last_change":"2024-03-12T17:06:30.018594+0000","last_active":"2024-03-12T17:06:30.509831+0000","last_peered":"2024-03-12T17:06:30.509831+0000","last_clean":"2024-03-12T17:06:30.509831+0000","last_became_active":"2024-03-12T17:06:30.016503+0000","last_became_peered":"2024-03-12T17:06:30.016503+0000","last_unstale":"2024-03-12T17:06:30.509831+0000","last_undegraded":"2024-03-12T17:06:30.509831+0000","last_fullsized":"2024-03-12T17:06:30.509831+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:29:03.020890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.995919+0000","last_change":"2024-03-12T17:06:25.443901+0000","last_active":"2024-03-12T17:06:29.995919+0000","last_peered":"2024-03-12T17:06:29.995919+0000","last_clean":"2024-03-12T17:06:29.995919+0000","last_became_active":"2024-03-12T17:06:25.443607+0000","last_became_peered":"2024-03-12T17:06:25.443607+0000","last_unstale":"2024-03-12T17:06:29.995919+0000","last_undegraded":"2024-03-12T17:06:29.995919+0000","last_fullsized":"2024-03-12T17:06:29.995919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T05:00:49.177317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.011582+0000","last_change":"2024-03-12T17:06:25.931011+0000","last_active":"2024-03-12T17:06:30.011582+0000","last_peered":"2024-03-12T17:06:30.011582+0000","last_clean":"2024-03-12T17:06:30.011582+0000","last_became_active":"2024-03-12T17:06:25.930306+0000","last_became_peered":"2024-03-12T17:06:25.930306+0000","last_unstale":"2024-03-12T17:06:30.011582+0000","last_undegraded":"2024-03-12T17:06:30.011582+0000","last_fullsized":"2024-03-12T17:06:30.011582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:58:33.513346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.478204+0000","last_change":"2024-03-12T17:06:30.018640+0000","last_active":"2024-03-12T17:06:30.478204+0000","last_peered":"2024-03-12T17:06:30.478204+0000","last_clean":"2024-03-12T17:06:30.478204+0000","last_became_active":"2024-03-12T17:06:30.018278+0000","last_became_peered":"2024-03-12T17:06:30.018278+0000","last_unstale":"2024-03-12T17:06:30.478204+0000","last_undegraded":"2024-03-12T17:06:30.478204+0000","last_fullsized":"2024-03-12T17:06:30.478204+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:01:05.684140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.984555+0000","last_change":"2024-03-12T17:06:25.947304+0000","last_active":"2024-03-12T17:06:29.984555+0000","last_peered":"2024-03-12T17:06:29.984555+0000","last_clean":"2024-03-12T17:06:29.984555+0000","last_became_active":"2024-03-12T17:06:25.947030+0000","last_became_peered":"2024-03-12T17:06:25.947030+0000","last_unstale":"2024-03-12T17:06:29.984555+0000","last_undegraded":"2024-03-12T17:06:29.984555+0000","last_fullsized":"2024-03-12T17:06:29.984555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:29:49.461651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.979886+0000","last_change":"2024-03-12T17:06:25.929080+0000","last_active":"2024-03-12T17:06:29.979886+0000","last_peered":"2024-03-12T17:06:29.979886+0000","last_clean":"2024-03-12T17:06:29.979886+0000","last_became_active":"2024-03-12T17:06:25.928820+0000","last_became_peered":"2024-03-12T17:06:25.928820+0000","last_unstale":"2024-03-12T17:06:29.979886+0000","last_undegraded":"2024-03-12T17:06:29.979886+0000","last_fullsized":"2024-03-12T17:06:29.979886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:36:26.472050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.992041+0000","last_change":"2024-03-12T17:06:25.932309+0000","last_active":"2024-03-12T17:06:29.992041+0000","last_peered":"2024-03-12T17:06:29.992041+0000","last_clean":"2024-03-12T17:06:29.992041+0000","last_became_active":"2024-03-12T17:06:25.932014+0000","last_became_peered":"2024-03-12T17:06:25.932014+0000","last_unstale":"2024-03-12T17:06:29.992041+0000","last_undegraded":"2024-03-12T17:06:29.992041+0000","last_fullsized":"2024-03-12T17:06:29.992041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:11:15.145295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.996376+0000","last_change":"2024-03-12T17:06:25.937407+0000","last_active":"2024-03-12T17:06:29.996376+0000","last_peered":"2024-03-12T17:06:29.996376+0000","last_clean":"2024-03-12T17:06:29.996376+0000","last_became_active":"2024-03-12T17:06:25.937065+0000","last_became_peered":"2024-03-12T17:06:25.937065+0000","last_unstale":"2024-03-12T17:06:29.996376+0000","last_undegraded":"2024-03-12T17:06:29.996376+0000","last_fullsized":"2024-03-12T17:06:29.996376+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:04:28.921316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.994102+0000","last_change":"2024-03-12T17:06:25.930358+0000","last_active":"2024-03-12T17:06:29.994102+0000","last_peered":"2024-03-12T17:06:29.994102+0000","last_clean":"2024-03-12T17:06:29.994102+0000","last_became_active":"2024-03-12T17:06:25.929870+0000","last_became_peered":"2024-03-12T17:06:25.929870+0000","last_unstale":"2024-03-12T17:06:29.994102+0000","last_undegraded":"2024-03-12T17:06:29.994102+0000","last_fullsized":"2024-03-12T17:06:29.994102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:57:56.009742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.021765+0000","last_change":"2024-03-12T17:06:25.928987+0000","last_active":"2024-03-12T17:06:30.021765+0000","last_peered":"2024-03-12T17:06:30.021765+0000","last_clean":"2024-03-12T17:06:30.021765+0000","last_became_active":"2024-03-12T17:06:25.928717+0000","last_became_peered":"2024-03-12T17:06:25.928717+0000","last_unstale":"2024-03-12T17:06:30.021765+0000","last_undegraded":"2024-03-12T17:06:30.021765+0000","last_fullsized":"2024-03-12T17:06:30.021765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:23:25.984714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990238+0000","last_change":"2024-03-12T17:06:25.943871+0000","last_active":"2024-03-12T17:06:29.990238+0000","last_peered":"2024-03-12T17:06:29.990238+0000","last_clean":"2024-03-12T17:06:29.990238+0000","last_became_active":"2024-03-12T17:06:25.938802+0000","last_became_peered":"2024-03-12T17:06:25.938802+0000","last_unstale":"2024-03-12T17:06:29.990238+0000","last_undegraded":"2024-03-12T17:06:29.990238+0000","last_fullsized":"2024-03-12T17:06:29.990238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:23:25.984714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.004028+0000","last_change":"2024-03-12T17:06:25.935272+0000","last_active":"2024-03-12T17:06:30.004028+0000","last_peered":"2024-03-12T17:06:30.004028+0000","last_clean":"2024-03-12T17:06:30.004028+0000","last_became_active":"2024-03-12T17:06:25.928645+0000","last_became_peered":"2024-03-12T17:06:25.928645+0000","last_unstale":"2024-03-12T17:06:30.004028+0000","last_undegraded":"2024-03-12T17:06:30.004028+0000","last_fullsized":"2024-03-12T17:06:30.004028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:44:25.750833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.989166+0000","last_change":"2024-03-12T17:06:25.937035+0000","last_active":"2024-03-12T17:06:29.989166+0000","last_peered":"2024-03-12T17:06:29.989166+0000","last_clean":"2024-03-12T17:06:29.989166+0000","last_became_active":"2024-03-12T17:06:25.936598+0000","last_became_peered":"2024-03-12T17:06:25.936598+0000","last_unstale":"2024-03-12T17:06:29.989166+0000","last_undegraded":"2024-03-12T17:06:29.989166+0000","last_fullsized":"2024-03-12T17:06:29.989166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:21:02.182215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.989614+0000","last_change":"2024-03-12T17:06:25.931310+0000","last_active":"2024-03-12T17:06:29.989614+0000","last_peered":"2024-03-12T17:06:29.989614+0000","last_clean":"2024-03-12T17:06:29.989614+0000","last_became_active":"2024-03-12T17:06:25.930910+0000","last_became_peered":"2024-03-12T17:06:25.930910+0000","last_unstale":"2024-03-12T17:06:29.989614+0000","last_undegraded":"2024-03-12T17:06:29.989614+0000","last_fullsized":"2024-03-12T17:06:29.989614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:48:33.016370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.991821+0000","last_change":"2024-03-12T17:06:25.937111+0000","last_active":"2024-03-12T17:06:29.991821+0000","last_peered":"2024-03-12T17:06:29.991821+0000","last_clean":"2024-03-12T17:06:29.991821+0000","last_became_active":"2024-03-12T17:06:25.936895+0000","last_became_peered":"2024-03-12T17:06:25.936895+0000","last_unstale":"2024-03-12T17:06:29.991821+0000","last_undegraded":"2024-03-12T17:06:29.991821+0000","last_fullsized":"2024-03-12T17:06:29.991821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:38:01.478195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.996309+0000","last_change":"2024-03-12T17:06:25.947115+0000","last_active":"2024-03-12T17:06:29.996309+0000","last_peered":"2024-03-12T17:06:29.996309+0000","last_clean":"2024-03-12T17:06:29.996309+0000","last_became_active":"2024-03-12T17:06:25.946820+0000","last_became_peered":"2024-03-12T17:06:25.946820+0000","last_unstale":"2024-03-12T17:06:29.996309+0000","last_undegraded":"2024-03-12T17:06:29.996309+0000","last_fullsized":"2024-03-12T17:06:29.996309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:48:33.016370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.984992+0000","last_change":"2024-03-12T17:06:25.950973+0000","last_active":"2024-03-12T17:06:29.984992+0000","last_peered":"2024-03-12T17:06:29.984992+0000","last_clean":"2024-03-12T17:06:29.984992+0000","last_became_active":"2024-03-12T17:06:25.948336+0000","last_became_peered":"2024-03-12T17:06:25.948336+0000","last_unstale":"2024-03-12T17:06:29.984992+0000","last_undegraded":"2024-03-12T17:06:29.984992+0000","last_fullsized":"2024-03-12T17:06:29.984992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:04:04.198678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012237+0000","last_change":"2024-03-12T17:06:25.961536+0000","last_active":"2024-03-12T17:06:30.012237+0000","last_peered":"2024-03-12T17:06:30.012237+0000","last_clean":"2024-03-12T17:06:30.012237+0000","last_became_active":"2024-03-12T17:06:25.961147+0000","last_became_peered":"2024-03-12T17:06:25.961147+0000","last_unstale":"2024-03-12T17:06:30.012237+0000","last_undegraded":"2024-03-12T17:06:30.012237+0000","last_fullsized":"2024-03-12T17:06:30.012237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:06:19.464242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.979994+0000","last_change":"2024-03-12T17:06:25.939977+0000","last_active":"2024-03-12T17:06:29.979994+0000","last_peered":"2024-03-12T17:06:29.979994+0000","last_clean":"2024-03-12T17:06:29.979994+0000","last_became_active":"2024-03-12T17:06:25.939725+0000","last_became_peered":"2024-03-12T17:06:25.939725+0000","last_unstale":"2024-03-12T17:06:29.979994+0000","last_undegraded":"2024-03-12T17:06:29.979994+0000","last_fullsized":"2024-03-12T17:06:29.979994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:45:02.929901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.991517+0000","last_change":"2024-03-12T17:06:25.939649+0000","last_active":"2024-03-12T17:06:29.991517+0000","last_peered":"2024-03-12T17:06:29.991517+0000","last_clean":"2024-03-12T17:06:29.991517+0000","last_became_active":"2024-03-12T17:06:25.939379+0000","last_became_peered":"2024-03-12T17:06:25.939379+0000","last_unstale":"2024-03-12T17:06:29.991517+0000","last_undegraded":"2024-03-12T17:06:29.991517+0000","last_fullsized":"2024-03-12T17:06:29.991517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:17:09.067136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.988936+0000","last_change":"2024-03-12T17:06:25.948556+0000","last_active":"2024-03-12T17:06:29.988936+0000","last_peered":"2024-03-12T17:06:29.988936+0000","last_clean":"2024-03-12T17:06:29.988936+0000","last_became_active":"2024-03-12T17:06:25.948217+0000","last_became_peered":"2024-03-12T17:06:25.948217+0000","last_unstale":"2024-03-12T17:06:29.988936+0000","last_undegraded":"2024-03-12T17:06:29.988936+0000","last_fullsized":"2024-03-12T17:06:29.988936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:26:54.235648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.013984+0000","last_change":"2024-03-12T17:06:25.931098+0000","last_active":"2024-03-12T17:06:30.013984+0000","last_peered":"2024-03-12T17:06:30.013984+0000","last_clean":"2024-03-12T17:06:30.013984+0000","last_became_active":"2024-03-12T17:06:25.930273+0000","last_became_peered":"2024-03-12T17:06:25.930273+0000","last_unstale":"2024-03-12T17:06:30.013984+0000","last_undegraded":"2024-03-12T17:06:30.013984+0000","last_fullsized":"2024-03-12T17:06:30.013984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T20:48:48.332409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.996609+0000","last_change":"2024-03-12T17:06:25.933421+0000","last_active":"2024-03-12T17:06:29.996609+0000","last_peered":"2024-03-12T17:06:29.996609+0000","last_clean":"2024-03-12T17:06:29.996609+0000","last_became_active":"2024-03-12T17:06:25.933133+0000","last_became_peered":"2024-03-12T17:06:25.933133+0000","last_unstale":"2024-03-12T17:06:29.996609+0000","last_undegraded":"2024-03-12T17:06:29.996609+0000","last_fullsized":"2024-03-12T17:06:29.996609+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:28:59.465721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.991543+0000","last_change":"2024-03-12T17:06:25.948622+0000","last_active":"2024-03-12T17:06:29.991543+0000","last_peered":"2024-03-12T17:06:29.991543+0000","last_clean":"2024-03-12T17:06:29.991543+0000","last_became_active":"2024-03-12T17:06:25.948293+0000","last_became_peered":"2024-03-12T17:06:25.948293+0000","last_unstale":"2024-03-12T17:06:29.991543+0000","last_undegraded":"2024-03-12T17:06:29.991543+0000","last_fullsized":"2024-03-12T17:06:29.991543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:52:14.405796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012611+0000","last_change":"2024-03-12T17:06:25.944171+0000","last_active":"2024-03-12T17:06:30.012611+0000","last_peered":"2024-03-12T17:06:30.012611+0000","last_clean":"2024-03-12T17:06:30.012611+0000","last_became_active":"2024-03-12T17:06:25.940309+0000","last_became_peered":"2024-03-12T17:06:25.940309+0000","last_unstale":"2024-03-12T17:06:30.012611+0000","last_undegraded":"2024-03-12T17:06:30.012611+0000","last_fullsized":"2024-03-12T17:06:30.012611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:00:07.118282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.987793+0000","last_change":"2024-03-12T17:06:25.946194+0000","last_active":"2024-03-12T17:06:29.987793+0000","last_peered":"2024-03-12T17:06:29.987793+0000","last_clean":"2024-03-12T17:06:29.987793+0000","last_became_active":"2024-03-12T17:06:25.944383+0000","last_became_peered":"2024-03-12T17:06:25.944383+0000","last_unstale":"2024-03-12T17:06:29.987793+0000","last_undegraded":"2024-03-12T17:06:29.987793+0000","last_fullsized":"2024-03-12T17:06:29.987793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:48:33.016370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.989993+0000","last_change":"2024-03-12T17:06:25.936660+0000","last_active":"2024-03-12T17:06:29.989993+0000","last_peered":"2024-03-12T17:06:29.989993+0000","last_clean":"2024-03-12T17:06:29.989993+0000","last_became_active":"2024-03-12T17:06:25.936366+0000","last_became_peered":"2024-03-12T17:06:25.936366+0000","last_unstale":"2024-03-12T17:06:29.989993+0000","last_undegraded":"2024-03-12T17:06:29.989993+0000","last_fullsized":"2024-03-12T17:06:29.989993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:44:34.582031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.013702+0000","last_change":"2024-03-12T17:06:25.960447+0000","last_active":"2024-03-12T17:06:30.013702+0000","last_peered":"2024-03-12T17:06:30.013702+0000","last_clean":"2024-03-12T17:06:30.013702+0000","last_became_active":"2024-03-12T17:06:25.958764+0000","last_became_peered":"2024-03-12T17:06:25.958764+0000","last_unstale":"2024-03-12T17:06:30.013702+0000","last_undegraded":"2024-03-12T17:06:30.013702+0000","last_fullsized":"2024-03-12T17:06:30.013702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:16:50.369045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.987173+0000","last_change":"2024-03-12T17:06:25.928162+0000","last_active":"2024-03-12T17:06:29.987173+0000","last_peered":"2024-03-12T17:06:29.987173+0000","last_clean":"2024-03-12T17:06:29.987173+0000","last_became_active":"2024-03-12T17:06:25.927864+0000","last_became_peered":"2024-03-12T17:06:25.927864+0000","last_unstale":"2024-03-12T17:06:29.987173+0000","last_undegraded":"2024-03-12T17:06:29.987173+0000","last_fullsized":"2024-03-12T17:06:29.987173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:28:58.739417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.018239+0000","last_change":"2024-03-12T17:06:25.956401+0000","last_active":"2024-03-12T17:06:30.018239+0000","last_peered":"2024-03-12T17:06:30.018239+0000","last_clean":"2024-03-12T17:06:30.018239+0000","last_became_active":"2024-03-12T17:06:25.955835+0000","last_became_peered":"2024-03-12T17:06:25.955835+0000","last_unstale":"2024-03-12T17:06:30.018239+0000","last_undegraded":"2024-03-12T17:06:30.018239+0000","last_fullsized":"2024-03-12T17:06:30.018239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:11:15.145295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.985932+0000","last_change":"2024-03-12T17:06:25.952812+0000","last_active":"2024-03-12T17:06:29.985932+0000","last_peered":"2024-03-12T17:06:29.985932+0000","last_clean":"2024-03-12T17:06:29.985932+0000","last_became_active":"2024-03-12T17:06:25.946931+0000","last_became_peered":"2024-03-12T17:06:25.946931+0000","last_unstale":"2024-03-12T17:06:29.985932+0000","last_undegraded":"2024-03-12T17:06:29.985932+0000","last_fullsized":"2024-03-12T17:06:29.985932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:16:50.369045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012000+0000","last_change":"2024-03-12T17:06:25.950752+0000","last_active":"2024-03-12T17:06:30.012000+0000","last_peered":"2024-03-12T17:06:30.012000+0000","last_clean":"2024-03-12T17:06:30.012000+0000","last_became_active":"2024-03-12T17:06:25.950440+0000","last_became_peered":"2024-03-12T17:06:25.950440+0000","last_unstale":"2024-03-12T17:06:30.012000+0000","last_undegraded":"2024-03-12T17:06:30.012000+0000","last_fullsized":"2024-03-12T17:06:30.012000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:38:51.036295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990839+0000","last_change":"2024-03-12T17:06:25.944733+0000","last_active":"2024-03-12T17:06:29.990839+0000","last_peered":"2024-03-12T17:06:29.990839+0000","last_clean":"2024-03-12T17:06:29.990839+0000","last_became_active":"2024-03-12T17:06:25.942878+0000","last_became_peered":"2024-03-12T17:06:25.942878+0000","last_unstale":"2024-03-12T17:06:29.990839+0000","last_undegraded":"2024-03-12T17:06:29.990839+0000","last_fullsized":"2024-03-12T17:06:29.990839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:15:36.195143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.996580+0000","last_change":"2024-03-12T17:06:25.943005+0000","last_active":"2024-03-12T17:06:29.996580+0000","last_peered":"2024-03-12T17:06:29.996580+0000","last_clean":"2024-03-12T17:06:29.996580+0000","last_became_active":"2024-03-12T17:06:25.933118+0000","last_became_peered":"2024-03-12T17:06:25.933118+0000","last_unstale":"2024-03-12T17:06:29.996580+0000","last_undegraded":"2024-03-12T17:06:29.996580+0000","last_fullsized":"2024-03-12T17:06:29.996580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:44:25.750833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.022750+0000","last_change":"2024-03-12T17:06:25.947390+0000","last_active":"2024-03-12T17:06:30.022750+0000","last_peered":"2024-03-12T17:06:30.022750+0000","last_clean":"2024-03-12T17:06:30.022750+0000","last_became_active":"2024-03-12T17:06:25.947027+0000","last_became_peered":"2024-03-12T17:06:25.947027+0000","last_unstale":"2024-03-12T17:06:30.022750+0000","last_undegraded":"2024-03-12T17:06:30.022750+0000","last_fullsized":"2024-03-12T17:06:30.022750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:44:34.582031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.013617+0000","last_change":"2024-03-12T17:06:25.958459+0000","last_active":"2024-03-12T17:06:30.013617+0000","last_peered":"2024-03-12T17:06:30.013617+0000","last_clean":"2024-03-12T17:06:30.013617+0000","last_became_active":"2024-03-12T17:06:25.956687+0000","last_became_peered":"2024-03-12T17:06:25.956687+0000","last_unstale":"2024-03-12T17:06:30.013617+0000","last_undegraded":"2024-03-12T17:06:30.013617+0000","last_fullsized":"2024-03-12T17:06:30.013617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:26:54.235648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4593,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":864256,"data_stored":695401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.286}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2130000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]}]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.143}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":270336,"data_stored":104541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3129999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.518}]}]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":104577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3120000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.339}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.45}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.393}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2270000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3720000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":278528,"data_stored":108246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.228}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3779999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.331}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":104451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4159999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1950000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.456}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":274432,"data_stored":104611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.107}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.496}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":872448,"data_stored":695457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.204}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0980000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5489999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-12T17:07:01.223 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-12T17:07:01.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.485+0000 7f9797200640 1 Processor -- start 2024-03-12T17:07:01.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.485+0000 7f9797200640 1 -- start start 2024-03-12T17:07:01.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.487+0000 7f9797200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f979815cab0 0x7f97980b8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:01.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.487+0000 7f9797200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f97980d1c30 0x7f97980cf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:01.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.487+0000 7f9797200640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97980cf850 con 0x7f97980ceeb0 2024-03-12T17:07:01.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.487+0000 7f9797200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f979815ce90 con 0x7f979815cab0 2024-03-12T17:07:01.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.487+0000 7f9797200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97980cddf0 con 0x7f97980d1c30 2024-03-12T17:07:01.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.487+0000 7f9795e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f979815cab0 0x7f97980b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:01.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.487+0000 7f9795e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f979815cab0 0x7f97980b8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:32930/0 (socket says 172.21.15.52:32930) 2024-03-12T17:07:01.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.487+0000 7f9795e00640 1 -- 172.21.15.52:0/3111165649 learned_addr learned my addr 172.21.15.52:0/3111165649 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:01.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.488+0000 7f9795e00640 1 -- 172.21.15.52:0/3111165649 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f97980d1c30 msgr2=0x7f97980cf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:01.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.488+0000 7f9795e00640 1 --2- 172.21.15.52:0/3111165649 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f97980d1c30 0x7f97980cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:01.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.488+0000 7f9795e00640 1 -- 172.21.15.52:0/3111165649 >> v1:172.21.15.52:6789/0 conn(0x7f97980ceeb0 legacy=0x7f97980b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:01.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.488+0000 7f9795e00640 1 -- 172.21.15.52:0/3111165649 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97980cd570 con 0x7f979815cab0 2024-03-12T17:07:01.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.489+0000 7f9795e00640 1 --2- 172.21.15.52:0/3111165649 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f979815cab0 0x7f97980b8050 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9788009870 tx=0x7f978802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=752cd0eecde07286 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:01.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.489+0000 7f9795400640 1 -- 172.21.15.52:0/3111165649 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9788030840 con 0x7f979815cab0 2024-03-12T17:07:01.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.489+0000 7f9795400640 1 -- 172.21.15.52:0/3111165649 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97880309a0 con 0x7f979815cab0 2024-03-12T17:07:01.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.489+0000 7f9795400640 1 -- 172.21.15.52:0/3111165649 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9788030d30 con 0x7f979815cab0 2024-03-12T17:07:01.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.490+0000 7f9797200640 1 -- 172.21.15.52:0/3111165649 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f979815cab0 msgr2=0x7f97980b8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:01.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.490+0000 7f9797200640 1 --2- 172.21.15.52:0/3111165649 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f979815cab0 0x7f97980b8050 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9788009870 tx=0x7f978802b980 comp rx=0 tx=0).stop 2024-03-12T17:07:01.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.490+0000 7f9797200640 1 -- 172.21.15.52:0/3111165649 shutdown_connections 2024-03-12T17:07:01.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.490+0000 7f9797200640 1 --2- 172.21.15.52:0/3111165649 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f97980d1c30 0x7f97980cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:01.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.490+0000 7f9797200640 1 --2- 172.21.15.52:0/3111165649 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f979815cab0 0x7f97980b8050 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:01.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.490+0000 7f9797200640 1 -- 172.21.15.52:0/3111165649 >> 172.21.15.52:0/3111165649 conn(0x7f979806ae90 msgr2=0x7f979806b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:01.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.490+0000 7f9797200640 1 -- 172.21.15.52:0/3111165649 shutdown_connections 2024-03-12T17:07:01.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.490+0000 7f9797200640 1 -- 172.21.15.52:0/3111165649 wait complete. 2024-03-12T17:07:01.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.492+0000 7f9797200640 1 Processor -- start 2024-03-12T17:07:01.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.492+0000 7f9797200640 1 -- start start 2024-03-12T17:07:01.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.493+0000 7f9797200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97980ceeb0 0x7f97980ca810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:01.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.493+0000 7f9797200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97980d1c30 0x7f9798147a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:01.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.493+0000 7f9797200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f979815cab0 0x7f9798156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:01.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.493+0000 7f9797200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97980c9770 con 0x7f97980ceeb0 2024-03-12T17:07:01.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.493+0000 7f9797200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97980c9a50 con 0x7f97980d1c30 2024-03-12T17:07:01.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.493+0000 7f9797200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97980c9d30 con 0x7f979815cab0 2024-03-12T17:07:01.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.493+0000 7f9795e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97980ceeb0 0x7f97980ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:01.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.493+0000 7f9795e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97980ceeb0 0x7f97980ca810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:32932/0 (socket says 172.21.15.52:32932) 2024-03-12T17:07:01.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.493+0000 7f9795e00640 1 -- 172.21.15.52:0/753910267 learned_addr learned my addr 172.21.15.52:0/753910267 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:01.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.493+0000 7f9795e00640 1 -- 172.21.15.52:0/753910267 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f979815cab0 msgr2=0x7f9798156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:01.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.493+0000 7f9795e00640 1 --2- 172.21.15.52:0/753910267 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f979815cab0 0x7f9798156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:01.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.493+0000 7f9795e00640 1 -- 172.21.15.52:0/753910267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97980d1c30 msgr2=0x7f9798147a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:01.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.493+0000 7f9795e00640 1 --2- 172.21.15.52:0/753910267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97980d1c30 0x7f9798147a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:01.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.493+0000 7f9795e00640 1 -- 172.21.15.52:0/753910267 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97880094d0 con 0x7f97980ceeb0 2024-03-12T17:07:01.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.494+0000 7f9795e00640 1 --2- 172.21.15.52:0/753910267 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97980ceeb0 0x7f97980ca810 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9788009870 tx=0x7f9788004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:01.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.494+0000 7f978e000640 1 -- 172.21.15.52:0/753910267 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97880307f0 con 0x7f97980ceeb0 2024-03-12T17:07:01.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.494+0000 7f978e000640 1 -- 172.21.15.52:0/753910267 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9788043400 con 0x7f97980ceeb0 2024-03-12T17:07:01.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.494+0000 7f9797200640 1 -- 172.21.15.52:0/753910267 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97980ca010 con 0x7f97980ceeb0 2024-03-12T17:07:01.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.494+0000 7f978e000640 1 -- 172.21.15.52:0/753910267 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9788043790 con 0x7f97980ceeb0 2024-03-12T17:07:01.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.494+0000 7f9797200640 1 -- 172.21.15.52:0/753910267 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9798156fb0 con 0x7f97980ceeb0 2024-03-12T17:07:01.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.495+0000 7f9797200640 1 -- 172.21.15.52:0/753910267 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97980ca530 con 0x7f97980ceeb0 2024-03-12T17:07:01.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.496+0000 7f978e000640 1 -- 172.21.15.52:0/753910267 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f97880438f0 con 0x7f97980ceeb0 2024-03-12T17:07:01.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.497+0000 7f978e000640 1 --2- 172.21.15.52:0/753910267 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f977406ac30 0x7f977406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:01.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.497+0000 7f978e000640 1 -- 172.21.15.52:0/753910267 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f978803e070 con 0x7f97980ceeb0 2024-03-12T17:07:01.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.498+0000 7f978fe00640 1 --2- 172.21.15.52:0/753910267 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f977406ac30 0x7f977406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:01.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.499+0000 7f978fe00640 1 --2- 172.21.15.52:0/753910267 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f977406ac30 0x7f977406d0f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9780000f60 tx=0x7f97800079b0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:01.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.502+0000 7f978e000640 1 -- 172.21.15.52:0/753910267 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f97980ca530 con 0x7f97980ceeb0 2024-03-12T17:07:01.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.692+0000 7f9797200640 1 -- 172.21.15.52:0/753910267 --> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f97980cdb10 con 0x7f977406ac30 2024-03-12T17:07:01.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.719+0000 7f978e000640 1 -- 172.21.15.52:0/753910267 <== mgr.4109 v2:172.21.15.102:6832/36622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345027 (secure 0 0 0) 0x7f9780005080 con 0x7f977406ac30 2024-03-12T17:07:01.717 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:07:01.719 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-03-12T17:07:01.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.725+0000 7f9797200640 1 -- 172.21.15.52:0/753910267 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f977406ac30 msgr2=0x7f977406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:01.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.725+0000 7f9797200640 1 --2- 172.21.15.52:0/753910267 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f977406ac30 0x7f977406d0f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9780000f60 tx=0x7f97800079b0 comp rx=0 tx=0).stop 2024-03-12T17:07:01.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.725+0000 7f9797200640 1 -- 172.21.15.52:0/753910267 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97980ceeb0 msgr2=0x7f97980ca810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:01.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.725+0000 7f9797200640 1 --2- 172.21.15.52:0/753910267 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97980ceeb0 0x7f97980ca810 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9788009870 tx=0x7f9788004730 comp rx=0 tx=0).stop 2024-03-12T17:07:01.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.725+0000 7f9797200640 1 -- 172.21.15.52:0/753910267 shutdown_connections 2024-03-12T17:07:01.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.725+0000 7f9797200640 1 --2- 172.21.15.52:0/753910267 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f977406ac30 0x7f977406d0f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:01.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.725+0000 7f9797200640 1 --2- 172.21.15.52:0/753910267 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f979815cab0 0x7f9798156a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:01.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.725+0000 7f9797200640 1 --2- 172.21.15.52:0/753910267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97980d1c30 0x7f9798147a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:01.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.725+0000 7f9797200640 1 --2- 172.21.15.52:0/753910267 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97980ceeb0 0x7f97980ca810 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:01.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.725+0000 7f9797200640 1 -- 172.21.15.52:0/753910267 >> 172.21.15.52:0/753910267 conn(0x7f979806ae90 msgr2=0x7f97981602c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:01.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.725+0000 7f9797200640 1 -- 172.21.15.52:0/753910267 shutdown_connections 2024-03-12T17:07:01.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:01.725+0000 7f9797200640 1 -- 172.21.15.52:0/753910267 wait complete. 2024-03-12T17:07:01.737 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-03-12T17:07:01.160543+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594961,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231808,"kb_used_data":3304,"kb_used_omap":13,"kb_used_meta":228338,"kb_avail":754742912,"statfs":{"total":773094113280,"available":772856741888,"internally_reserved":0,"allocated":3383296,"data_stored":2025530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":233818200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005926"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.999570+0000","last_change":"2024-03-12T17:06:25.442740+0000","last_active":"2024-03-12T17:06:29.999570+0000","last_peered":"2024-03-12T17:06:29.999570+0000","last_clean":"2024-03-12T17:06:29.999570+0000","last_became_active":"2024-03-12T17:06:25.438608+0000","last_became_peered":"2024-03-12T17:06:25.438608+0000","last_unstale":"2024-03-12T17:06:29.999570+0000","last_undegraded":"2024-03-12T17:06:29.999570+0000","last_fullsized":"2024-03-12T17:06:29.999570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:59:41.676210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.004064+0000","last_change":"2024-03-12T17:06:25.939770+0000","last_active":"2024-03-12T17:06:30.004064+0000","last_peered":"2024-03-12T17:06:30.004064+0000","last_clean":"2024-03-12T17:06:30.004064+0000","last_became_active":"2024-03-12T17:06:25.938969+0000","last_became_peered":"2024-03-12T17:06:25.938969+0000","last_unstale":"2024-03-12T17:06:30.004064+0000","last_undegraded":"2024-03-12T17:06:30.004064+0000","last_fullsized":"2024-03-12T17:06:30.004064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:23:25.984714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.476841+0000","last_change":"2024-03-12T17:06:30.023660+0000","last_active":"2024-03-12T17:06:30.476841+0000","last_peered":"2024-03-12T17:06:30.476841+0000","last_clean":"2024-03-12T17:06:30.476841+0000","last_became_active":"2024-03-12T17:06:30.022948+0000","last_became_peered":"2024-03-12T17:06:30.022948+0000","last_unstale":"2024-03-12T17:06:30.476841+0000","last_undegraded":"2024-03-12T17:06:30.476841+0000","last_fullsized":"2024-03-12T17:06:30.476841+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:04:13.473422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.003821+0000","last_change":"2024-03-12T17:06:25.446629+0000","last_active":"2024-03-12T17:06:30.003821+0000","last_peered":"2024-03-12T17:06:30.003821+0000","last_clean":"2024-03-12T17:06:30.003821+0000","last_became_active":"2024-03-12T17:06:25.442784+0000","last_became_peered":"2024-03-12T17:06:25.442784+0000","last_unstale":"2024-03-12T17:06:30.003821+0000","last_undegraded":"2024-03-12T17:06:30.003821+0000","last_fullsized":"2024-03-12T17:06:30.003821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:59:41.676210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.999425+0000","last_change":"2024-03-12T17:06:25.953462+0000","last_active":"2024-03-12T17:06:29.999425+0000","last_peered":"2024-03-12T17:06:29.999425+0000","last_clean":"2024-03-12T17:06:29.999425+0000","last_became_active":"2024-03-12T17:06:25.953198+0000","last_became_peered":"2024-03-12T17:06:25.953198+0000","last_unstale":"2024-03-12T17:06:29.999425+0000","last_undegraded":"2024-03-12T17:06:29.999425+0000","last_fullsized":"2024-03-12T17:06:29.999425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:52:14.405796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.480606+0000","last_change":"2024-03-12T17:06:30.017287+0000","last_active":"2024-03-12T17:06:30.480606+0000","last_peered":"2024-03-12T17:06:30.480606+0000","last_clean":"2024-03-12T17:06:30.480606+0000","last_became_active":"2024-03-12T17:06:30.016834+0000","last_became_peered":"2024-03-12T17:06:30.016834+0000","last_unstale":"2024-03-12T17:06:30.480606+0000","last_undegraded":"2024-03-12T17:06:30.480606+0000","last_fullsized":"2024-03-12T17:06:30.480606+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:21:26.853333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012212+0000","last_change":"2024-03-12T17:06:25.454124+0000","last_active":"2024-03-12T17:06:30.012212+0000","last_peered":"2024-03-12T17:06:30.012212+0000","last_clean":"2024-03-12T17:06:30.012212+0000","last_became_active":"2024-03-12T17:06:25.453877+0000","last_became_peered":"2024-03-12T17:06:25.453877+0000","last_unstale":"2024-03-12T17:06:30.012212+0000","last_undegraded":"2024-03-12T17:06:30.012212+0000","last_fullsized":"2024-03-12T17:06:30.012212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:45:01.926495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.991068+0000","last_change":"2024-03-12T17:06:25.926772+0000","last_active":"2024-03-12T17:06:29.991068+0000","last_peered":"2024-03-12T17:06:29.991068+0000","last_clean":"2024-03-12T17:06:29.991068+0000","last_became_active":"2024-03-12T17:06:25.926486+0000","last_became_peered":"2024-03-12T17:06:25.926486+0000","last_unstale":"2024-03-12T17:06:29.991068+0000","last_undegraded":"2024-03-12T17:06:29.991068+0000","last_fullsized":"2024-03-12T17:06:29.991068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:28:58.739417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.482311+0000","last_change":"2024-03-12T17:06:30.028104+0000","last_active":"2024-03-12T17:06:30.482311+0000","last_peered":"2024-03-12T17:06:30.482311+0000","last_clean":"2024-03-12T17:06:30.482311+0000","last_became_active":"2024-03-12T17:06:30.027645+0000","last_became_peered":"2024-03-12T17:06:30.027645+0000","last_unstale":"2024-03-12T17:06:30.482311+0000","last_undegraded":"2024-03-12T17:06:30.482311+0000","last_fullsized":"2024-03-12T17:06:30.482311+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:13:55.311599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.011875+0000","last_change":"2024-03-12T17:06:25.455911+0000","last_active":"2024-03-12T17:06:30.011875+0000","last_peered":"2024-03-12T17:06:30.011875+0000","last_clean":"2024-03-12T17:06:30.011875+0000","last_became_active":"2024-03-12T17:06:25.455251+0000","last_became_peered":"2024-03-12T17:06:25.455251+0000","last_unstale":"2024-03-12T17:06:30.011875+0000","last_undegraded":"2024-03-12T17:06:30.011875+0000","last_fullsized":"2024-03-12T17:06:30.011875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:36:25.468644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990029+0000","last_change":"2024-03-12T17:06:25.940304+0000","last_active":"2024-03-12T17:06:29.990029+0000","last_peered":"2024-03-12T17:06:29.990029+0000","last_clean":"2024-03-12T17:06:29.990029+0000","last_became_active":"2024-03-12T17:06:25.940019+0000","last_became_peered":"2024-03-12T17:06:25.940019+0000","last_unstale":"2024-03-12T17:06:29.990029+0000","last_undegraded":"2024-03-12T17:06:29.990029+0000","last_fullsized":"2024-03-12T17:06:29.990029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:44:34.582031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.481007+0000","last_change":"2024-03-12T17:06:30.030105+0000","last_active":"2024-03-12T17:06:30.481007+0000","last_peered":"2024-03-12T17:06:30.481007+0000","last_clean":"2024-03-12T17:06:30.481007+0000","last_became_active":"2024-03-12T17:06:30.029702+0000","last_became_peered":"2024-03-12T17:06:30.029702+0000","last_unstale":"2024-03-12T17:06:30.481007+0000","last_undegraded":"2024-03-12T17:06:30.481007+0000","last_fullsized":"2024-03-12T17:06:30.481007+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:33:44.182800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.986543+0000","last_change":"2024-03-12T17:06:25.942985+0000","last_active":"2024-03-12T17:06:29.986543+0000","last_peered":"2024-03-12T17:06:29.986543+0000","last_clean":"2024-03-12T17:06:29.986543+0000","last_became_active":"2024-03-12T17:06:25.942526+0000","last_became_peered":"2024-03-12T17:06:25.942526+0000","last_unstale":"2024-03-12T17:06:29.986543+0000","last_undegraded":"2024-03-12T17:06:29.986543+0000","last_fullsized":"2024-03-12T17:06:29.986543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:44:25.750833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012322+0000","last_change":"2024-03-12T17:06:25.453557+0000","last_active":"2024-03-12T17:06:30.012322+0000","last_peered":"2024-03-12T17:06:30.012322+0000","last_clean":"2024-03-12T17:06:30.012322+0000","last_became_active":"2024-03-12T17:06:25.451663+0000","last_became_peered":"2024-03-12T17:06:25.451663+0000","last_unstale":"2024-03-12T17:06:30.012322+0000","last_undegraded":"2024-03-12T17:06:30.012322+0000","last_fullsized":"2024-03-12T17:06:30.012322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:36:47.407275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.482348+0000","last_change":"2024-03-12T17:06:30.018487+0000","last_active":"2024-03-12T17:06:30.482348+0000","last_peered":"2024-03-12T17:06:30.482348+0000","last_clean":"2024-03-12T17:06:30.482348+0000","last_became_active":"2024-03-12T17:06:30.017933+0000","last_became_peered":"2024-03-12T17:06:30.017933+0000","last_unstale":"2024-03-12T17:06:30.482348+0000","last_undegraded":"2024-03-12T17:06:30.482348+0000","last_fullsized":"2024-03-12T17:06:30.482348+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:42:49.327905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.995828+0000","last_change":"2024-03-12T17:06:25.952715+0000","last_active":"2024-03-12T17:06:29.995828+0000","last_peered":"2024-03-12T17:06:29.995828+0000","last_clean":"2024-03-12T17:06:29.995828+0000","last_became_active":"2024-03-12T17:06:25.952288+0000","last_became_peered":"2024-03-12T17:06:25.952288+0000","last_unstale":"2024-03-12T17:06:29.995828+0000","last_undegraded":"2024-03-12T17:06:29.995828+0000","last_fullsized":"2024-03-12T17:06:29.995828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:29:49.461651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.015251+0000","last_change":"2024-03-12T17:06:25.448043+0000","last_active":"2024-03-12T17:06:30.015251+0000","last_peered":"2024-03-12T17:06:30.015251+0000","last_clean":"2024-03-12T17:06:30.015251+0000","last_became_active":"2024-03-12T17:06:25.447665+0000","last_became_peered":"2024-03-12T17:06:25.447665+0000","last_unstale":"2024-03-12T17:06:30.015251+0000","last_undegraded":"2024-03-12T17:06:30.015251+0000","last_fullsized":"2024-03-12T17:06:30.015251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:28:25.809578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.481001+0000","last_change":"2024-03-12T17:06:30.020157+0000","last_active":"2024-03-12T17:06:30.481001+0000","last_peered":"2024-03-12T17:06:30.481001+0000","last_clean":"2024-03-12T17:06:30.481001+0000","last_became_active":"2024-03-12T17:06:30.019631+0000","last_became_peered":"2024-03-12T17:06:30.019631+0000","last_unstale":"2024-03-12T17:06:30.481001+0000","last_undegraded":"2024-03-12T17:06:30.481001+0000","last_fullsized":"2024-03-12T17:06:30.481001+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:46:40.007538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.987316+0000","last_change":"2024-03-12T17:06:25.925596+0000","last_active":"2024-03-12T17:06:29.987316+0000","last_peered":"2024-03-12T17:06:29.987316+0000","last_clean":"2024-03-12T17:06:29.987316+0000","last_became_active":"2024-03-12T17:06:25.925225+0000","last_became_peered":"2024-03-12T17:06:25.925225+0000","last_unstale":"2024-03-12T17:06:29.987316+0000","last_undegraded":"2024-03-12T17:06:29.987316+0000","last_fullsized":"2024-03-12T17:06:29.987316+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:44:34.582031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.014268+0000","last_change":"2024-03-12T17:06:25.450336+0000","last_active":"2024-03-12T17:06:30.014268+0000","last_peered":"2024-03-12T17:06:30.014268+0000","last_clean":"2024-03-12T17:06:30.014268+0000","last_became_active":"2024-03-12T17:06:25.450049+0000","last_became_peered":"2024-03-12T17:06:25.450049+0000","last_unstale":"2024-03-12T17:06:30.014268+0000","last_undegraded":"2024-03-12T17:06:30.014268+0000","last_fullsized":"2024-03-12T17:06:30.014268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:00:06.114876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.511116+0000","last_change":"2024-03-12T17:06:30.024652+0000","last_active":"2024-03-12T17:06:30.511116+0000","last_peered":"2024-03-12T17:06:30.511116+0000","last_clean":"2024-03-12T17:06:30.511116+0000","last_became_active":"2024-03-12T17:06:30.024156+0000","last_became_peered":"2024-03-12T17:06:30.024156+0000","last_unstale":"2024-03-12T17:06:30.511116+0000","last_undegraded":"2024-03-12T17:06:30.511116+0000","last_fullsized":"2024-03-12T17:06:30.511116+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:21:26.853333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012195+0000","last_change":"2024-03-12T17:06:25.935633+0000","last_active":"2024-03-12T17:06:30.012195+0000","last_peered":"2024-03-12T17:06:30.012195+0000","last_clean":"2024-03-12T17:06:30.012195+0000","last_became_active":"2024-03-12T17:06:25.935322+0000","last_became_peered":"2024-03-12T17:06:25.935322+0000","last_unstale":"2024-03-12T17:06:30.012195+0000","last_undegraded":"2024-03-12T17:06:30.012195+0000","last_fullsized":"2024-03-12T17:06:30.012195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:34:27.281126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.014503+0000","last_change":"2024-03-12T17:06:25.456881+0000","last_active":"2024-03-12T17:06:30.014503+0000","last_peered":"2024-03-12T17:06:30.014503+0000","last_clean":"2024-03-12T17:06:30.014503+0000","last_became_active":"2024-03-12T17:06:25.456425+0000","last_became_peered":"2024-03-12T17:06:25.456425+0000","last_unstale":"2024-03-12T17:06:30.014503+0000","last_undegraded":"2024-03-12T17:06:30.014503+0000","last_fullsized":"2024-03-12T17:06:30.014503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:33:42.671579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.482388+0000","last_change":"2024-03-12T17:06:30.021976+0000","last_active":"2024-03-12T17:06:30.482388+0000","last_peered":"2024-03-12T17:06:30.482388+0000","last_clean":"2024-03-12T17:06:30.482388+0000","last_became_active":"2024-03-12T17:06:30.021512+0000","last_became_peered":"2024-03-12T17:06:30.021512+0000","last_unstale":"2024-03-12T17:06:30.482388+0000","last_undegraded":"2024-03-12T17:06:30.482388+0000","last_fullsized":"2024-03-12T17:06:30.482388+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:31:47.699683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.999227+0000","last_change":"2024-03-12T17:06:25.937491+0000","last_active":"2024-03-12T17:06:29.999227+0000","last_peered":"2024-03-12T17:06:29.999227+0000","last_clean":"2024-03-12T17:06:29.999227+0000","last_became_active":"2024-03-12T17:06:25.937173+0000","last_became_peered":"2024-03-12T17:06:25.937173+0000","last_unstale":"2024-03-12T17:06:29.999227+0000","last_undegraded":"2024-03-12T17:06:29.999227+0000","last_fullsized":"2024-03-12T17:06:29.999227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:29:49.461651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990464+0000","last_change":"2024-03-12T17:06:25.450121+0000","last_active":"2024-03-12T17:06:29.990464+0000","last_peered":"2024-03-12T17:06:29.990464+0000","last_clean":"2024-03-12T17:06:29.990464+0000","last_became_active":"2024-03-12T17:06:25.448549+0000","last_became_peered":"2024-03-12T17:06:25.448549+0000","last_unstale":"2024-03-12T17:06:29.990464+0000","last_undegraded":"2024-03-12T17:06:29.990464+0000","last_fullsized":"2024-03-12T17:06:29.990464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:36:47.407275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.481700+0000","last_change":"2024-03-12T17:06:30.020438+0000","last_active":"2024-03-12T17:06:30.481700+0000","last_peered":"2024-03-12T17:06:30.481700+0000","last_clean":"2024-03-12T17:06:30.481700+0000","last_became_active":"2024-03-12T17:06:30.020055+0000","last_became_peered":"2024-03-12T17:06:30.020055+0000","last_unstale":"2024-03-12T17:06:30.481700+0000","last_undegraded":"2024-03-12T17:06:30.481700+0000","last_fullsized":"2024-03-12T17:06:30.481700+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:17:09.055890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.004123+0000","last_change":"2024-03-12T17:06:25.947199+0000","last_active":"2024-03-12T17:06:30.004123+0000","last_peered":"2024-03-12T17:06:30.004123+0000","last_clean":"2024-03-12T17:06:30.004123+0000","last_became_active":"2024-03-12T17:06:25.941750+0000","last_became_peered":"2024-03-12T17:06:25.941750+0000","last_unstale":"2024-03-12T17:06:30.004123+0000","last_undegraded":"2024-03-12T17:06:30.004123+0000","last_fullsized":"2024-03-12T17:06:30.004123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:04:04.198678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.021973+0000","last_change":"2024-03-12T17:06:25.452843+0000","last_active":"2024-03-12T17:06:30.021973+0000","last_peered":"2024-03-12T17:06:30.021973+0000","last_clean":"2024-03-12T17:06:30.021973+0000","last_became_active":"2024-03-12T17:06:25.452410+0000","last_became_peered":"2024-03-12T17:06:25.452410+0000","last_unstale":"2024-03-12T17:06:30.021973+0000","last_undegraded":"2024-03-12T17:06:30.021973+0000","last_fullsized":"2024-03-12T17:06:30.021973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:36:25.468644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.476561+0000","last_change":"2024-03-12T17:06:30.028820+0000","last_active":"2024-03-12T17:06:30.476561+0000","last_peered":"2024-03-12T17:06:30.476561+0000","last_clean":"2024-03-12T17:06:30.476561+0000","last_became_active":"2024-03-12T17:06:30.028425+0000","last_became_peered":"2024-03-12T17:06:30.028425+0000","last_unstale":"2024-03-12T17:06:30.476561+0000","last_undegraded":"2024-03-12T17:06:30.476561+0000","last_fullsized":"2024-03-12T17:06:30.476561+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:46:40.007538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.019530+0000","last_change":"2024-03-12T17:06:25.929758+0000","last_active":"2024-03-12T17:06:30.019530+0000","last_peered":"2024-03-12T17:06:30.019530+0000","last_clean":"2024-03-12T17:06:30.019530+0000","last_became_active":"2024-03-12T17:06:25.929481+0000","last_became_peered":"2024-03-12T17:06:25.929481+0000","last_unstale":"2024-03-12T17:06:30.019530+0000","last_undegraded":"2024-03-12T17:06:30.019530+0000","last_fullsized":"2024-03-12T17:06:30.019530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:44:25.750833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012765+0000","last_change":"2024-03-12T17:06:25.456024+0000","last_active":"2024-03-12T17:06:30.012765+0000","last_peered":"2024-03-12T17:06:30.012765+0000","last_clean":"2024-03-12T17:06:30.012765+0000","last_became_active":"2024-03-12T17:06:25.455718+0000","last_became_peered":"2024-03-12T17:06:25.455718+0000","last_unstale":"2024-03-12T17:06:30.012765+0000","last_undegraded":"2024-03-12T17:06:30.012765+0000","last_fullsized":"2024-03-12T17:06:30.012765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:00:06.114876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.508448+0000","last_change":"2024-03-12T17:06:30.020641+0000","last_active":"2024-03-12T17:06:30.508448+0000","last_peered":"2024-03-12T17:06:30.508448+0000","last_clean":"2024-03-12T17:06:30.508448+0000","last_became_active":"2024-03-12T17:06:30.020373+0000","last_became_peered":"2024-03-12T17:06:30.020373+0000","last_unstale":"2024-03-12T17:06:30.508448+0000","last_undegraded":"2024-03-12T17:06:30.508448+0000","last_fullsized":"2024-03-12T17:06:30.508448+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:03:49.781309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990659+0000","last_change":"2024-03-12T17:06:25.943799+0000","last_active":"2024-03-12T17:06:29.990659+0000","last_peered":"2024-03-12T17:06:29.990659+0000","last_clean":"2024-03-12T17:06:29.990659+0000","last_became_active":"2024-03-12T17:06:25.943543+0000","last_became_peered":"2024-03-12T17:06:25.943543+0000","last_unstale":"2024-03-12T17:06:29.990659+0000","last_undegraded":"2024-03-12T17:06:29.990659+0000","last_fullsized":"2024-03-12T17:06:29.990659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:38:51.036295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.019581+0000","last_change":"2024-03-12T17:06:25.456899+0000","last_active":"2024-03-12T17:06:30.019581+0000","last_peered":"2024-03-12T17:06:30.019581+0000","last_clean":"2024-03-12T17:06:30.019581+0000","last_became_active":"2024-03-12T17:06:25.456513+0000","last_became_peered":"2024-03-12T17:06:25.456513+0000","last_unstale":"2024-03-12T17:06:30.019581+0000","last_undegraded":"2024-03-12T17:06:30.019581+0000","last_fullsized":"2024-03-12T17:06:30.019581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:58:32.509940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.508585+0000","last_change":"2024-03-12T17:06:30.018815+0000","last_active":"2024-03-12T17:06:30.508585+0000","last_peered":"2024-03-12T17:06:30.508585+0000","last_clean":"2024-03-12T17:06:30.508585+0000","last_became_active":"2024-03-12T17:06:30.014225+0000","last_became_peered":"2024-03-12T17:06:30.014225+0000","last_unstale":"2024-03-12T17:06:30.508585+0000","last_undegraded":"2024-03-12T17:06:30.508585+0000","last_fullsized":"2024-03-12T17:06:30.508585+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:49:38.782632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012440+0000","last_change":"2024-03-12T17:06:25.954609+0000","last_active":"2024-03-12T17:06:30.012440+0000","last_peered":"2024-03-12T17:06:30.012440+0000","last_clean":"2024-03-12T17:06:30.012440+0000","last_became_active":"2024-03-12T17:06:25.954337+0000","last_became_peered":"2024-03-12T17:06:25.954337+0000","last_unstale":"2024-03-12T17:06:30.012440+0000","last_undegraded":"2024-03-12T17:06:30.012440+0000","last_fullsized":"2024-03-12T17:06:30.012440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:33:43.674985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.992408+0000","last_change":"2024-03-12T17:06:25.442368+0000","last_active":"2024-03-12T17:06:29.992408+0000","last_peered":"2024-03-12T17:06:29.992408+0000","last_clean":"2024-03-12T17:06:29.992408+0000","last_became_active":"2024-03-12T17:06:25.442084+0000","last_became_peered":"2024-03-12T17:06:25.442084+0000","last_unstale":"2024-03-12T17:06:29.992408+0000","last_undegraded":"2024-03-12T17:06:29.992408+0000","last_fullsized":"2024-03-12T17:06:29.992408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:58:32.509940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.509666+0000","last_change":"2024-03-12T17:06:30.031785+0000","last_active":"2024-03-12T17:06:30.509666+0000","last_peered":"2024-03-12T17:06:30.509666+0000","last_clean":"2024-03-12T17:06:30.509666+0000","last_became_active":"2024-03-12T17:06:30.031328+0000","last_became_peered":"2024-03-12T17:06:30.031328+0000","last_unstale":"2024-03-12T17:06:30.509666+0000","last_undegraded":"2024-03-12T17:06:30.509666+0000","last_fullsized":"2024-03-12T17:06:30.509666+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:35:06.129757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012185+0000","last_change":"2024-03-12T17:06:25.931255+0000","last_active":"2024-03-12T17:06:30.012185+0000","last_peered":"2024-03-12T17:06:30.012185+0000","last_clean":"2024-03-12T17:06:30.012185+0000","last_became_active":"2024-03-12T17:06:25.929387+0000","last_became_peered":"2024-03-12T17:06:25.929387+0000","last_unstale":"2024-03-12T17:06:30.012185+0000","last_undegraded":"2024-03-12T17:06:30.012185+0000","last_fullsized":"2024-03-12T17:06:30.012185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:04:04.198678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.980151+0000","last_change":"2024-03-12T17:06:25.443901+0000","last_active":"2024-03-12T17:06:29.980151+0000","last_peered":"2024-03-12T17:06:29.980151+0000","last_clean":"2024-03-12T17:06:29.980151+0000","last_became_active":"2024-03-12T17:06:25.443602+0000","last_became_peered":"2024-03-12T17:06:25.443602+0000","last_unstale":"2024-03-12T17:06:29.980151+0000","last_undegraded":"2024-03-12T17:06:29.980151+0000","last_fullsized":"2024-03-12T17:06:29.980151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:00:32.347614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.511172+0000","last_change":"2024-03-12T17:06:30.025943+0000","last_active":"2024-03-12T17:06:30.511172+0000","last_peered":"2024-03-12T17:06:30.511172+0000","last_clean":"2024-03-12T17:06:30.511172+0000","last_became_active":"2024-03-12T17:06:30.025481+0000","last_became_peered":"2024-03-12T17:06:30.025481+0000","last_unstale":"2024-03-12T17:06:30.511172+0000","last_undegraded":"2024-03-12T17:06:30.511172+0000","last_fullsized":"2024-03-12T17:06:30.511172+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:17:09.055890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.996440+0000","last_change":"2024-03-12T17:06:25.940996+0000","last_active":"2024-03-12T17:06:29.996440+0000","last_peered":"2024-03-12T17:06:29.996440+0000","last_clean":"2024-03-12T17:06:29.996440+0000","last_became_active":"2024-03-12T17:06:25.940740+0000","last_became_peered":"2024-03-12T17:06:25.940740+0000","last_unstale":"2024-03-12T17:06:29.996440+0000","last_undegraded":"2024-03-12T17:06:29.996440+0000","last_fullsized":"2024-03-12T17:06:29.996440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:36:26.472050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.014590+0000","last_change":"2024-03-12T17:06:25.450168+0000","last_active":"2024-03-12T17:06:30.014590+0000","last_peered":"2024-03-12T17:06:30.014590+0000","last_clean":"2024-03-12T17:06:30.014590+0000","last_became_active":"2024-03-12T17:06:25.449620+0000","last_became_peered":"2024-03-12T17:06:25.449620+0000","last_unstale":"2024-03-12T17:06:30.014590+0000","last_undegraded":"2024-03-12T17:06:30.014590+0000","last_fullsized":"2024-03-12T17:06:30.014590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:58:32.509940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.509124+0000","last_change":"2024-03-12T17:06:30.020545+0000","last_active":"2024-03-12T17:06:30.509124+0000","last_peered":"2024-03-12T17:06:30.509124+0000","last_clean":"2024-03-12T17:06:30.509124+0000","last_became_active":"2024-03-12T17:06:30.020246+0000","last_became_peered":"2024-03-12T17:06:30.020246+0000","last_unstale":"2024-03-12T17:06:30.509124+0000","last_undegraded":"2024-03-12T17:06:30.509124+0000","last_fullsized":"2024-03-12T17:06:30.509124+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:08:39.661179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990996+0000","last_change":"2024-03-12T17:06:25.931831+0000","last_active":"2024-03-12T17:06:29.990996+0000","last_peered":"2024-03-12T17:06:29.990996+0000","last_clean":"2024-03-12T17:06:29.990996+0000","last_became_active":"2024-03-12T17:06:25.931525+0000","last_became_peered":"2024-03-12T17:06:25.931525+0000","last_unstale":"2024-03-12T17:06:29.990996+0000","last_undegraded":"2024-03-12T17:06:29.990996+0000","last_fullsized":"2024-03-12T17:06:29.990996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:06:19.464242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.989265+0000","last_change":"2024-03-12T17:06:25.435909+0000","last_active":"2024-03-12T17:06:29.989265+0000","last_peered":"2024-03-12T17:06:29.989265+0000","last_clean":"2024-03-12T17:06:29.989265+0000","last_became_active":"2024-03-12T17:06:25.434838+0000","last_became_peered":"2024-03-12T17:06:25.434838+0000","last_unstale":"2024-03-12T17:06:29.989265+0000","last_undegraded":"2024-03-12T17:06:29.989265+0000","last_fullsized":"2024-03-12T17:06:29.989265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:36:47.407275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.014690+0000","last_change":"2024-03-12T17:06:21.714813+0000","last_active":"2024-03-12T17:06:30.014690+0000","last_peered":"2024-03-12T17:06:30.014690+0000","last_clean":"2024-03-12T17:06:30.014690+0000","last_became_active":"2024-03-12T17:06:21.712488+0000","last_became_peered":"2024-03-12T17:06:21.712488+0000","last_unstale":"2024-03-12T17:06:30.014690+0000","last_undegraded":"2024-03-12T17:06:30.014690+0000","last_fullsized":"2024-03-12T17:06:30.014690+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:19.815688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:19.815688+0000","last_clean_scrub_stamp":"2024-03-12T17:06:19.815688+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:38:54.300793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.510890+0000","last_change":"2024-03-12T17:06:30.017415+0000","last_active":"2024-03-12T17:06:30.510890+0000","last_peered":"2024-03-12T17:06:30.510890+0000","last_clean":"2024-03-12T17:06:30.510890+0000","last_became_active":"2024-03-12T17:06:30.015976+0000","last_became_peered":"2024-03-12T17:06:30.015976+0000","last_unstale":"2024-03-12T17:06:30.510890+0000","last_undegraded":"2024-03-12T17:06:30.510890+0000","last_fullsized":"2024-03-12T17:06:30.510890+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:33:44.182800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012124+0000","last_change":"2024-03-12T17:06:25.959282+0000","last_active":"2024-03-12T17:06:30.012124+0000","last_peered":"2024-03-12T17:06:30.012124+0000","last_clean":"2024-03-12T17:06:30.012124+0000","last_became_active":"2024-03-12T17:06:25.959028+0000","last_became_peered":"2024-03-12T17:06:25.959028+0000","last_unstale":"2024-03-12T17:06:30.012124+0000","last_undegraded":"2024-03-12T17:06:30.012124+0000","last_fullsized":"2024-03-12T17:06:30.012124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:38:01.478195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.980482+0000","last_change":"2024-03-12T17:06:25.442608+0000","last_active":"2024-03-12T17:06:29.980482+0000","last_peered":"2024-03-12T17:06:29.980482+0000","last_clean":"2024-03-12T17:06:29.980482+0000","last_became_active":"2024-03-12T17:06:25.441652+0000","last_became_peered":"2024-03-12T17:06:25.441652+0000","last_unstale":"2024-03-12T17:06:29.980482+0000","last_undegraded":"2024-03-12T17:06:29.980482+0000","last_fullsized":"2024-03-12T17:06:29.980482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:34:26.277720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.510468+0000","last_change":"2024-03-12T17:06:30.020225+0000","last_active":"2024-03-12T17:06:30.510468+0000","last_peered":"2024-03-12T17:06:30.510468+0000","last_clean":"2024-03-12T17:06:30.510468+0000","last_became_active":"2024-03-12T17:06:30.017277+0000","last_became_peered":"2024-03-12T17:06:30.017277+0000","last_unstale":"2024-03-12T17:06:30.510468+0000","last_undegraded":"2024-03-12T17:06:30.510468+0000","last_fullsized":"2024-03-12T17:06:30.510468+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:35:06.129757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.991439+0000","last_change":"2024-03-12T17:06:25.942345+0000","last_active":"2024-03-12T17:06:29.991439+0000","last_peered":"2024-03-12T17:06:29.991439+0000","last_clean":"2024-03-12T17:06:29.991439+0000","last_became_active":"2024-03-12T17:06:25.941771+0000","last_became_peered":"2024-03-12T17:06:25.941771+0000","last_unstale":"2024-03-12T17:06:29.991439+0000","last_undegraded":"2024-03-12T17:06:29.991439+0000","last_fullsized":"2024-03-12T17:06:29.991439+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:32:08.311478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.022019+0000","last_change":"2024-03-12T17:06:25.448435+0000","last_active":"2024-03-12T17:06:30.022019+0000","last_peered":"2024-03-12T17:06:30.022019+0000","last_clean":"2024-03-12T17:06:30.022019+0000","last_became_active":"2024-03-12T17:06:25.448141+0000","last_became_peered":"2024-03-12T17:06:25.448141+0000","last_unstale":"2024-03-12T17:06:30.022019+0000","last_undegraded":"2024-03-12T17:06:30.022019+0000","last_fullsized":"2024-03-12T17:06:30.022019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:36:47.407275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.508634+0000","last_change":"2024-03-12T17:06:30.017805+0000","last_active":"2024-03-12T17:06:30.508634+0000","last_peered":"2024-03-12T17:06:30.508634+0000","last_clean":"2024-03-12T17:06:30.508634+0000","last_became_active":"2024-03-12T17:06:30.017383+0000","last_became_peered":"2024-03-12T17:06:30.017383+0000","last_unstale":"2024-03-12T17:06:30.508634+0000","last_undegraded":"2024-03-12T17:06:30.508634+0000","last_fullsized":"2024-03-12T17:06:30.508634+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:17:09.055890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.013754+0000","last_change":"2024-03-12T17:06:25.958864+0000","last_active":"2024-03-12T17:06:30.013754+0000","last_peered":"2024-03-12T17:06:30.013754+0000","last_clean":"2024-03-12T17:06:30.013754+0000","last_became_active":"2024-03-12T17:06:25.958588+0000","last_became_peered":"2024-03-12T17:06:25.958588+0000","last_unstale":"2024-03-12T17:06:30.013754+0000","last_undegraded":"2024-03-12T17:06:30.013754+0000","last_fullsized":"2024-03-12T17:06:30.013754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:21:02.182215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.980396+0000","last_change":"2024-03-12T17:06:25.453348+0000","last_active":"2024-03-12T17:06:29.980396+0000","last_peered":"2024-03-12T17:06:29.980396+0000","last_clean":"2024-03-12T17:06:29.980396+0000","last_became_active":"2024-03-12T17:06:25.449918+0000","last_became_peered":"2024-03-12T17:06:25.449918+0000","last_unstale":"2024-03-12T17:06:29.980396+0000","last_undegraded":"2024-03-12T17:06:29.980396+0000","last_fullsized":"2024-03-12T17:06:29.980396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:49:52.284732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.479802+0000","last_change":"2024-03-12T17:06:30.023418+0000","last_active":"2024-03-12T17:06:30.479802+0000","last_peered":"2024-03-12T17:06:30.479802+0000","last_clean":"2024-03-12T17:06:30.479802+0000","last_became_active":"2024-03-12T17:06:30.022919+0000","last_became_peered":"2024-03-12T17:06:30.022919+0000","last_unstale":"2024-03-12T17:06:30.479802+0000","last_undegraded":"2024-03-12T17:06:30.479802+0000","last_fullsized":"2024-03-12T17:06:30.479802+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:13:55.311599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.015933+0000","last_change":"2024-03-12T17:06:25.961193+0000","last_active":"2024-03-12T17:06:30.015933+0000","last_peered":"2024-03-12T17:06:30.015933+0000","last_clean":"2024-03-12T17:06:30.015933+0000","last_became_active":"2024-03-12T17:06:25.960906+0000","last_became_peered":"2024-03-12T17:06:25.960906+0000","last_unstale":"2024-03-12T17:06:30.015933+0000","last_undegraded":"2024-03-12T17:06:30.015933+0000","last_fullsized":"2024-03-12T17:06:30.015933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:17:09.067136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.013327+0000","last_change":"2024-03-12T17:06:25.442363+0000","last_active":"2024-03-12T17:06:30.013327+0000","last_peered":"2024-03-12T17:06:30.013327+0000","last_clean":"2024-03-12T17:06:30.013327+0000","last_became_active":"2024-03-12T17:06:25.442053+0000","last_became_peered":"2024-03-12T17:06:25.442053+0000","last_unstale":"2024-03-12T17:06:30.013327+0000","last_undegraded":"2024-03-12T17:06:30.013327+0000","last_fullsized":"2024-03-12T17:06:30.013327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:06:05.702956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.481199+0000","last_change":"2024-03-12T17:06:30.020741+0000","last_active":"2024-03-12T17:06:30.481199+0000","last_peered":"2024-03-12T17:06:30.481199+0000","last_clean":"2024-03-12T17:06:30.481199+0000","last_became_active":"2024-03-12T17:06:30.018043+0000","last_became_peered":"2024-03-12T17:06:30.018043+0000","last_unstale":"2024-03-12T17:06:30.481199+0000","last_undegraded":"2024-03-12T17:06:30.481199+0000","last_fullsized":"2024-03-12T17:06:30.481199+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:11:34.105824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.016104+0000","last_change":"2024-03-12T17:06:25.958260+0000","last_active":"2024-03-12T17:06:30.016104+0000","last_peered":"2024-03-12T17:06:30.016104+0000","last_clean":"2024-03-12T17:06:30.016104+0000","last_became_active":"2024-03-12T17:06:25.955966+0000","last_became_peered":"2024-03-12T17:06:25.955966+0000","last_unstale":"2024-03-12T17:06:30.016104+0000","last_undegraded":"2024-03-12T17:06:30.016104+0000","last_fullsized":"2024-03-12T17:06:30.016104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:33:16.553805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.979851+0000","last_change":"2024-03-12T17:06:25.452222+0000","last_active":"2024-03-12T17:06:29.979851+0000","last_peered":"2024-03-12T17:06:29.979851+0000","last_clean":"2024-03-12T17:06:29.979851+0000","last_became_active":"2024-03-12T17:06:25.451696+0000","last_became_peered":"2024-03-12T17:06:25.451696+0000","last_unstale":"2024-03-12T17:06:29.979851+0000","last_undegraded":"2024-03-12T17:06:29.979851+0000","last_fullsized":"2024-03-12T17:06:29.979851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:28:25.809578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.508879+0000","last_change":"2024-03-12T17:06:30.009551+0000","last_active":"2024-03-12T17:06:30.508879+0000","last_peered":"2024-03-12T17:06:30.508879+0000","last_clean":"2024-03-12T17:06:30.508879+0000","last_became_active":"2024-03-12T17:06:30.009269+0000","last_became_peered":"2024-03-12T17:06:30.009269+0000","last_unstale":"2024-03-12T17:06:30.508879+0000","last_undegraded":"2024-03-12T17:06:30.508879+0000","last_fullsized":"2024-03-12T17:06:30.508879+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:42:49.327905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.984652+0000","last_change":"2024-03-12T17:06:25.949275+0000","last_active":"2024-03-12T17:06:29.984652+0000","last_peered":"2024-03-12T17:06:29.984652+0000","last_clean":"2024-03-12T17:06:29.984652+0000","last_became_active":"2024-03-12T17:06:25.947884+0000","last_became_peered":"2024-03-12T17:06:25.947884+0000","last_unstale":"2024-03-12T17:06:29.984652+0000","last_undegraded":"2024-03-12T17:06:29.984652+0000","last_fullsized":"2024-03-12T17:06:29.984652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:28:59.465721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.984927+0000","last_change":"2024-03-12T17:06:25.450408+0000","last_active":"2024-03-12T17:06:29.984927+0000","last_peered":"2024-03-12T17:06:29.984927+0000","last_clean":"2024-03-12T17:06:29.984927+0000","last_became_active":"2024-03-12T17:06:25.448867+0000","last_became_peered":"2024-03-12T17:06:25.448867+0000","last_unstale":"2024-03-12T17:06:29.984927+0000","last_undegraded":"2024-03-12T17:06:29.984927+0000","last_fullsized":"2024-03-12T17:06:29.984927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:49:52.284732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.477643+0000","last_change":"2024-03-12T17:06:30.022424+0000","last_active":"2024-03-12T17:06:30.477643+0000","last_peered":"2024-03-12T17:06:30.477643+0000","last_clean":"2024-03-12T17:06:30.477643+0000","last_became_active":"2024-03-12T17:06:30.021994+0000","last_became_peered":"2024-03-12T17:06:30.021994+0000","last_unstale":"2024-03-12T17:06:30.477643+0000","last_undegraded":"2024-03-12T17:06:30.477643+0000","last_fullsized":"2024-03-12T17:06:30.477643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:32:37.399700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012128+0000","last_change":"2024-03-12T17:06:25.958282+0000","last_active":"2024-03-12T17:06:30.012128+0000","last_peered":"2024-03-12T17:06:30.012128+0000","last_clean":"2024-03-12T17:06:30.012128+0000","last_became_active":"2024-03-12T17:06:25.955357+0000","last_became_peered":"2024-03-12T17:06:25.955357+0000","last_unstale":"2024-03-12T17:06:30.012128+0000","last_undegraded":"2024-03-12T17:06:30.012128+0000","last_fullsized":"2024-03-12T17:06:30.012128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:15:36.195143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.992283+0000","last_change":"2024-03-12T17:06:25.448887+0000","last_active":"2024-03-12T17:06:29.992283+0000","last_peered":"2024-03-12T17:06:29.992283+0000","last_clean":"2024-03-12T17:06:29.992283+0000","last_became_active":"2024-03-12T17:06:25.448299+0000","last_became_peered":"2024-03-12T17:06:25.448299+0000","last_unstale":"2024-03-12T17:06:29.992283+0000","last_undegraded":"2024-03-12T17:06:29.992283+0000","last_fullsized":"2024-03-12T17:06:29.992283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:00:06.114876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.477924+0000","last_change":"2024-03-12T17:06:30.022221+0000","last_active":"2024-03-12T17:06:30.477924+0000","last_peered":"2024-03-12T17:06:30.477924+0000","last_clean":"2024-03-12T17:06:30.477924+0000","last_became_active":"2024-03-12T17:06:30.021765+0000","last_became_peered":"2024-03-12T17:06:30.021765+0000","last_unstale":"2024-03-12T17:06:30.477924+0000","last_undegraded":"2024-03-12T17:06:30.477924+0000","last_fullsized":"2024-03-12T17:06:30.477924+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:28:34.686064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.996652+0000","last_change":"2024-03-12T17:06:25.944466+0000","last_active":"2024-03-12T17:06:29.996652+0000","last_peered":"2024-03-12T17:06:29.996652+0000","last_clean":"2024-03-12T17:06:29.996652+0000","last_became_active":"2024-03-12T17:06:25.944075+0000","last_became_peered":"2024-03-12T17:06:25.944075+0000","last_unstale":"2024-03-12T17:06:29.996652+0000","last_undegraded":"2024-03-12T17:06:29.996652+0000","last_fullsized":"2024-03-12T17:06:29.996652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T20:48:48.332409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.996480+0000","last_change":"2024-03-12T17:06:25.444436+0000","last_active":"2024-03-12T17:06:29.996480+0000","last_peered":"2024-03-12T17:06:29.996480+0000","last_clean":"2024-03-12T17:06:29.996480+0000","last_became_active":"2024-03-12T17:06:25.443700+0000","last_became_peered":"2024-03-12T17:06:25.443700+0000","last_unstale":"2024-03-12T17:06:29.996480+0000","last_undegraded":"2024-03-12T17:06:29.996480+0000","last_fullsized":"2024-03-12T17:06:29.996480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:55:17.158520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.483163+0000","last_change":"2024-03-12T17:06:30.022439+0000","last_active":"2024-03-12T17:06:30.483163+0000","last_peered":"2024-03-12T17:06:30.483163+0000","last_clean":"2024-03-12T17:06:30.483163+0000","last_became_active":"2024-03-12T17:06:30.017040+0000","last_became_peered":"2024-03-12T17:06:30.017040+0000","last_unstale":"2024-03-12T17:06:30.483163+0000","last_undegraded":"2024-03-12T17:06:30.483163+0000","last_fullsized":"2024-03-12T17:06:30.483163+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:08:39.661179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012020+0000","last_change":"2024-03-12T17:06:25.958251+0000","last_active":"2024-03-12T17:06:30.012020+0000","last_peered":"2024-03-12T17:06:30.012020+0000","last_clean":"2024-03-12T17:06:30.012020+0000","last_became_active":"2024-03-12T17:06:25.956505+0000","last_became_peered":"2024-03-12T17:06:25.956505+0000","last_unstale":"2024-03-12T17:06:30.012020+0000","last_undegraded":"2024-03-12T17:06:30.012020+0000","last_fullsized":"2024-03-12T17:06:30.012020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:32:08.311478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:44.758048+0000","last_change":"2024-03-12T17:06:25.448251+0000","last_active":"2024-03-12T17:06:44.758048+0000","last_peered":"2024-03-12T17:06:44.758048+0000","last_clean":"2024-03-12T17:06:44.758048+0000","last_became_active":"2024-03-12T17:06:25.447768+0000","last_became_peered":"2024-03-12T17:06:25.447768+0000","last_unstale":"2024-03-12T17:06:44.758048+0000","last_undegraded":"2024-03-12T17:06:44.758048+0000","last_fullsized":"2024-03-12T17:06:44.758048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:34:26.277720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3795,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.508684+0000","last_change":"2024-03-12T17:06:30.019328+0000","last_active":"2024-03-12T17:06:30.508684+0000","last_peered":"2024-03-12T17:06:30.508684+0000","last_clean":"2024-03-12T17:06:30.508684+0000","last_became_active":"2024-03-12T17:06:30.014815+0000","last_became_peered":"2024-03-12T17:06:30.014815+0000","last_unstale":"2024-03-12T17:06:30.508684+0000","last_undegraded":"2024-03-12T17:06:30.508684+0000","last_fullsized":"2024-03-12T17:06:30.508684+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:08:39.661179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012873+0000","last_change":"2024-03-12T17:06:25.945208+0000","last_active":"2024-03-12T17:06:30.012873+0000","last_peered":"2024-03-12T17:06:30.012873+0000","last_clean":"2024-03-12T17:06:30.012873+0000","last_became_active":"2024-03-12T17:06:25.944043+0000","last_became_peered":"2024-03-12T17:06:25.944043+0000","last_unstale":"2024-03-12T17:06:30.012873+0000","last_undegraded":"2024-03-12T17:06:30.012873+0000","last_fullsized":"2024-03-12T17:06:30.012873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:36:48.410681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.021900+0000","last_change":"2024-03-12T17:06:25.455080+0000","last_active":"2024-03-12T17:06:30.021900+0000","last_peered":"2024-03-12T17:06:30.021900+0000","last_clean":"2024-03-12T17:06:30.021900+0000","last_became_active":"2024-03-12T17:06:25.454788+0000","last_became_peered":"2024-03-12T17:06:25.454788+0000","last_unstale":"2024-03-12T17:06:30.021900+0000","last_undegraded":"2024-03-12T17:06:30.021900+0000","last_fullsized":"2024-03-12T17:06:30.021900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:33:42.671579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.482470+0000","last_change":"2024-03-12T17:06:30.018328+0000","last_active":"2024-03-12T17:06:30.482470+0000","last_peered":"2024-03-12T17:06:30.482470+0000","last_clean":"2024-03-12T17:06:30.482470+0000","last_became_active":"2024-03-12T17:06:30.017852+0000","last_became_peered":"2024-03-12T17:06:30.017852+0000","last_unstale":"2024-03-12T17:06:30.482470+0000","last_undegraded":"2024-03-12T17:06:30.482470+0000","last_fullsized":"2024-03-12T17:06:30.482470+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T02:49:07.614077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.999490+0000","last_change":"2024-03-12T17:06:25.938178+0000","last_active":"2024-03-12T17:06:29.999490+0000","last_peered":"2024-03-12T17:06:29.999490+0000","last_clean":"2024-03-12T17:06:29.999490+0000","last_became_active":"2024-03-12T17:06:25.937926+0000","last_became_peered":"2024-03-12T17:06:25.937926+0000","last_unstale":"2024-03-12T17:06:29.999490+0000","last_undegraded":"2024-03-12T17:06:29.999490+0000","last_fullsized":"2024-03-12T17:06:29.999490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:57:56.009742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990778+0000","last_change":"2024-03-12T17:06:25.451993+0000","last_active":"2024-03-12T17:06:29.990778+0000","last_peered":"2024-03-12T17:06:29.990778+0000","last_clean":"2024-03-12T17:06:29.990778+0000","last_became_active":"2024-03-12T17:06:25.451794+0000","last_became_peered":"2024-03-12T17:06:25.451794+0000","last_unstale":"2024-03-12T17:06:29.990778+0000","last_undegraded":"2024-03-12T17:06:29.990778+0000","last_fullsized":"2024-03-12T17:06:29.990778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:33:42.671579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.476846+0000","last_change":"2024-03-12T17:06:30.016232+0000","last_active":"2024-03-12T17:06:30.476846+0000","last_peered":"2024-03-12T17:06:30.476846+0000","last_clean":"2024-03-12T17:06:30.476846+0000","last_became_active":"2024-03-12T17:06:30.015825+0000","last_became_peered":"2024-03-12T17:06:30.015825+0000","last_unstale":"2024-03-12T17:06:30.476846+0000","last_undegraded":"2024-03-12T17:06:30.476846+0000","last_fullsized":"2024-03-12T17:06:30.476846+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:28:28.745853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990537+0000","last_change":"2024-03-12T17:06:25.938249+0000","last_active":"2024-03-12T17:06:29.990537+0000","last_peered":"2024-03-12T17:06:29.990537+0000","last_clean":"2024-03-12T17:06:29.990537+0000","last_became_active":"2024-03-12T17:06:25.937744+0000","last_became_peered":"2024-03-12T17:06:25.937744+0000","last_unstale":"2024-03-12T17:06:29.990537+0000","last_undegraded":"2024-03-12T17:06:29.990537+0000","last_fullsized":"2024-03-12T17:06:29.990537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:06:19.464242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990879+0000","last_change":"2024-03-12T17:06:25.452356+0000","last_active":"2024-03-12T17:06:29.990879+0000","last_peered":"2024-03-12T17:06:29.990879+0000","last_clean":"2024-03-12T17:06:29.990879+0000","last_became_active":"2024-03-12T17:06:25.452033+0000","last_became_peered":"2024-03-12T17:06:25.452033+0000","last_unstale":"2024-03-12T17:06:29.990879+0000","last_undegraded":"2024-03-12T17:06:29.990879+0000","last_fullsized":"2024-03-12T17:06:29.990879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:58:32.509940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.477019+0000","last_change":"2024-03-12T17:06:30.023373+0000","last_active":"2024-03-12T17:06:30.477019+0000","last_peered":"2024-03-12T17:06:30.477019+0000","last_clean":"2024-03-12T17:06:30.477019+0000","last_became_active":"2024-03-12T17:06:30.022331+0000","last_became_peered":"2024-03-12T17:06:30.022331+0000","last_unstale":"2024-03-12T17:06:30.477019+0000","last_undegraded":"2024-03-12T17:06:30.477019+0000","last_fullsized":"2024-03-12T17:06:30.477019+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:50:22.021457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.508638+0000","last_change":"2024-03-12T17:06:30.020503+0000","last_active":"2024-03-12T17:06:30.508638+0000","last_peered":"2024-03-12T17:06:30.508638+0000","last_clean":"2024-03-12T17:06:30.508638+0000","last_became_active":"2024-03-12T17:06:30.019897+0000","last_became_peered":"2024-03-12T17:06:30.019897+0000","last_unstale":"2024-03-12T17:06:30.508638+0000","last_undegraded":"2024-03-12T17:06:30.508638+0000","last_fullsized":"2024-03-12T17:06:30.508638+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:13:55.311599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.013985+0000","last_change":"2024-03-12T17:06:25.927847+0000","last_active":"2024-03-12T17:06:30.013985+0000","last_peered":"2024-03-12T17:06:30.013985+0000","last_clean":"2024-03-12T17:06:30.013985+0000","last_became_active":"2024-03-12T17:06:25.927572+0000","last_became_peered":"2024-03-12T17:06:25.927572+0000","last_unstale":"2024-03-12T17:06:30.013985+0000","last_undegraded":"2024-03-12T17:06:30.013985+0000","last_fullsized":"2024-03-12T17:06:30.013985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:29:49.461651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.989420+0000","last_change":"2024-03-12T17:06:25.443473+0000","last_active":"2024-03-12T17:06:29.989420+0000","last_peered":"2024-03-12T17:06:29.989420+0000","last_clean":"2024-03-12T17:06:29.989420+0000","last_became_active":"2024-03-12T17:06:25.443046+0000","last_became_peered":"2024-03-12T17:06:25.443046+0000","last_unstale":"2024-03-12T17:06:29.989420+0000","last_undegraded":"2024-03-12T17:06:29.989420+0000","last_fullsized":"2024-03-12T17:06:29.989420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:49:52.284732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.513045+0000","last_change":"2024-03-12T17:06:30.017610+0000","last_active":"2024-03-12T17:06:30.513045+0000","last_peered":"2024-03-12T17:06:30.513045+0000","last_clean":"2024-03-12T17:06:30.513045+0000","last_became_active":"2024-03-12T17:06:30.017119+0000","last_became_peered":"2024-03-12T17:06:30.017119+0000","last_unstale":"2024-03-12T17:06:30.513045+0000","last_undegraded":"2024-03-12T17:06:30.513045+0000","last_fullsized":"2024-03-12T17:06:30.513045+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:42:49.327905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.986700+0000","last_change":"2024-03-12T17:06:25.935139+0000","last_active":"2024-03-12T17:06:29.986700+0000","last_peered":"2024-03-12T17:06:29.986700+0000","last_clean":"2024-03-12T17:06:29.986700+0000","last_became_active":"2024-03-12T17:06:25.934842+0000","last_became_peered":"2024-03-12T17:06:25.934842+0000","last_unstale":"2024-03-12T17:06:29.986700+0000","last_undegraded":"2024-03-12T17:06:29.986700+0000","last_fullsized":"2024-03-12T17:06:29.986700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:28:59.465721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.989691+0000","last_change":"2024-03-12T17:06:25.447398+0000","last_active":"2024-03-12T17:06:29.989691+0000","last_peered":"2024-03-12T17:06:29.989691+0000","last_clean":"2024-03-12T17:06:29.989691+0000","last_became_active":"2024-03-12T17:06:25.447080+0000","last_became_peered":"2024-03-12T17:06:25.447080+0000","last_unstale":"2024-03-12T17:06:29.989691+0000","last_undegraded":"2024-03-12T17:06:29.989691+0000","last_fullsized":"2024-03-12T17:06:29.989691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:00:32.347614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.986717+0000","last_change":"2024-03-12T17:06:25.447858+0000","last_active":"2024-03-12T17:06:29.986717+0000","last_peered":"2024-03-12T17:06:29.986717+0000","last_clean":"2024-03-12T17:06:29.986717+0000","last_became_active":"2024-03-12T17:06:25.447551+0000","last_became_peered":"2024-03-12T17:06:25.447551+0000","last_unstale":"2024-03-12T17:06:29.986717+0000","last_undegraded":"2024-03-12T17:06:29.986717+0000","last_fullsized":"2024-03-12T17:06:29.986717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:28:25.809578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.996427+0000","last_change":"2024-03-12T17:06:25.943160+0000","last_active":"2024-03-12T17:06:29.996427+0000","last_peered":"2024-03-12T17:06:29.996427+0000","last_clean":"2024-03-12T17:06:29.996427+0000","last_became_active":"2024-03-12T17:06:25.942883+0000","last_became_peered":"2024-03-12T17:06:25.942883+0000","last_unstale":"2024-03-12T17:06:29.996427+0000","last_undegraded":"2024-03-12T17:06:29.996427+0000","last_fullsized":"2024-03-12T17:06:29.996427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:04:04.198678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.509831+0000","last_change":"2024-03-12T17:06:30.018594+0000","last_active":"2024-03-12T17:06:30.509831+0000","last_peered":"2024-03-12T17:06:30.509831+0000","last_clean":"2024-03-12T17:06:30.509831+0000","last_became_active":"2024-03-12T17:06:30.016503+0000","last_became_peered":"2024-03-12T17:06:30.016503+0000","last_unstale":"2024-03-12T17:06:30.509831+0000","last_undegraded":"2024-03-12T17:06:30.509831+0000","last_fullsized":"2024-03-12T17:06:30.509831+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:29:03.020890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.995919+0000","last_change":"2024-03-12T17:06:25.443901+0000","last_active":"2024-03-12T17:06:29.995919+0000","last_peered":"2024-03-12T17:06:29.995919+0000","last_clean":"2024-03-12T17:06:29.995919+0000","last_became_active":"2024-03-12T17:06:25.443607+0000","last_became_peered":"2024-03-12T17:06:25.443607+0000","last_unstale":"2024-03-12T17:06:29.995919+0000","last_undegraded":"2024-03-12T17:06:29.995919+0000","last_fullsized":"2024-03-12T17:06:29.995919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:24.401047+0000","last_clean_scrub_stamp":"2024-03-12T17:06:24.401047+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T05:00:49.177317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.011582+0000","last_change":"2024-03-12T17:06:25.931011+0000","last_active":"2024-03-12T17:06:30.011582+0000","last_peered":"2024-03-12T17:06:30.011582+0000","last_clean":"2024-03-12T17:06:30.011582+0000","last_became_active":"2024-03-12T17:06:25.930306+0000","last_became_peered":"2024-03-12T17:06:25.930306+0000","last_unstale":"2024-03-12T17:06:30.011582+0000","last_undegraded":"2024-03-12T17:06:30.011582+0000","last_fullsized":"2024-03-12T17:06:30.011582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:58:33.513346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.478204+0000","last_change":"2024-03-12T17:06:30.018640+0000","last_active":"2024-03-12T17:06:30.478204+0000","last_peered":"2024-03-12T17:06:30.478204+0000","last_clean":"2024-03-12T17:06:30.478204+0000","last_became_active":"2024-03-12T17:06:30.018278+0000","last_became_peered":"2024-03-12T17:06:30.018278+0000","last_unstale":"2024-03-12T17:06:30.478204+0000","last_undegraded":"2024-03-12T17:06:30.478204+0000","last_fullsized":"2024-03-12T17:06:30.478204+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:28.959623+0000","last_clean_scrub_stamp":"2024-03-12T17:06:28.959623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:01:05.684140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.984555+0000","last_change":"2024-03-12T17:06:25.947304+0000","last_active":"2024-03-12T17:06:29.984555+0000","last_peered":"2024-03-12T17:06:29.984555+0000","last_clean":"2024-03-12T17:06:29.984555+0000","last_became_active":"2024-03-12T17:06:25.947030+0000","last_became_peered":"2024-03-12T17:06:25.947030+0000","last_unstale":"2024-03-12T17:06:29.984555+0000","last_undegraded":"2024-03-12T17:06:29.984555+0000","last_fullsized":"2024-03-12T17:06:29.984555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:29:49.461651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.979886+0000","last_change":"2024-03-12T17:06:25.929080+0000","last_active":"2024-03-12T17:06:29.979886+0000","last_peered":"2024-03-12T17:06:29.979886+0000","last_clean":"2024-03-12T17:06:29.979886+0000","last_became_active":"2024-03-12T17:06:25.928820+0000","last_became_peered":"2024-03-12T17:06:25.928820+0000","last_unstale":"2024-03-12T17:06:29.979886+0000","last_undegraded":"2024-03-12T17:06:29.979886+0000","last_fullsized":"2024-03-12T17:06:29.979886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:36:26.472050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.992041+0000","last_change":"2024-03-12T17:06:25.932309+0000","last_active":"2024-03-12T17:06:29.992041+0000","last_peered":"2024-03-12T17:06:29.992041+0000","last_clean":"2024-03-12T17:06:29.992041+0000","last_became_active":"2024-03-12T17:06:25.932014+0000","last_became_peered":"2024-03-12T17:06:25.932014+0000","last_unstale":"2024-03-12T17:06:29.992041+0000","last_undegraded":"2024-03-12T17:06:29.992041+0000","last_fullsized":"2024-03-12T17:06:29.992041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:11:15.145295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.996376+0000","last_change":"2024-03-12T17:06:25.937407+0000","last_active":"2024-03-12T17:06:29.996376+0000","last_peered":"2024-03-12T17:06:29.996376+0000","last_clean":"2024-03-12T17:06:29.996376+0000","last_became_active":"2024-03-12T17:06:25.937065+0000","last_became_peered":"2024-03-12T17:06:25.937065+0000","last_unstale":"2024-03-12T17:06:29.996376+0000","last_undegraded":"2024-03-12T17:06:29.996376+0000","last_fullsized":"2024-03-12T17:06:29.996376+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:04:28.921316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.994102+0000","last_change":"2024-03-12T17:06:25.930358+0000","last_active":"2024-03-12T17:06:29.994102+0000","last_peered":"2024-03-12T17:06:29.994102+0000","last_clean":"2024-03-12T17:06:29.994102+0000","last_became_active":"2024-03-12T17:06:25.929870+0000","last_became_peered":"2024-03-12T17:06:25.929870+0000","last_unstale":"2024-03-12T17:06:29.994102+0000","last_undegraded":"2024-03-12T17:06:29.994102+0000","last_fullsized":"2024-03-12T17:06:29.994102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:57:56.009742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.021765+0000","last_change":"2024-03-12T17:06:25.928987+0000","last_active":"2024-03-12T17:06:30.021765+0000","last_peered":"2024-03-12T17:06:30.021765+0000","last_clean":"2024-03-12T17:06:30.021765+0000","last_became_active":"2024-03-12T17:06:25.928717+0000","last_became_peered":"2024-03-12T17:06:25.928717+0000","last_unstale":"2024-03-12T17:06:30.021765+0000","last_undegraded":"2024-03-12T17:06:30.021765+0000","last_fullsized":"2024-03-12T17:06:30.021765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:23:25.984714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990238+0000","last_change":"2024-03-12T17:06:25.943871+0000","last_active":"2024-03-12T17:06:29.990238+0000","last_peered":"2024-03-12T17:06:29.990238+0000","last_clean":"2024-03-12T17:06:29.990238+0000","last_became_active":"2024-03-12T17:06:25.938802+0000","last_became_peered":"2024-03-12T17:06:25.938802+0000","last_unstale":"2024-03-12T17:06:29.990238+0000","last_undegraded":"2024-03-12T17:06:29.990238+0000","last_fullsized":"2024-03-12T17:06:29.990238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:23:25.984714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.004028+0000","last_change":"2024-03-12T17:06:25.935272+0000","last_active":"2024-03-12T17:06:30.004028+0000","last_peered":"2024-03-12T17:06:30.004028+0000","last_clean":"2024-03-12T17:06:30.004028+0000","last_became_active":"2024-03-12T17:06:25.928645+0000","last_became_peered":"2024-03-12T17:06:25.928645+0000","last_unstale":"2024-03-12T17:06:30.004028+0000","last_undegraded":"2024-03-12T17:06:30.004028+0000","last_fullsized":"2024-03-12T17:06:30.004028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:44:25.750833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.989166+0000","last_change":"2024-03-12T17:06:25.937035+0000","last_active":"2024-03-12T17:06:29.989166+0000","last_peered":"2024-03-12T17:06:29.989166+0000","last_clean":"2024-03-12T17:06:29.989166+0000","last_became_active":"2024-03-12T17:06:25.936598+0000","last_became_peered":"2024-03-12T17:06:25.936598+0000","last_unstale":"2024-03-12T17:06:29.989166+0000","last_undegraded":"2024-03-12T17:06:29.989166+0000","last_fullsized":"2024-03-12T17:06:29.989166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:21:02.182215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.989614+0000","last_change":"2024-03-12T17:06:25.931310+0000","last_active":"2024-03-12T17:06:29.989614+0000","last_peered":"2024-03-12T17:06:29.989614+0000","last_clean":"2024-03-12T17:06:29.989614+0000","last_became_active":"2024-03-12T17:06:25.930910+0000","last_became_peered":"2024-03-12T17:06:25.930910+0000","last_unstale":"2024-03-12T17:06:29.989614+0000","last_undegraded":"2024-03-12T17:06:29.989614+0000","last_fullsized":"2024-03-12T17:06:29.989614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:48:33.016370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.991821+0000","last_change":"2024-03-12T17:06:25.937111+0000","last_active":"2024-03-12T17:06:29.991821+0000","last_peered":"2024-03-12T17:06:29.991821+0000","last_clean":"2024-03-12T17:06:29.991821+0000","last_became_active":"2024-03-12T17:06:25.936895+0000","last_became_peered":"2024-03-12T17:06:25.936895+0000","last_unstale":"2024-03-12T17:06:29.991821+0000","last_undegraded":"2024-03-12T17:06:29.991821+0000","last_fullsized":"2024-03-12T17:06:29.991821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:38:01.478195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.996309+0000","last_change":"2024-03-12T17:06:25.947115+0000","last_active":"2024-03-12T17:06:29.996309+0000","last_peered":"2024-03-12T17:06:29.996309+0000","last_clean":"2024-03-12T17:06:29.996309+0000","last_became_active":"2024-03-12T17:06:25.946820+0000","last_became_peered":"2024-03-12T17:06:25.946820+0000","last_unstale":"2024-03-12T17:06:29.996309+0000","last_undegraded":"2024-03-12T17:06:29.996309+0000","last_fullsized":"2024-03-12T17:06:29.996309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:48:33.016370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.984992+0000","last_change":"2024-03-12T17:06:25.950973+0000","last_active":"2024-03-12T17:06:29.984992+0000","last_peered":"2024-03-12T17:06:29.984992+0000","last_clean":"2024-03-12T17:06:29.984992+0000","last_became_active":"2024-03-12T17:06:25.948336+0000","last_became_peered":"2024-03-12T17:06:25.948336+0000","last_unstale":"2024-03-12T17:06:29.984992+0000","last_undegraded":"2024-03-12T17:06:29.984992+0000","last_fullsized":"2024-03-12T17:06:29.984992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:04:04.198678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012237+0000","last_change":"2024-03-12T17:06:25.961536+0000","last_active":"2024-03-12T17:06:30.012237+0000","last_peered":"2024-03-12T17:06:30.012237+0000","last_clean":"2024-03-12T17:06:30.012237+0000","last_became_active":"2024-03-12T17:06:25.961147+0000","last_became_peered":"2024-03-12T17:06:25.961147+0000","last_unstale":"2024-03-12T17:06:30.012237+0000","last_undegraded":"2024-03-12T17:06:30.012237+0000","last_fullsized":"2024-03-12T17:06:30.012237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:06:19.464242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.979994+0000","last_change":"2024-03-12T17:06:25.939977+0000","last_active":"2024-03-12T17:06:29.979994+0000","last_peered":"2024-03-12T17:06:29.979994+0000","last_clean":"2024-03-12T17:06:29.979994+0000","last_became_active":"2024-03-12T17:06:25.939725+0000","last_became_peered":"2024-03-12T17:06:25.939725+0000","last_unstale":"2024-03-12T17:06:29.979994+0000","last_undegraded":"2024-03-12T17:06:29.979994+0000","last_fullsized":"2024-03-12T17:06:29.979994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:45:02.929901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.991517+0000","last_change":"2024-03-12T17:06:25.939649+0000","last_active":"2024-03-12T17:06:29.991517+0000","last_peered":"2024-03-12T17:06:29.991517+0000","last_clean":"2024-03-12T17:06:29.991517+0000","last_became_active":"2024-03-12T17:06:25.939379+0000","last_became_peered":"2024-03-12T17:06:25.939379+0000","last_unstale":"2024-03-12T17:06:29.991517+0000","last_undegraded":"2024-03-12T17:06:29.991517+0000","last_fullsized":"2024-03-12T17:06:29.991517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:17:09.067136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.988936+0000","last_change":"2024-03-12T17:06:25.948556+0000","last_active":"2024-03-12T17:06:29.988936+0000","last_peered":"2024-03-12T17:06:29.988936+0000","last_clean":"2024-03-12T17:06:29.988936+0000","last_became_active":"2024-03-12T17:06:25.948217+0000","last_became_peered":"2024-03-12T17:06:25.948217+0000","last_unstale":"2024-03-12T17:06:29.988936+0000","last_undegraded":"2024-03-12T17:06:29.988936+0000","last_fullsized":"2024-03-12T17:06:29.988936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:26:54.235648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.013984+0000","last_change":"2024-03-12T17:06:25.931098+0000","last_active":"2024-03-12T17:06:30.013984+0000","last_peered":"2024-03-12T17:06:30.013984+0000","last_clean":"2024-03-12T17:06:30.013984+0000","last_became_active":"2024-03-12T17:06:25.930273+0000","last_became_peered":"2024-03-12T17:06:25.930273+0000","last_unstale":"2024-03-12T17:06:30.013984+0000","last_undegraded":"2024-03-12T17:06:30.013984+0000","last_fullsized":"2024-03-12T17:06:30.013984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T20:48:48.332409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.996609+0000","last_change":"2024-03-12T17:06:25.933421+0000","last_active":"2024-03-12T17:06:29.996609+0000","last_peered":"2024-03-12T17:06:29.996609+0000","last_clean":"2024-03-12T17:06:29.996609+0000","last_became_active":"2024-03-12T17:06:25.933133+0000","last_became_peered":"2024-03-12T17:06:25.933133+0000","last_unstale":"2024-03-12T17:06:29.996609+0000","last_undegraded":"2024-03-12T17:06:29.996609+0000","last_fullsized":"2024-03-12T17:06:29.996609+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:28:59.465721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.991543+0000","last_change":"2024-03-12T17:06:25.948622+0000","last_active":"2024-03-12T17:06:29.991543+0000","last_peered":"2024-03-12T17:06:29.991543+0000","last_clean":"2024-03-12T17:06:29.991543+0000","last_became_active":"2024-03-12T17:06:25.948293+0000","last_became_peered":"2024-03-12T17:06:25.948293+0000","last_unstale":"2024-03-12T17:06:29.991543+0000","last_undegraded":"2024-03-12T17:06:29.991543+0000","last_fullsized":"2024-03-12T17:06:29.991543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:52:14.405796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012611+0000","last_change":"2024-03-12T17:06:25.944171+0000","last_active":"2024-03-12T17:06:30.012611+0000","last_peered":"2024-03-12T17:06:30.012611+0000","last_clean":"2024-03-12T17:06:30.012611+0000","last_became_active":"2024-03-12T17:06:25.940309+0000","last_became_peered":"2024-03-12T17:06:25.940309+0000","last_unstale":"2024-03-12T17:06:30.012611+0000","last_undegraded":"2024-03-12T17:06:30.012611+0000","last_fullsized":"2024-03-12T17:06:30.012611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:00:07.118282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.987793+0000","last_change":"2024-03-12T17:06:25.946194+0000","last_active":"2024-03-12T17:06:29.987793+0000","last_peered":"2024-03-12T17:06:29.987793+0000","last_clean":"2024-03-12T17:06:29.987793+0000","last_became_active":"2024-03-12T17:06:25.944383+0000","last_became_peered":"2024-03-12T17:06:25.944383+0000","last_unstale":"2024-03-12T17:06:29.987793+0000","last_undegraded":"2024-03-12T17:06:29.987793+0000","last_fullsized":"2024-03-12T17:06:29.987793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:48:33.016370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.989993+0000","last_change":"2024-03-12T17:06:25.936660+0000","last_active":"2024-03-12T17:06:29.989993+0000","last_peered":"2024-03-12T17:06:29.989993+0000","last_clean":"2024-03-12T17:06:29.989993+0000","last_became_active":"2024-03-12T17:06:25.936366+0000","last_became_peered":"2024-03-12T17:06:25.936366+0000","last_unstale":"2024-03-12T17:06:29.989993+0000","last_undegraded":"2024-03-12T17:06:29.989993+0000","last_fullsized":"2024-03-12T17:06:29.989993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:44:34.582031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.013702+0000","last_change":"2024-03-12T17:06:25.960447+0000","last_active":"2024-03-12T17:06:30.013702+0000","last_peered":"2024-03-12T17:06:30.013702+0000","last_clean":"2024-03-12T17:06:30.013702+0000","last_became_active":"2024-03-12T17:06:25.958764+0000","last_became_peered":"2024-03-12T17:06:25.958764+0000","last_unstale":"2024-03-12T17:06:30.013702+0000","last_undegraded":"2024-03-12T17:06:30.013702+0000","last_fullsized":"2024-03-12T17:06:30.013702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:16:50.369045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.987173+0000","last_change":"2024-03-12T17:06:25.928162+0000","last_active":"2024-03-12T17:06:29.987173+0000","last_peered":"2024-03-12T17:06:29.987173+0000","last_clean":"2024-03-12T17:06:29.987173+0000","last_became_active":"2024-03-12T17:06:25.927864+0000","last_became_peered":"2024-03-12T17:06:25.927864+0000","last_unstale":"2024-03-12T17:06:29.987173+0000","last_undegraded":"2024-03-12T17:06:29.987173+0000","last_fullsized":"2024-03-12T17:06:29.987173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:28:58.739417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.018239+0000","last_change":"2024-03-12T17:06:25.956401+0000","last_active":"2024-03-12T17:06:30.018239+0000","last_peered":"2024-03-12T17:06:30.018239+0000","last_clean":"2024-03-12T17:06:30.018239+0000","last_became_active":"2024-03-12T17:06:25.955835+0000","last_became_peered":"2024-03-12T17:06:25.955835+0000","last_unstale":"2024-03-12T17:06:30.018239+0000","last_undegraded":"2024-03-12T17:06:30.018239+0000","last_fullsized":"2024-03-12T17:06:30.018239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:11:15.145295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.985932+0000","last_change":"2024-03-12T17:06:25.952812+0000","last_active":"2024-03-12T17:06:29.985932+0000","last_peered":"2024-03-12T17:06:29.985932+0000","last_clean":"2024-03-12T17:06:29.985932+0000","last_became_active":"2024-03-12T17:06:25.946931+0000","last_became_peered":"2024-03-12T17:06:25.946931+0000","last_unstale":"2024-03-12T17:06:29.985932+0000","last_undegraded":"2024-03-12T17:06:29.985932+0000","last_fullsized":"2024-03-12T17:06:29.985932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:16:50.369045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.012000+0000","last_change":"2024-03-12T17:06:25.950752+0000","last_active":"2024-03-12T17:06:30.012000+0000","last_peered":"2024-03-12T17:06:30.012000+0000","last_clean":"2024-03-12T17:06:30.012000+0000","last_became_active":"2024-03-12T17:06:25.950440+0000","last_became_peered":"2024-03-12T17:06:25.950440+0000","last_unstale":"2024-03-12T17:06:30.012000+0000","last_undegraded":"2024-03-12T17:06:30.012000+0000","last_fullsized":"2024-03-12T17:06:30.012000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:38:51.036295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.990839+0000","last_change":"2024-03-12T17:06:25.944733+0000","last_active":"2024-03-12T17:06:29.990839+0000","last_peered":"2024-03-12T17:06:29.990839+0000","last_clean":"2024-03-12T17:06:29.990839+0000","last_became_active":"2024-03-12T17:06:25.942878+0000","last_became_peered":"2024-03-12T17:06:25.942878+0000","last_unstale":"2024-03-12T17:06:29.990839+0000","last_undegraded":"2024-03-12T17:06:29.990839+0000","last_fullsized":"2024-03-12T17:06:29.990839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:15:36.195143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:29.996580+0000","last_change":"2024-03-12T17:06:25.943005+0000","last_active":"2024-03-12T17:06:29.996580+0000","last_peered":"2024-03-12T17:06:29.996580+0000","last_clean":"2024-03-12T17:06:29.996580+0000","last_became_active":"2024-03-12T17:06:25.933118+0000","last_became_peered":"2024-03-12T17:06:25.933118+0000","last_unstale":"2024-03-12T17:06:29.996580+0000","last_undegraded":"2024-03-12T17:06:29.996580+0000","last_fullsized":"2024-03-12T17:06:29.996580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:44:25.750833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.022750+0000","last_change":"2024-03-12T17:06:25.947390+0000","last_active":"2024-03-12T17:06:30.022750+0000","last_peered":"2024-03-12T17:06:30.022750+0000","last_clean":"2024-03-12T17:06:30.022750+0000","last_became_active":"2024-03-12T17:06:25.947027+0000","last_became_peered":"2024-03-12T17:06:25.947027+0000","last_unstale":"2024-03-12T17:06:30.022750+0000","last_undegraded":"2024-03-12T17:06:30.022750+0000","last_fullsized":"2024-03-12T17:06:30.022750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:44:34.582031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-12T17:06:30.013617+0000","last_change":"2024-03-12T17:06:25.958459+0000","last_active":"2024-03-12T17:06:30.013617+0000","last_peered":"2024-03-12T17:06:30.013617+0000","last_clean":"2024-03-12T17:06:30.013617+0000","last_became_active":"2024-03-12T17:06:25.956687+0000","last_became_peered":"2024-03-12T17:06:25.956687+0000","last_unstale":"2024-03-12T17:06:30.013617+0000","last_undegraded":"2024-03-12T17:06:30.013617+0000","last_fullsized":"2024-03-12T17:06:30.013617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:25.404453+0000","last_clean_scrub_stamp":"2024-03-12T17:06:25.404453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:26:54.235648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4593,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":864256,"data_stored":695401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.286}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2130000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]}]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.143}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":270336,"data_stored":104541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3129999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.518}]}]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":104577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3120000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.339}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.45}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.393}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2270000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3720000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":278528,"data_stored":108246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.228}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3779999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.331}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":104451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4159999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1950000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.456}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":274432,"data_stored":104611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.107}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.496}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":872448,"data_stored":695457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.204}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0980000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5489999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-12T17:07:01.742 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-12T17:07:01.742 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-12T17:07:01.743 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-12T17:07:01.743 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-12T17:07:02.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.006+0000 7feb13600640 1 Processor -- start 2024-03-12T17:07:02.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.007+0000 7feb13600640 1 -- start start 2024-03-12T17:07:02.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.008+0000 7feb13600640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feb0c0b39b0 0x7feb0c0b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:02.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.008+0000 7feb13600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7feb0c15c400 0x7feb0c15c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:02.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.008+0000 7feb13600640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb0c06c6f0 con 0x7feb0c15b560 2024-03-12T17:07:02.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.008+0000 7feb13600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb0c0b4370 con 0x7feb0c15c400 2024-03-12T17:07:02.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.008+0000 7feb13600640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c06c9d0 con 0x7feb0c0b39b0 2024-03-12T17:07:02.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.009+0000 7feb11600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7feb0c15c400 0x7feb0c15c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:02.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.009+0000 7feb03e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feb0c0b39b0 0x7feb0c0b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:02.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.009+0000 7feb11600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7feb0c15c400 0x7feb0c15c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:38900/0 (socket says 172.21.15.52:38900) 2024-03-12T17:07:02.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.009+0000 7feb11600640 1 -- 172.21.15.52:0/1438073095 learned_addr learned my addr 172.21.15.52:0/1438073095 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:02.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.010+0000 7feb03e00640 1 -- 172.21.15.52:0/1438073095 >> v2:172.21.15.102:3300/0 conn(0x7feb0c15c400 msgr2=0x7feb0c15c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:02.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.010+0000 7feb03e00640 1 --2- 172.21.15.52:0/1438073095 >> v2:172.21.15.102:3300/0 conn(0x7feb0c15c400 0x7feb0c15c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:02.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.010+0000 7feb03e00640 1 -- 172.21.15.52:0/1438073095 >> v1:172.21.15.102:6789/0 conn(0x7feb0c15b560 legacy=0x7feb0c15b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:02.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.010+0000 7feb03e00640 1 -- 172.21.15.52:0/1438073095 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0c06ccb0 con 0x7feb0c0b39b0 2024-03-12T17:07:02.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.010+0000 7feb11600640 1 --2- 172.21.15.52:0/1438073095 >> v2:172.21.15.102:3300/0 conn(0x7feb0c15c400 0x7feb0c15c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T17:07:02.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.011+0000 7feb03e00640 1 --2- 172.21.15.52:0/1438073095 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feb0c0b39b0 0x7feb0c0b3e30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7feafc0097a0 tx=0x7feafc02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b03d4fb6d18671de server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:02.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.012+0000 7feb0be00640 1 -- 172.21.15.52:0/1438073095 <== mon.2 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc031830 con 0x7feb0c0b39b0 2024-03-12T17:07:02.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.012+0000 7feb0be00640 1 -- 172.21.15.52:0/1438073095 <== mon.2 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feafc031990 con 0x7feb0c0b39b0 2024-03-12T17:07:02.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.012+0000 7feb0be00640 1 -- 172.21.15.52:0/1438073095 <== mon.2 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc031d20 con 0x7feb0c0b39b0 2024-03-12T17:07:02.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.012+0000 7feb13600640 1 -- 172.21.15.52:0/1438073095 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feb0c0b39b0 msgr2=0x7feb0c0b3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:02.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.012+0000 7feb13600640 1 --2- 172.21.15.52:0/1438073095 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feb0c0b39b0 0x7feb0c0b3e30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7feafc0097a0 tx=0x7feafc02b8d0 comp rx=0 tx=0).stop 2024-03-12T17:07:02.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.013+0000 7feb13600640 1 -- 172.21.15.52:0/1438073095 shutdown_connections 2024-03-12T17:07:02.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.013+0000 7feb13600640 1 --2- 172.21.15.52:0/1438073095 >> v2:172.21.15.102:3300/0 conn(0x7feb0c15c400 0x7feb0c15c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:02.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.013+0000 7feb13600640 1 --2- 172.21.15.52:0/1438073095 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feb0c0b39b0 0x7feb0c0b3e30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:02.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.013+0000 7feb13600640 1 -- 172.21.15.52:0/1438073095 >> 172.21.15.52:0/1438073095 conn(0x7feb0c06a3d0 msgr2=0x7feb0c0b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:02.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.013+0000 7feb13600640 1 -- 172.21.15.52:0/1438073095 shutdown_connections 2024-03-12T17:07:02.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.013+0000 7feb13600640 1 -- 172.21.15.52:0/1438073095 wait complete. 2024-03-12T17:07:02.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.015+0000 7feb13600640 1 Processor -- start 2024-03-12T17:07:02.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.015+0000 7feb13600640 1 -- start start 2024-03-12T17:07:02.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.017+0000 7feb13600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb0c0b39b0 0x7feb0c145040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:02.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.017+0000 7feb13600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb0c15b560 0x7feb0c151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:02.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.017+0000 7feb13600640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feb0c15c400 0x7feb0c14d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:02.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.017+0000 7feb13600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c145580 con 0x7feb0c0b39b0 2024-03-12T17:07:02.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.017+0000 7feb13600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c1e4f90 con 0x7feb0c15b560 2024-03-12T17:07:02.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.017+0000 7feb13600640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c14cdd0 con 0x7feb0c15c400 2024-03-12T17:07:02.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.017+0000 7feb10c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb0c0b39b0 0x7feb0c145040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:02.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.018+0000 7feb10c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb0c0b39b0 0x7feb0c145040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:32942/0 (socket says 172.21.15.52:32942) 2024-03-12T17:07:02.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.018+0000 7feb10c00640 1 -- 172.21.15.52:0/2912163493 learned_addr learned my addr 172.21.15.52:0/2912163493 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:02.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.018+0000 7feb03e00640 1 --2- 172.21.15.52:0/2912163493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb0c15b560 0x7feb0c151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:02.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.018+0000 7feb10c00640 1 -- 172.21.15.52:0/2912163493 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feb0c15c400 msgr2=0x7feb0c14d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:02.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.018+0000 7feb10c00640 1 --2- 172.21.15.52:0/2912163493 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feb0c15c400 0x7feb0c14d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:02.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.018+0000 7feb10c00640 1 -- 172.21.15.52:0/2912163493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb0c15b560 msgr2=0x7feb0c151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:02.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.018+0000 7feb10c00640 1 --2- 172.21.15.52:0/2912163493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb0c15b560 0x7feb0c151fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:02.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.018+0000 7feb10c00640 1 -- 172.21.15.52:0/2912163493 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb04009b30 con 0x7feb0c0b39b0 2024-03-12T17:07:02.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.018+0000 7feb03e00640 1 --2- 172.21.15.52:0/2912163493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb0c15b560 0x7feb0c151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:02.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.018+0000 7feb10c00640 1 --2- 172.21.15.52:0/2912163493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb0c0b39b0 0x7feb0c145040 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7feaf400c480 tx=0x7feaf400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:02.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.019+0000 7feb09600640 1 -- 172.21.15.52:0/2912163493 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf40052d0 con 0x7feb0c0b39b0 2024-03-12T17:07:02.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.019+0000 7feb09600640 1 -- 172.21.15.52:0/2912163493 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf4005430 con 0x7feb0c0b39b0 2024-03-12T17:07:02.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.019+0000 7feb09600640 1 -- 172.21.15.52:0/2912163493 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf4003830 con 0x7feb0c0b39b0 2024-03-12T17:07:02.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.019+0000 7feb13600640 1 -- 172.21.15.52:0/2912163493 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feafc009400 con 0x7feb0c0b39b0 2024-03-12T17:07:02.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.019+0000 7feb13600640 1 -- 172.21.15.52:0/2912163493 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb0c1e4660 con 0x7feb0c0b39b0 2024-03-12T17:07:02.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.021+0000 7feb13600640 1 -- 172.21.15.52:0/2912163493 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0c06ac20 con 0x7feb0c0b39b0 2024-03-12T17:07:02.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.022+0000 7feb09600640 1 -- 172.21.15.52:0/2912163493 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7feaf4003990 con 0x7feb0c0b39b0 2024-03-12T17:07:02.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.023+0000 7feb09600640 1 --2- 172.21.15.52:0/2912163493 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7feae806ac80 0x7feae806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:02.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.023+0000 7feb09600640 1 -- 172.21.15.52:0/2912163493 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7feaf4088c10 con 0x7feb0c0b39b0 2024-03-12T17:07:02.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.024+0000 7feb03e00640 1 --2- 172.21.15.52:0/2912163493 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7feae806ac80 0x7feae806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:02.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.027+0000 7feb03e00640 1 --2- 172.21.15.52:0/2912163493 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7feae806ac80 0x7feae806d140 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7feafc036000 tx=0x7feafc03c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:02.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.029+0000 7feb09600640 1 -- 172.21.15.52:0/2912163493 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7feaf4053b70 con 0x7feb0c0b39b0 2024-03-12T17:07:02.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.270+0000 7feb13600640 1 -- 172.21.15.52:0/2912163493 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7feb0c06dd50 con 0x7feb0c0b39b0 2024-03-12T17:07:02.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.270+0000 7feb09600640 1 -- 172.21.15.52:0/2912163493 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7feaf4057190 con 0x7feb0c0b39b0 2024-03-12T17:07:02.264 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:07:02.264 INFO:teuthology.orchestra.run.smithi052.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-12T17:07:02.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.273+0000 7feb13600640 1 -- 172.21.15.52:0/2912163493 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7feae806ac80 msgr2=0x7feae806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:02.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.273+0000 7feb13600640 1 --2- 172.21.15.52:0/2912163493 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7feae806ac80 0x7feae806d140 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7feafc036000 tx=0x7feafc03c040 comp rx=0 tx=0).stop 2024-03-12T17:07:02.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.273+0000 7feb13600640 1 -- 172.21.15.52:0/2912163493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb0c0b39b0 msgr2=0x7feb0c145040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:02.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.273+0000 7feb13600640 1 --2- 172.21.15.52:0/2912163493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb0c0b39b0 0x7feb0c145040 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7feaf400c480 tx=0x7feaf400c940 comp rx=0 tx=0).stop 2024-03-12T17:07:02.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.273+0000 7feb13600640 1 -- 172.21.15.52:0/2912163493 shutdown_connections 2024-03-12T17:07:02.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.273+0000 7feb13600640 1 --2- 172.21.15.52:0/2912163493 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7feae806ac80 0x7feae806d140 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:02.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.273+0000 7feb13600640 1 --2- 172.21.15.52:0/2912163493 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feb0c15c400 0x7feb0c14d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:02.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.273+0000 7feb13600640 1 --2- 172.21.15.52:0/2912163493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb0c15b560 0x7feb0c151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:02.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.273+0000 7feb13600640 1 --2- 172.21.15.52:0/2912163493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb0c0b39b0 0x7feb0c145040 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:02.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.273+0000 7feb13600640 1 -- 172.21.15.52:0/2912163493 >> 172.21.15.52:0/2912163493 conn(0x7feb0c06a3d0 msgr2=0x7feb0c15b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:02.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.273+0000 7feb13600640 1 -- 172.21.15.52:0/2912163493 shutdown_connections 2024-03-12T17:07:02.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.274+0000 7feb13600640 1 -- 172.21.15.52:0/2912163493 wait complete. 2024-03-12T17:07:02.286 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-12T17:07:02.287 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:07:02.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.513+0000 7f455e200640 1 Processor -- start 2024-03-12T17:07:02.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.514+0000 7f455e200640 1 -- start start 2024-03-12T17:07:02.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.515+0000 7f455e200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f45580b5a30 0x7f45580b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:02.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.515+0000 7f455e200640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45580d1830 con 0x7f455815cda0 2024-03-12T17:07:02.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.515+0000 7f455e200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45580b1f20 con 0x7f45580b4350 2024-03-12T17:07:02.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.515+0000 7f455e200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45580d0c20 con 0x7f45580b5a30 2024-03-12T17:07:02.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.515+0000 7f4557400640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f455815cda0 0x7f45580cdb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55070/0 (socket says 172.21.15.52:55070) 2024-03-12T17:07:02.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.515+0000 7f4557400640 1 -- 172.21.15.52:0/3901322270 learned_addr learned my addr 172.21.15.52:0/3901322270 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:02.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.516+0000 7f4556a00640 1 --2- 172.21.15.52:0/3901322270 >> v2:172.21.15.102:3300/0 conn(0x7f45580b5a30 0x7f45580b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:02.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.516+0000 7f4556000640 1 -- 172.21.15.52:0/3901322270 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2287686306 0 0) 0x7f45580d1830 con 0x7f455815cda0 2024-03-12T17:07:02.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.516+0000 7f4556000640 1 -- 172.21.15.52:0/3901322270 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f453c003520 con 0x7f455815cda0 2024-03-12T17:07:02.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.516+0000 7f4556000640 1 -- 172.21.15.52:0/3901322270 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2644748020 0 0) 0x7f453c003520 con 0x7f455815cda0 2024-03-12T17:07:02.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.516+0000 7f4556000640 1 -- 172.21.15.52:0/3901322270 >> v2:172.21.15.102:3300/0 conn(0x7f45580b5a30 msgr2=0x7f45580b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:02.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.516+0000 7f4556000640 1 --2- 172.21.15.52:0/3901322270 >> v2:172.21.15.102:3300/0 conn(0x7f45580b5a30 0x7f45580b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:02.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.516+0000 7f4556000640 1 -- 172.21.15.52:0/3901322270 >> v1:172.21.15.102:6789/0 conn(0x7f45580b4350 legacy=0x7f45580d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:02.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.516+0000 7f4556000640 1 -- 172.21.15.52:0/3901322270 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45580d0f00 con 0x7f455815cda0 2024-03-12T17:07:02.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.516+0000 7f4556000640 1 -- 172.21.15.52:0/3901322270 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f45480019f0 con 0x7f455815cda0 2024-03-12T17:07:02.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.516+0000 7f4556a00640 1 --2- 172.21.15.52:0/3901322270 >> v2:172.21.15.102:3300/0 conn(0x7f45580b5a30 0x7f45580b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:07:02.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.517+0000 7f4556000640 1 -- 172.21.15.52:0/3901322270 >> v1:172.21.15.52:6789/0 conn(0x7f455815cda0 legacy=0x7f45580cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:02.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.517+0000 7f4556000640 1 --2- 172.21.15.52:0/3901322270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f453c003970 0x7f453c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:02.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.517+0000 7f4556000640 1 --2- 172.21.15.52:0/3901322270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f453c0047c0 0x7f453c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:02.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.517+0000 7f4556000640 1 --2- 172.21.15.52:0/3901322270 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f453c005180 0x7f453c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:02.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.517+0000 7f4556000640 1 -- 172.21.15.52:0/3901322270 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45580d0c20 con 0x7f453c003970 2024-03-12T17:07:02.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.517+0000 7f4556000640 1 -- 172.21.15.52:0/3901322270 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453c00c0d0 con 0x7f453c0047c0 2024-03-12T17:07:02.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.517+0000 7f4556000640 1 -- 172.21.15.52:0/3901322270 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f453c00c290 con 0x7f453c005180 2024-03-12T17:07:02.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.517+0000 7f4557400640 1 --2- 172.21.15.52:0/3901322270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f453c003970 0x7f453c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:02.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.517+0000 7f4556a00640 1 --2- 172.21.15.52:0/3901322270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f453c0047c0 0x7f453c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:02.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.517+0000 7f4557e00640 1 --2- 172.21.15.52:0/3901322270 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f453c005180 0x7f453c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:02.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.517+0000 7f4557400640 1 -- 172.21.15.52:0/3901322270 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f453c005180 msgr2=0x7f453c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:02.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.517+0000 7f4557400640 1 --2- 172.21.15.52:0/3901322270 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f453c005180 0x7f453c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:02.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.517+0000 7f4557400640 1 -- 172.21.15.52:0/3901322270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f453c0047c0 msgr2=0x7f453c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:02.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.517+0000 7f4557400640 1 --2- 172.21.15.52:0/3901322270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f453c0047c0 0x7f453c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:02.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.517+0000 7f4557400640 1 -- 172.21.15.52:0/3901322270 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f453c003520 con 0x7f453c003970 2024-03-12T17:07:02.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.517+0000 7f4557e00640 1 -- 172.21.15.52:0/3901322270 reap_dead start 2024-03-12T17:07:02.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.518+0000 7f4557400640 1 --2- 172.21.15.52:0/3901322270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f453c003970 0x7f453c003d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f454800a830 tx=0x7f453c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9062a7263844bb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:02.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.519+0000 7f4556000640 1 -- 172.21.15.52:0/3901322270 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45480072d0 con 0x7f453c003970 2024-03-12T17:07:02.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.519+0000 7f4556000640 1 -- 172.21.15.52:0/3901322270 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f454802f070 con 0x7f453c003970 2024-03-12T17:07:02.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.519+0000 7f4556000640 1 -- 172.21.15.52:0/3901322270 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4548007730 con 0x7f453c003970 2024-03-12T17:07:02.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.519+0000 7f455e200640 1 -- 172.21.15.52:0/3901322270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f453c003970 msgr2=0x7f453c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:02.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.519+0000 7f455e200640 1 --2- 172.21.15.52:0/3901322270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f453c003970 0x7f453c003d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f454800a830 tx=0x7f453c0057f0 comp rx=0 tx=0).stop 2024-03-12T17:07:02.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.520+0000 7f455e200640 1 -- 172.21.15.52:0/3901322270 shutdown_connections 2024-03-12T17:07:02.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.520+0000 7f455e200640 1 --2- 172.21.15.52:0/3901322270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f453c003970 0x7f453c003d90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:02.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.520+0000 7f455e200640 1 -- 172.21.15.52:0/3901322270 >> 172.21.15.52:0/3901322270 conn(0x7f45580cf350 msgr2=0x7f45580cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:02.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.520+0000 7f455e200640 1 -- 172.21.15.52:0/3901322270 shutdown_connections 2024-03-12T17:07:02.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.520+0000 7f455e200640 1 -- 172.21.15.52:0/3901322270 wait complete. 2024-03-12T17:07:02.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.521+0000 7f455e200640 1 Processor -- start 2024-03-12T17:07:02.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.522+0000 7f455e200640 1 -- start start 2024-03-12T17:07:02.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.522+0000 7f455e200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f453c003970 0x7f455814f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:02.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.522+0000 7f455e200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4558152170 0x7f455814d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:02.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.522+0000 7f455e200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f455814bb40 0x7f455814d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:02.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.522+0000 7f455e200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45580c92a0 con 0x7f455814bb40 2024-03-12T17:07:02.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.522+0000 7f455e200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455814ded0 con 0x7f453c003970 2024-03-12T17:07:02.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.522+0000 7f455e200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f455814c0d0 con 0x7f4558152170 2024-03-12T17:07:02.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.523+0000 7f4557e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f455814bb40 0x7f455814d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:02.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.523+0000 7f4557e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f455814bb40 0x7f455814d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:32966/0 (socket says 172.21.15.52:32966) 2024-03-12T17:07:02.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.523+0000 7f4557e00640 1 -- 172.21.15.52:0/1118347546 learned_addr learned my addr 172.21.15.52:0/1118347546 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:02.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.523+0000 7f4556a00640 1 --2- 172.21.15.52:0/1118347546 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4558152170 0x7f455814d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:02.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.523+0000 7f4557400640 1 --2- 172.21.15.52:0/1118347546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f453c003970 0x7f455814f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:02.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.523+0000 7f4557e00640 1 -- 172.21.15.52:0/1118347546 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4558152170 msgr2=0x7f455814d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:02.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.523+0000 7f4557e00640 1 --2- 172.21.15.52:0/1118347546 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4558152170 0x7f455814d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:02.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.523+0000 7f4557e00640 1 -- 172.21.15.52:0/1118347546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f453c003970 msgr2=0x7f455814f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:02.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.523+0000 7f4557e00640 1 --2- 172.21.15.52:0/1118347546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f453c003970 0x7f455814f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:02.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.523+0000 7f4557e00640 1 -- 172.21.15.52:0/1118347546 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455814c3b0 con 0x7f455814bb40 2024-03-12T17:07:02.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.524+0000 7f4557e00640 1 --2- 172.21.15.52:0/1118347546 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f455814bb40 0x7f455814d990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f45580b54c0 tx=0x7f455000d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:02.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.524+0000 7f454f400640 1 -- 172.21.15.52:0/1118347546 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455000b070 con 0x7f455814bb40 2024-03-12T17:07:02.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.524+0000 7f454f400640 1 -- 172.21.15.52:0/1118347546 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45500054e0 con 0x7f455814bb40 2024-03-12T17:07:02.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.524+0000 7f454f400640 1 -- 172.21.15.52:0/1118347546 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4550007ea0 con 0x7f455814bb40 2024-03-12T17:07:02.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.524+0000 7f455e200640 1 -- 172.21.15.52:0/1118347546 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f455814c690 con 0x7f455814bb40 2024-03-12T17:07:02.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.524+0000 7f455e200640 1 -- 172.21.15.52:0/1118347546 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f455814cc50 con 0x7f455814bb40 2024-03-12T17:07:02.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.525+0000 7f455e200640 1 -- 172.21.15.52:0/1118347546 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4558073390 con 0x7f455814bb40 2024-03-12T17:07:02.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.526+0000 7f454f400640 1 -- 172.21.15.52:0/1118347546 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4550003810 con 0x7f455814bb40 2024-03-12T17:07:02.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.527+0000 7f454f400640 1 --2- 172.21.15.52:0/1118347546 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f453406ac30 0x7f453406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:02.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.527+0000 7f454f400640 1 -- 172.21.15.52:0/1118347546 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4550088bc0 con 0x7f455814bb40 2024-03-12T17:07:02.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.527+0000 7f4557400640 1 --2- 172.21.15.52:0/1118347546 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f453406ac30 0x7f453406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:02.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.529+0000 7f4557400640 1 --2- 172.21.15.52:0/1118347546 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f453406ac30 0x7f453406d0f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4548034000 tx=0x7f4548007000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:02.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.531+0000 7f454f400640 1 -- 172.21.15.52:0/1118347546 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f4550053aa0 con 0x7f455814bb40 2024-03-12T17:07:02.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.734+0000 7f455e200640 1 -- 172.21.15.52:0/1118347546 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f455815d4e0 con 0x7f455814bb40 2024-03-12T17:07:02.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.735+0000 7f454f400640 1 -- 172.21.15.52:0/1118347546 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7f4550018020 con 0x7f455814bb40 2024-03-12T17:07:02.730 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:07:02.730 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":25,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:06:30.468476+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T17:06:19.218543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T17:06:23.525474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T17:06:24.911574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-12T17:06:28.017172+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.733031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.775178+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.591522+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.820838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.829252+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.582481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:07:02.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.738+0000 7f455e200640 1 -- 172.21.15.52:0/1118347546 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f453406ac30 msgr2=0x7f453406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:02.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.738+0000 7f455e200640 1 --2- 172.21.15.52:0/1118347546 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f453406ac30 0x7f453406d0f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4548034000 tx=0x7f4548007000 comp rx=0 tx=0).stop 2024-03-12T17:07:02.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.738+0000 7f455e200640 1 -- 172.21.15.52:0/1118347546 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f455814bb40 msgr2=0x7f455814d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:02.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.738+0000 7f455e200640 1 --2- 172.21.15.52:0/1118347546 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f455814bb40 0x7f455814d990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f45580b54c0 tx=0x7f455000d890 comp rx=0 tx=0).stop 2024-03-12T17:07:02.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.739+0000 7f455e200640 1 -- 172.21.15.52:0/1118347546 shutdown_connections 2024-03-12T17:07:02.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.739+0000 7f455e200640 1 --2- 172.21.15.52:0/1118347546 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f453406ac30 0x7f453406d0f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:02.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.739+0000 7f455e200640 1 --2- 172.21.15.52:0/1118347546 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4558152170 0x7f455814d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:02.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.739+0000 7f455e200640 1 --2- 172.21.15.52:0/1118347546 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f455814bb40 0x7f455814d990 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:02.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.739+0000 7f455e200640 1 --2- 172.21.15.52:0/1118347546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f453c003970 0x7f455814f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:02.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.739+0000 7f455e200640 1 -- 172.21.15.52:0/1118347546 >> 172.21.15.52:0/1118347546 conn(0x7f45580cf350 msgr2=0x7f45580b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:02.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.739+0000 7f455e200640 1 -- 172.21.15.52:0/1118347546 shutdown_connections 2024-03-12T17:07:02.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.739+0000 7f455e200640 1 -- 172.21.15.52:0/1118347546 wait complete. 2024-03-12T17:07:02.751 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T17:06:19.218543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-12T17:06:23.525474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-12T17:06:24.911574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-12T17:06:28.017172+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-12T17:07:02.752 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T17:07:02.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.984+0000 7f2d64a00640 1 Processor -- start 2024-03-12T17:07:02.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.984+0000 7f2d64a00640 1 -- start start 2024-03-12T17:07:02.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.985+0000 7f2d64a00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2d6015cc30 0x7f2d600d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:02.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.985+0000 7f2d64a00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d60160ae0 con 0x7f2d600d1be0 2024-03-12T17:07:02.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.986+0000 7f2d64a00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d600d0c00 con 0x7f2d600b62a0 2024-03-12T17:07:02.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.986+0000 7f2d64a00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d600cdde0 con 0x7f2d6015cc30 2024-03-12T17:07:02.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.986+0000 7f2d5f400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2d6015cc30 0x7f2d600d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:02.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.986+0000 7f2d5ea00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f2d600d1be0 0x7f2d600b5b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55074/0 (socket says 172.21.15.52:55074) 2024-03-12T17:07:02.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.986+0000 7f2d5f400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2d6015cc30 0x7f2d600d16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:32970/0 (socket says 172.21.15.52:32970) 2024-03-12T17:07:02.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.986+0000 7f2d5ea00640 1 -- 172.21.15.52:0/916706479 learned_addr learned my addr 172.21.15.52:0/916706479 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:02.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.986+0000 7f2d5f400640 1 -- 172.21.15.52:0/916706479 >> v1:172.21.15.102:6789/0 conn(0x7f2d600b62a0 legacy=0x7f2d600cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:02.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.986+0000 7f2d5f400640 1 -- 172.21.15.52:0/916706479 >> v1:172.21.15.52:6789/0 conn(0x7f2d600d1be0 legacy=0x7f2d600b5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:02.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.987+0000 7f2d5e000640 1 -- 172.21.15.52:0/916706479 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2482853541 0 0) 0x7f2d60160ae0 con 0x7f2d600d1be0 2024-03-12T17:07:02.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.987+0000 7f2d5f400640 1 -- 172.21.15.52:0/916706479 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d600d0ea0 con 0x7f2d6015cc30 2024-03-12T17:07:02.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.988+0000 7f2d5f400640 1 --2- 172.21.15.52:0/916706479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d6015cc30 0x7f2d600d16a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2d50009d90 tx=0x7f2d5002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dfa9d4068ebe3bc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:02.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.989+0000 7f2d5e000640 1 -- 172.21.15.52:0/916706479 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5002cc60 con 0x7f2d6015cc30 2024-03-12T17:07:02.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.989+0000 7f2d5e000640 1 -- 172.21.15.52:0/916706479 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d5002cdc0 con 0x7f2d6015cc30 2024-03-12T17:07:02.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.989+0000 7f2d5e000640 1 -- 172.21.15.52:0/916706479 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d50034660 con 0x7f2d6015cc30 2024-03-12T17:07:02.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.989+0000 7f2d64a00640 1 -- 172.21.15.52:0/916706479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d6015cc30 msgr2=0x7f2d600d16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:02.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.989+0000 7f2d64a00640 1 --2- 172.21.15.52:0/916706479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d6015cc30 0x7f2d600d16a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2d50009d90 tx=0x7f2d5002beb0 comp rx=0 tx=0).stop 2024-03-12T17:07:02.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.989+0000 7f2d64a00640 1 -- 172.21.15.52:0/916706479 shutdown_connections 2024-03-12T17:07:02.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.989+0000 7f2d64a00640 1 --2- 172.21.15.52:0/916706479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d6015cc30 0x7f2d600d16a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:02.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.989+0000 7f2d64a00640 1 -- 172.21.15.52:0/916706479 >> 172.21.15.52:0/916706479 conn(0x7f2d6006b100 msgr2=0x7f2d6006b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:02.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.989+0000 7f2d64a00640 1 -- 172.21.15.52:0/916706479 shutdown_connections 2024-03-12T17:07:02.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.990+0000 7f2d64a00640 1 -- 172.21.15.52:0/916706479 wait complete. 2024-03-12T17:07:02.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.991+0000 7f2d64a00640 1 Processor -- start 2024-03-12T17:07:02.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.992+0000 7f2d64a00640 1 -- start start 2024-03-12T17:07:02.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.992+0000 7f2d64a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d600b62a0 0x7f2d600ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:02.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.992+0000 7f2d64a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d600d1be0 0x7f2d600cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:02.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.993+0000 7f2d64a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2d6015cc30 0x7f2d601560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:02.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.993+0000 7f2d64a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d600cacc0 con 0x7f2d600b62a0 2024-03-12T17:07:02.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.993+0000 7f2d64a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d600ca710 con 0x7f2d600d1be0 2024-03-12T17:07:02.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.993+0000 7f2d64a00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d600ca9e0 con 0x7f2d6015cc30 2024-03-12T17:07:02.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.993+0000 7f2d5f400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d600b62a0 0x7f2d600ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:02.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.993+0000 7f2d5f400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d600b62a0 0x7f2d600ca1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:32984/0 (socket says 172.21.15.52:32984) 2024-03-12T17:07:02.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.993+0000 7f2d5f400640 1 -- 172.21.15.52:0/1159129172 learned_addr learned my addr 172.21.15.52:0/1159129172 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:02.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.993+0000 7f2d5ea00640 1 --2- 172.21.15.52:0/1159129172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d600d1be0 0x7f2d600cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:02.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.993+0000 7f2d5f400640 1 -- 172.21.15.52:0/1159129172 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2d6015cc30 msgr2=0x7f2d601560c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:02.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.993+0000 7f2d5f400640 1 --2- 172.21.15.52:0/1159129172 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2d6015cc30 0x7f2d601560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:02.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.993+0000 7f2d5f400640 1 -- 172.21.15.52:0/1159129172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d600d1be0 msgr2=0x7f2d600cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:02.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.993+0000 7f2d5f400640 1 --2- 172.21.15.52:0/1159129172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d600d1be0 0x7f2d600cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:02.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.993+0000 7f2d5f400640 1 -- 172.21.15.52:0/1159129172 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d50009a70 con 0x7f2d600b62a0 2024-03-12T17:07:02.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.994+0000 7f2d5f400640 1 --2- 172.21.15.52:0/1159129172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d600b62a0 0x7f2d600ca1d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2d50037040 tx=0x7f2d50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:02.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.994+0000 7f2d57400640 1 -- 172.21.15.52:0/1159129172 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5003e610 con 0x7f2d600b62a0 2024-03-12T17:07:02.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.994+0000 7f2d57400640 1 -- 172.21.15.52:0/1159129172 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d5003e770 con 0x7f2d600b62a0 2024-03-12T17:07:02.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.994+0000 7f2d57400640 1 -- 172.21.15.52:0/1159129172 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5003eb20 con 0x7f2d600b62a0 2024-03-12T17:07:02.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.994+0000 7f2d64a00640 1 -- 172.21.15.52:0/1159129172 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d6015e3a0 con 0x7f2d600b62a0 2024-03-12T17:07:02.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.994+0000 7f2d64a00640 1 -- 172.21.15.52:0/1159129172 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d6015e960 con 0x7f2d600b62a0 2024-03-12T17:07:02.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.995+0000 7f2d64a00640 1 -- 172.21.15.52:0/1159129172 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d6004e030 con 0x7f2d600b62a0 2024-03-12T17:07:02.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.996+0000 7f2d57400640 1 -- 172.21.15.52:0/1159129172 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2d5003b050 con 0x7f2d600b62a0 2024-03-12T17:07:02.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.997+0000 7f2d57400640 1 --2- 172.21.15.52:0/1159129172 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2d3c06ac30 0x7f2d3c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:02.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.997+0000 7f2d57400640 1 -- 172.21.15.52:0/1159129172 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2d500abd00 con 0x7f2d600b62a0 2024-03-12T17:07:02.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.998+0000 7f2d5ea00640 1 --2- 172.21.15.52:0/1159129172 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2d3c06ac30 0x7f2d3c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:02.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:02.999+0000 7f2d5ea00640 1 --2- 172.21.15.52:0/1159129172 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2d3c06ac30 0x7f2d3c06d0f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2d48007fd0 tx=0x7f2d48009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:02.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.001+0000 7f2d57400640 1 -- 172.21.15.52:0/1159129172 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f2d50076c80 con 0x7f2d600b62a0 2024-03-12T17:07:03.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.201+0000 7f2d64a00640 1 -- 172.21.15.52:0/1159129172 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2d600d0710 con 0x7f2d600b62a0 2024-03-12T17:07:03.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.202+0000 7f2d57400640 1 -- 172.21.15.52:0/1159129172 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2d500390a0 con 0x7f2d600b62a0 2024-03-12T17:07:03.196 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-03-12T17:07:03.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.205+0000 7f2d64a00640 1 -- 172.21.15.52:0/1159129172 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2d3c06ac30 msgr2=0x7f2d3c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:03.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.205+0000 7f2d64a00640 1 --2- 172.21.15.52:0/1159129172 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2d3c06ac30 0x7f2d3c06d0f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2d48007fd0 tx=0x7f2d48009040 comp rx=0 tx=0).stop 2024-03-12T17:07:03.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.205+0000 7f2d64a00640 1 -- 172.21.15.52:0/1159129172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d600b62a0 msgr2=0x7f2d600ca1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:03.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.205+0000 7f2d64a00640 1 --2- 172.21.15.52:0/1159129172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d600b62a0 0x7f2d600ca1d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2d50037040 tx=0x7f2d50004a40 comp rx=0 tx=0).stop 2024-03-12T17:07:03.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.205+0000 7f2d64a00640 1 -- 172.21.15.52:0/1159129172 shutdown_connections 2024-03-12T17:07:03.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.205+0000 7f2d64a00640 1 --2- 172.21.15.52:0/1159129172 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2d3c06ac30 0x7f2d3c06d0f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:03.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.205+0000 7f2d64a00640 1 --2- 172.21.15.52:0/1159129172 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2d6015cc30 0x7f2d601560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:03.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.205+0000 7f2d64a00640 1 --2- 172.21.15.52:0/1159129172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d600d1be0 0x7f2d600cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:03.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.205+0000 7f2d64a00640 1 --2- 172.21.15.52:0/1159129172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d600b62a0 0x7f2d600ca1d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:03.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.205+0000 7f2d64a00640 1 -- 172.21.15.52:0/1159129172 >> 172.21.15.52:0/1159129172 conn(0x7f2d6006b100 msgr2=0x7f2d6006b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:03.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.205+0000 7f2d64a00640 1 -- 172.21.15.52:0/1159129172 shutdown_connections 2024-03-12T17:07:03.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.205+0000 7f2d64a00640 1 -- 172.21.15.52:0/1159129172 wait complete. 2024-03-12T17:07:03.221 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-12T17:07:03.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.432+0000 7f708b200640 1 Processor -- start 2024-03-12T17:07:03.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.433+0000 7f708b200640 1 -- start start 2024-03-12T17:07:03.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.434+0000 7f708b200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f708c160170 0x7f708c0d2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:03.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.434+0000 7f708b200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f708c0d22d0 0x7f708c0cf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:03.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.434+0000 7f708b200640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f708c0c6290 con 0x7f708c0cf600 2024-03-12T17:07:03.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.434+0000 7f708b200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f708c0cfa90 con 0x7f708c0d22d0 2024-03-12T17:07:03.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.434+0000 7f708b200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f708c0cdcd0 con 0x7f708c160170 2024-03-12T17:07:03.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.435+0000 7f708a800640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f708c0cf600 0x7f708c0d0730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55084/0 (socket says 172.21.15.52:55084) 2024-03-12T17:07:03.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.435+0000 7f708a800640 1 -- 172.21.15.52:0/3542335070 learned_addr learned my addr 172.21.15.52:0/3542335070 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:03.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.435+0000 7f7089400640 1 --2- 172.21.15.52:0/3542335070 >> v2:172.21.15.102:3300/0 conn(0x7f708c0d22d0 0x7f708c0cf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:03.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.435+0000 7f7089e00640 1 --2- 172.21.15.52:0/3542335070 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f708c160170 0x7f708c0d2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:03.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.435+0000 7f7088a00640 1 -- 172.21.15.52:0/3542335070 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2191151099 0 0) 0x7f708c0c6290 con 0x7f708c0cf600 2024-03-12T17:07:03.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.435+0000 7f7088a00640 1 -- 172.21.15.52:0/3542335070 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7070003520 con 0x7f708c0cf600 2024-03-12T17:07:03.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.436+0000 7f7088a00640 1 -- 172.21.15.52:0/3542335070 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3757717478 0 0) 0x7f7070003520 con 0x7f708c0cf600 2024-03-12T17:07:03.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.436+0000 7f7088a00640 1 -- 172.21.15.52:0/3542335070 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f708c160170 msgr2=0x7f708c0d2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:03.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.436+0000 7f7088a00640 1 --2- 172.21.15.52:0/3542335070 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f708c160170 0x7f708c0d2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:03.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.436+0000 7f7088a00640 1 -- 172.21.15.52:0/3542335070 >> v2:172.21.15.102:3300/0 conn(0x7f708c0d22d0 msgr2=0x7f708c0cf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:03.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.436+0000 7f7088a00640 1 --2- 172.21.15.52:0/3542335070 >> v2:172.21.15.102:3300/0 conn(0x7f708c0d22d0 0x7f708c0cf0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:03.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.436+0000 7f7088a00640 1 -- 172.21.15.52:0/3542335070 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f708c0b3fd0 con 0x7f708c0cf600 2024-03-12T17:07:03.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.436+0000 7f7088a00640 1 -- 172.21.15.52:0/3542335070 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f70840019f0 con 0x7f708c0cf600 2024-03-12T17:07:03.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.436+0000 7f7088a00640 1 -- 172.21.15.52:0/3542335070 >> v1:172.21.15.52:6789/0 conn(0x7f708c0cf600 legacy=0x7f708c0d0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:03.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.436+0000 7f7088a00640 1 --2- 172.21.15.52:0/3542335070 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f708c160170 0x7f7070003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:03.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.436+0000 7f7088a00640 1 --2- 172.21.15.52:0/3542335070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7070003ef0 0x7f70700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:03.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.436+0000 7f7088a00640 1 --2- 172.21.15.52:0/3542335070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7070004d00 0x7f7070005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:03.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.436+0000 7f7088a00640 1 -- 172.21.15.52:0/3542335070 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708c0cfa90 con 0x7f7070004d00 2024-03-12T17:07:03.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.436+0000 7f7088a00640 1 -- 172.21.15.52:0/3542335070 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708c0cdcd0 con 0x7f7070003ef0 2024-03-12T17:07:03.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.436+0000 7f7088a00640 1 -- 172.21.15.52:0/3542335070 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f707000b9d0 con 0x7f708c160170 2024-03-12T17:07:03.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.436+0000 7f708a800640 1 --2- 172.21.15.52:0/3542335070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7070004d00 0x7f7070005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:03.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.436+0000 7f7089e00640 1 --2- 172.21.15.52:0/3542335070 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f708c160170 0x7f7070003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:03.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.436+0000 7f7089400640 1 --2- 172.21.15.52:0/3542335070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7070003ef0 0x7f70700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:03.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.437+0000 7f708a800640 1 -- 172.21.15.52:0/3542335070 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f708c160170 msgr2=0x7f7070003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:03.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.437+0000 7f708a800640 1 --2- 172.21.15.52:0/3542335070 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f708c160170 0x7f7070003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:03.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.437+0000 7f708a800640 1 -- 172.21.15.52:0/3542335070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7070003ef0 msgr2=0x7f70700047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:03.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.437+0000 7f708a800640 1 --2- 172.21.15.52:0/3542335070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7070003ef0 0x7f70700047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:03.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.437+0000 7f708a800640 1 -- 172.21.15.52:0/3542335070 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7070003520 con 0x7f7070004d00 2024-03-12T17:07:03.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.438+0000 7f708a800640 1 --2- 172.21.15.52:0/3542335070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7070004d00 0x7f7070005180 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f708400a900 tx=0x7f70700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=810ff5c1f9615e5c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:03.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.438+0000 7f7088a00640 1 -- 172.21.15.52:0/3542335070 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7084034630 con 0x7f7070004d00 2024-03-12T17:07:03.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.438+0000 7f7088a00640 1 -- 172.21.15.52:0/3542335070 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f708402f070 con 0x7f7070004d00 2024-03-12T17:07:03.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.438+0000 7f7088a00640 1 -- 172.21.15.52:0/3542335070 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70840349d0 con 0x7f7070004d00 2024-03-12T17:07:03.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.439+0000 7f708b200640 1 -- 172.21.15.52:0/3542335070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7070004d00 msgr2=0x7f7070005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:03.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.439+0000 7f708b200640 1 --2- 172.21.15.52:0/3542335070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7070004d00 0x7f7070005180 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f708400a900 tx=0x7f70700057f0 comp rx=0 tx=0).stop 2024-03-12T17:07:03.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.439+0000 7f708a800640 1 -- 172.21.15.52:0/3542335070 reap_dead start 2024-03-12T17:07:03.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.439+0000 7f708b200640 1 -- 172.21.15.52:0/3542335070 shutdown_connections 2024-03-12T17:07:03.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.439+0000 7f708b200640 1 -- 172.21.15.52:0/3542335070 >> 172.21.15.52:0/3542335070 conn(0x7f708c089fd0 msgr2=0x7f708c0d26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:03.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.439+0000 7f708b200640 1 -- 172.21.15.52:0/3542335070 shutdown_connections 2024-03-12T17:07:03.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.439+0000 7f708b200640 1 -- 172.21.15.52:0/3542335070 wait complete. 2024-03-12T17:07:03.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.441+0000 7f708b200640 1 Processor -- start 2024-03-12T17:07:03.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.441+0000 7f708b200640 1 -- start start 2024-03-12T17:07:03.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.442+0000 7f708b200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7070004d00 0x7f708c09fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:03.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.442+0000 7f708b200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f708c0b4280 0x7f708c0c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:03.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.442+0000 7f708b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f708c15b520 0x7f708c15b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:03.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.442+0000 7f708b200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708c0b5a20 con 0x7f708c0b4280 2024-03-12T17:07:03.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.442+0000 7f708b200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708c15bef0 con 0x7f708c15b520 2024-03-12T17:07:03.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.442+0000 7f708b200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f708c0b7fe0 con 0x7f7070004d00 2024-03-12T17:07:03.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.442+0000 7f7089400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f708c0b4280 0x7f708c0c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:03.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.442+0000 7f7089400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f708c0b4280 0x7f708c0c7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33000/0 (socket says 172.21.15.52:33000) 2024-03-12T17:07:03.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.442+0000 7f7089400640 1 -- 172.21.15.52:0/3341561329 learned_addr learned my addr 172.21.15.52:0/3341561329 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:03.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.443+0000 7f7089e00640 1 --2- 172.21.15.52:0/3341561329 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7070004d00 0x7f708c09fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:03.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.443+0000 7f708a800640 1 --2- 172.21.15.52:0/3341561329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f708c15b520 0x7f708c15b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:03.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.443+0000 7f7089400640 1 -- 172.21.15.52:0/3341561329 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7070004d00 msgr2=0x7f708c09fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:03.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.443+0000 7f7089400640 1 --2- 172.21.15.52:0/3341561329 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7070004d00 0x7f708c09fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:03.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.443+0000 7f7089400640 1 -- 172.21.15.52:0/3341561329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f708c15b520 msgr2=0x7f708c15b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:03.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.443+0000 7f7089400640 1 --2- 172.21.15.52:0/3341561329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f708c15b520 0x7f708c15b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:03.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.443+0000 7f7089400640 1 -- 172.21.15.52:0/3341561329 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f708c0b7220 con 0x7f708c0b4280 2024-03-12T17:07:03.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.443+0000 7f708a800640 1 --2- 172.21.15.52:0/3341561329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f708c15b520 0x7f708c15b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:03.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.443+0000 7f7089400640 1 --2- 172.21.15.52:0/3341561329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f708c0b4280 0x7f708c0c7d30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7078011600 tx=0x7f7078011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:03.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.443+0000 7f7082000640 1 -- 172.21.15.52:0/3341561329 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707800f8c0 con 0x7f708c0b4280 2024-03-12T17:07:03.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.443+0000 7f7082000640 1 -- 172.21.15.52:0/3341561329 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707800fa20 con 0x7f708c0b4280 2024-03-12T17:07:03.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.443+0000 7f708b200640 1 -- 172.21.15.52:0/3341561329 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f708c0b7500 con 0x7f708c0b4280 2024-03-12T17:07:03.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.443+0000 7f7082000640 1 -- 172.21.15.52:0/3341561329 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707800fd90 con 0x7f708c0b4280 2024-03-12T17:07:03.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.444+0000 7f708b200640 1 -- 172.21.15.52:0/3341561329 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f708c0b7ac0 con 0x7f708c0b4280 2024-03-12T17:07:03.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.445+0000 7f708b200640 1 -- 172.21.15.52:0/3341561329 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f708c04e030 con 0x7f708c0b4280 2024-03-12T17:07:03.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.446+0000 7f7082000640 1 -- 172.21.15.52:0/3341561329 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7078015460 con 0x7f708c0b4280 2024-03-12T17:07:03.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.446+0000 7f7082000640 1 --2- 172.21.15.52:0/3341561329 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f706c06ac00 0x7f706c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:03.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.446+0000 7f7082000640 1 -- 172.21.15.52:0/3341561329 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7078089b80 con 0x7f708c0b4280 2024-03-12T17:07:03.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.447+0000 7f7089e00640 1 --2- 172.21.15.52:0/3341561329 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f706c06ac00 0x7f706c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:03.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.448+0000 7f7089e00640 1 --2- 172.21.15.52:0/3341561329 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f706c06ac00 0x7f706c06d0c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f707c009950 tx=0x7f707c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:03.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.451+0000 7f7082000640 1 -- 172.21.15.52:0/3341561329 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f70780549d0 con 0x7f708c0b4280 2024-03-12T17:07:03.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.654+0000 7f708b200640 1 -- 172.21.15.52:0/3341561329 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f708c0ceea0 con 0x7f708c0b4280 2024-03-12T17:07:03.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.656+0000 7f7082000640 1 -- 172.21.15.52:0/3341561329 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7078057ff0 con 0x7f708c0b4280 2024-03-12T17:07:03.649 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-12T17:07:03.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.659+0000 7f708b200640 1 -- 172.21.15.52:0/3341561329 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f706c06ac00 msgr2=0x7f706c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:03.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.659+0000 7f708b200640 1 --2- 172.21.15.52:0/3341561329 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f706c06ac00 0x7f706c06d0c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f707c009950 tx=0x7f707c008040 comp rx=0 tx=0).stop 2024-03-12T17:07:03.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.659+0000 7f708b200640 1 -- 172.21.15.52:0/3341561329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f708c0b4280 msgr2=0x7f708c0c7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:03.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.659+0000 7f708b200640 1 --2- 172.21.15.52:0/3341561329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f708c0b4280 0x7f708c0c7d30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7078011600 tx=0x7f7078011ac0 comp rx=0 tx=0).stop 2024-03-12T17:07:03.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.659+0000 7f708b200640 1 -- 172.21.15.52:0/3341561329 shutdown_connections 2024-03-12T17:07:03.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.659+0000 7f708b200640 1 --2- 172.21.15.52:0/3341561329 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f706c06ac00 0x7f706c06d0c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:03.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.659+0000 7f708b200640 1 --2- 172.21.15.52:0/3341561329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f708c15b520 0x7f708c15b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:03.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.659+0000 7f708b200640 1 --2- 172.21.15.52:0/3341561329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f708c0b4280 0x7f708c0c7d30 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:03.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.659+0000 7f708b200640 1 --2- 172.21.15.52:0/3341561329 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7070004d00 0x7f708c09fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:03.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.659+0000 7f708b200640 1 -- 172.21.15.52:0/3341561329 >> 172.21.15.52:0/3341561329 conn(0x7f708c089fd0 msgr2=0x7f708c0c87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:03.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.659+0000 7f708b200640 1 -- 172.21.15.52:0/3341561329 shutdown_connections 2024-03-12T17:07:03.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.659+0000 7f708b200640 1 -- 172.21.15.52:0/3341561329 wait complete. 2024-03-12T17:07:03.670 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-12T17:07:03.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.889+0000 7f2716c00640 1 Processor -- start 2024-03-12T17:07:03.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.889+0000 7f2716c00640 1 -- start start 2024-03-12T17:07:03.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.890+0000 7f2716c00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f271015a850 0x7f27100cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:03.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.890+0000 7f2716c00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f27100cacf0 0x7f27100cb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:03.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.890+0000 7f2716c00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f271015e320 con 0x7f27100b22b0 2024-03-12T17:07:03.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.891+0000 7f2716c00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27100cb6c0 con 0x7f271015a850 2024-03-12T17:07:03.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.891+0000 7f2716c00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f271015ac30 con 0x7f27100cacf0 2024-03-12T17:07:03.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.891+0000 7f2715800640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f271015a850 0x7f27100cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:03.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.891+0000 7f2714e00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f27100b22b0 0x7f271015eae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55096/0 (socket says 172.21.15.52:55096) 2024-03-12T17:07:03.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.891+0000 7f2714e00640 1 -- 172.21.15.52:0/1657730745 learned_addr learned my addr 172.21.15.52:0/1657730745 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:03.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.891+0000 7f2716200640 1 --2- 172.21.15.52:0/1657730745 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f27100cacf0 0x7f27100cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:03.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.891+0000 7f270fe00640 1 -- 172.21.15.52:0/1657730745 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3137126116 0 0) 0x7f271015e320 con 0x7f27100b22b0 2024-03-12T17:07:03.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.891+0000 7f270fe00640 1 -- 172.21.15.52:0/1657730745 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26f8003520 con 0x7f27100b22b0 2024-03-12T17:07:03.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.891+0000 7f2715800640 1 -- 172.21.15.52:0/1657730745 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f27100cacf0 msgr2=0x7f27100cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:03.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.891+0000 7f2715800640 1 --2- 172.21.15.52:0/1657730745 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f27100cacf0 0x7f27100cb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:03.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.892+0000 7f2715800640 1 -- 172.21.15.52:0/1657730745 >> v1:172.21.15.52:6789/0 conn(0x7f27100b22b0 legacy=0x7f271015eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:03.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.892+0000 7f2715800640 1 -- 172.21.15.52:0/1657730745 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27100cda20 con 0x7f271015a850 2024-03-12T17:07:03.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.892+0000 7f2716200640 1 --2- 172.21.15.52:0/1657730745 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f27100cacf0 0x7f27100cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:07:03.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.893+0000 7f2715800640 1 --2- 172.21.15.52:0/1657730745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f271015a850 0x7f27100cd1d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2704009d90 tx=0x7f270402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=172540dec107e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:03.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.893+0000 7f270fe00640 1 -- 172.21.15.52:0/1657730745 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270402cc50 con 0x7f271015a850 2024-03-12T17:07:03.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.893+0000 7f270fe00640 1 -- 172.21.15.52:0/1657730745 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f270402cdb0 con 0x7f271015a850 2024-03-12T17:07:03.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.893+0000 7f270fe00640 1 -- 172.21.15.52:0/1657730745 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2704034660 con 0x7f271015a850 2024-03-12T17:07:03.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.894+0000 7f2716c00640 1 -- 172.21.15.52:0/1657730745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f271015a850 msgr2=0x7f27100cd1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:03.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.894+0000 7f2716c00640 1 --2- 172.21.15.52:0/1657730745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f271015a850 0x7f27100cd1d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2704009d90 tx=0x7f270402bea0 comp rx=0 tx=0).stop 2024-03-12T17:07:03.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.894+0000 7f2716c00640 1 -- 172.21.15.52:0/1657730745 shutdown_connections 2024-03-12T17:07:03.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.894+0000 7f2716c00640 1 --2- 172.21.15.52:0/1657730745 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f27100cacf0 0x7f27100cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:03.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.894+0000 7f2716c00640 1 --2- 172.21.15.52:0/1657730745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f271015a850 0x7f27100cd1d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:03.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.894+0000 7f2716c00640 1 -- 172.21.15.52:0/1657730745 >> 172.21.15.52:0/1657730745 conn(0x7f27100b01b0 msgr2=0x7f27100b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:03.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.894+0000 7f2716c00640 1 -- 172.21.15.52:0/1657730745 shutdown_connections 2024-03-12T17:07:03.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.894+0000 7f2716c00640 1 -- 172.21.15.52:0/1657730745 wait complete. 2024-03-12T17:07:03.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.896+0000 7f2716c00640 1 Processor -- start 2024-03-12T17:07:03.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.896+0000 7f2716c00640 1 -- start start 2024-03-12T17:07:03.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.897+0000 7f2716c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27100b22b0 0x7f27100c9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:03.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.897+0000 7f2716c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27100cacf0 0x7f2710151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:03.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.897+0000 7f2716c00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f271015a850 0x7f27100c7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:03.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.897+0000 7f2716c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27101579a0 con 0x7f27100cacf0 2024-03-12T17:07:03.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.897+0000 7f2716c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2710151d70 con 0x7f27100b22b0 2024-03-12T17:07:03.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.897+0000 7f2716c00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27101521e0 con 0x7f271015a850 2024-03-12T17:07:03.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.897+0000 7f2714e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27100cacf0 0x7f2710151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:03.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.897+0000 7f2714e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27100cacf0 0x7f2710151830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33026/0 (socket says 172.21.15.52:33026) 2024-03-12T17:07:03.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.897+0000 7f2714e00640 1 -- 172.21.15.52:0/3004544836 learned_addr learned my addr 172.21.15.52:0/3004544836 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:03.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.898+0000 7f2716200640 1 --2- 172.21.15.52:0/3004544836 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f271015a850 0x7f27100c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:03.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.898+0000 7f2715800640 1 --2- 172.21.15.52:0/3004544836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27100b22b0 0x7f27100c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:03.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.898+0000 7f2714e00640 1 -- 172.21.15.52:0/3004544836 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f271015a850 msgr2=0x7f27100c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:03.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.898+0000 7f2714e00640 1 --2- 172.21.15.52:0/3004544836 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f271015a850 0x7f27100c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:03.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.898+0000 7f2714e00640 1 -- 172.21.15.52:0/3004544836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27100b22b0 msgr2=0x7f27100c9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:03.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.898+0000 7f2714e00640 1 --2- 172.21.15.52:0/3004544836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27100b22b0 0x7f27100c9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:03.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.898+0000 7f2714e00640 1 -- 172.21.15.52:0/3004544836 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2704009a70 con 0x7f27100cacf0 2024-03-12T17:07:03.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.898+0000 7f2714e00640 1 --2- 172.21.15.52:0/3004544836 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27100cacf0 0x7f2710151830 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f26fc00c500 tx=0x7f26fc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:03.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.898+0000 7f270d600640 1 -- 172.21.15.52:0/3004544836 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26fc005ce0 con 0x7f27100cacf0 2024-03-12T17:07:03.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.899+0000 7f270d600640 1 -- 172.21.15.52:0/3004544836 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26fc005e40 con 0x7f27100cacf0 2024-03-12T17:07:03.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.899+0000 7f270d600640 1 -- 172.21.15.52:0/3004544836 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26fc007330 con 0x7f27100cacf0 2024-03-12T17:07:03.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.899+0000 7f2716c00640 1 -- 172.21.15.52:0/3004544836 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27100c9f70 con 0x7f27100cacf0 2024-03-12T17:07:03.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.899+0000 7f2716c00640 1 -- 172.21.15.52:0/3004544836 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2710157630 con 0x7f27100cacf0 2024-03-12T17:07:03.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.900+0000 7f2716c00640 1 -- 172.21.15.52:0/3004544836 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2710157b10 con 0x7f27100cacf0 2024-03-12T17:07:03.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.901+0000 7f270d600640 1 -- 172.21.15.52:0/3004544836 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f26fc007550 con 0x7f27100cacf0 2024-03-12T17:07:03.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.902+0000 7f270d600640 1 --2- 172.21.15.52:0/3004544836 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f26f006ac00 0x7f26f006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:03.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.902+0000 7f270d600640 1 -- 172.21.15.52:0/3004544836 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f26fc088d30 con 0x7f27100cacf0 2024-03-12T17:07:03.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.902+0000 7f2715800640 1 --2- 172.21.15.52:0/3004544836 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f26f006ac00 0x7f26f006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:03.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.904+0000 7f2715800640 1 --2- 172.21.15.52:0/3004544836 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f26f006ac00 0x7f26f006d0c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2704036000 tx=0x7f2704030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:03.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:03.907+0000 7f270d600640 1 -- 172.21.15.52:0/3004544836 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f26fc053cd0 con 0x7f27100cacf0 2024-03-12T17:07:04.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.108+0000 7f2716c00640 1 -- 172.21.15.52:0/3004544836 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f271004e030 con 0x7f27100cacf0 2024-03-12T17:07:04.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.109+0000 7f270d600640 1 -- 172.21.15.52:0/3004544836 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f26fc017020 con 0x7f27100cacf0 2024-03-12T17:07:04.103 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-03-12T17:07:04.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.112+0000 7f2716c00640 1 -- 172.21.15.52:0/3004544836 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f26f006ac00 msgr2=0x7f26f006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:04.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.112+0000 7f2716c00640 1 --2- 172.21.15.52:0/3004544836 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f26f006ac00 0x7f26f006d0c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2704036000 tx=0x7f2704030000 comp rx=0 tx=0).stop 2024-03-12T17:07:04.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.112+0000 7f2716c00640 1 -- 172.21.15.52:0/3004544836 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27100cacf0 msgr2=0x7f2710151830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:04.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.112+0000 7f2716c00640 1 --2- 172.21.15.52:0/3004544836 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27100cacf0 0x7f2710151830 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f26fc00c500 tx=0x7f26fc00c9c0 comp rx=0 tx=0).stop 2024-03-12T17:07:04.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.112+0000 7f2716c00640 1 -- 172.21.15.52:0/3004544836 shutdown_connections 2024-03-12T17:07:04.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.112+0000 7f2716c00640 1 --2- 172.21.15.52:0/3004544836 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f26f006ac00 0x7f26f006d0c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:04.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.112+0000 7f2716c00640 1 --2- 172.21.15.52:0/3004544836 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f271015a850 0x7f27100c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:04.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.112+0000 7f2716c00640 1 --2- 172.21.15.52:0/3004544836 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27100cacf0 0x7f2710151830 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:04.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.112+0000 7f2716c00640 1 --2- 172.21.15.52:0/3004544836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27100b22b0 0x7f27100c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:04.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.112+0000 7f2716c00640 1 -- 172.21.15.52:0/3004544836 >> 172.21.15.52:0/3004544836 conn(0x7f27100b01b0 msgr2=0x7f27100aefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:04.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.112+0000 7f2716c00640 1 -- 172.21.15.52:0/3004544836 shutdown_connections 2024-03-12T17:07:04.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.112+0000 7f2716c00640 1 -- 172.21.15.52:0/3004544836 wait complete. 2024-03-12T17:07:04.125 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-12T17:07:04.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.341+0000 7f270ae00640 1 Processor -- start 2024-03-12T17:07:04.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.341+0000 7f270ae00640 1 -- start start 2024-03-12T17:07:04.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.343+0000 7f270ae00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f27041456a0 0x7f27040ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:04.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.343+0000 7f270ae00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2704145c20 0x7f27040cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:04.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.343+0000 7f270ae00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f270406d390 con 0x7f2704070cd0 2024-03-12T17:07:04.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.343+0000 7f270ae00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27040ad890 con 0x7f2704145c20 2024-03-12T17:07:04.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.343+0000 7f270ae00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2704145a80 con 0x7f27041456a0 2024-03-12T17:07:04.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.343+0000 7f2703400640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f2704070cd0 0x7f27040ab8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55104/0 (socket says 172.21.15.52:55104) 2024-03-12T17:07:04.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.343+0000 7f2703400640 1 -- 172.21.15.52:0/3353395016 learned_addr learned my addr 172.21.15.52:0/3353395016 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:04.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.344+0000 7f2703e00640 1 --2- 172.21.15.52:0/3353395016 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f27041456a0 0x7f27040ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:04.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.344+0000 7f2708e00640 1 --2- 172.21.15.52:0/3353395016 >> v2:172.21.15.102:3300/0 conn(0x7f2704145c20 0x7f27040cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:04.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.344+0000 7f2702a00640 1 -- 172.21.15.52:0/3353395016 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4072555275 0 0) 0x7f270406d390 con 0x7f2704070cd0 2024-03-12T17:07:04.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.344+0000 7f2702a00640 1 -- 172.21.15.52:0/3353395016 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26e8003520 con 0x7f2704070cd0 2024-03-12T17:07:04.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.344+0000 7f2702a00640 1 -- 172.21.15.52:0/3353395016 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4203831309 0 0) 0x7f26e8003520 con 0x7f2704070cd0 2024-03-12T17:07:04.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.344+0000 7f2702a00640 1 -- 172.21.15.52:0/3353395016 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f27041456a0 msgr2=0x7f27040ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:04.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.344+0000 7f2702a00640 1 --2- 172.21.15.52:0/3353395016 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f27041456a0 0x7f27040ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:04.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.344+0000 7f2702a00640 1 -- 172.21.15.52:0/3353395016 >> v2:172.21.15.102:3300/0 conn(0x7f2704145c20 msgr2=0x7f27040cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:04.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.344+0000 7f2702a00640 1 --2- 172.21.15.52:0/3353395016 >> v2:172.21.15.102:3300/0 conn(0x7f2704145c20 0x7f27040cee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:04.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.344+0000 7f2702a00640 1 -- 172.21.15.52:0/3353395016 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f270406d670 con 0x7f2704070cd0 2024-03-12T17:07:04.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.344+0000 7f2702a00640 1 -- 172.21.15.52:0/3353395016 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f26f0001670 con 0x7f2704070cd0 2024-03-12T17:07:04.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.344+0000 7f2702a00640 1 -- 172.21.15.52:0/3353395016 >> v1:172.21.15.52:6789/0 conn(0x7f2704070cd0 legacy=0x7f27040ab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:04.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.344+0000 7f2702a00640 1 --2- 172.21.15.52:0/3353395016 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f27041456a0 0x7f26e80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:04.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.344+0000 7f2702a00640 1 --2- 172.21.15.52:0/3353395016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26e8003f20 0x7f26e80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:04.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.344+0000 7f2702a00640 1 --2- 172.21.15.52:0/3353395016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26e8004d00 0x7f26e8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:04.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.344+0000 7f2702a00640 1 -- 172.21.15.52:0/3353395016 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27040ad890 con 0x7f26e8003f20 2024-03-12T17:07:04.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.344+0000 7f2702a00640 1 -- 172.21.15.52:0/3353395016 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2704145a80 con 0x7f26e8004d00 2024-03-12T17:07:04.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.344+0000 7f2702a00640 1 -- 172.21.15.52:0/3353395016 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e800b980 con 0x7f27041456a0 2024-03-12T17:07:04.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.345+0000 7f2703400640 1 --2- 172.21.15.52:0/3353395016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26e8003f20 0x7f26e80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:04.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.345+0000 7f2708e00640 1 --2- 172.21.15.52:0/3353395016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26e8004d00 0x7f26e8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:04.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.345+0000 7f2703e00640 1 --2- 172.21.15.52:0/3353395016 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f27041456a0 0x7f26e80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:04.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.345+0000 7f2703400640 1 -- 172.21.15.52:0/3353395016 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f27041456a0 msgr2=0x7f26e80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:04.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.345+0000 7f2703400640 1 --2- 172.21.15.52:0/3353395016 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f27041456a0 0x7f26e80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:04.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.345+0000 7f2703400640 1 -- 172.21.15.52:0/3353395016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26e8004d00 msgr2=0x7f26e8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:04.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.345+0000 7f2703400640 1 --2- 172.21.15.52:0/3353395016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26e8004d00 0x7f26e8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:04.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.345+0000 7f2703400640 1 -- 172.21.15.52:0/3353395016 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e8003520 con 0x7f26e8003f20 2024-03-12T17:07:04.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.346+0000 7f2703400640 1 --2- 172.21.15.52:0/3353395016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26e8003f20 0x7f26e80047c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f26f000c210 tx=0x7f26e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c3cfccc5f16b58 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:04.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.346+0000 7f2702a00640 1 -- 172.21.15.52:0/3353395016 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f0005ad0 con 0x7f26e8003f20 2024-03-12T17:07:04.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.347+0000 7f2702a00640 1 -- 172.21.15.52:0/3353395016 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f0005c30 con 0x7f26e8003f20 2024-03-12T17:07:04.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.347+0000 7f2702a00640 1 -- 172.21.15.52:0/3353395016 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f002cad0 con 0x7f26e8003f20 2024-03-12T17:07:04.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.347+0000 7f270ae00640 1 -- 172.21.15.52:0/3353395016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26e8003f20 msgr2=0x7f26e80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:04.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.347+0000 7f270ae00640 1 --2- 172.21.15.52:0/3353395016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26e8003f20 0x7f26e80047c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f26f000c210 tx=0x7f26e80057f0 comp rx=0 tx=0).stop 2024-03-12T17:07:04.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.347+0000 7f2708e00640 1 -- 172.21.15.52:0/3353395016 reap_dead start 2024-03-12T17:07:04.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.347+0000 7f270ae00640 1 -- 172.21.15.52:0/3353395016 shutdown_connections 2024-03-12T17:07:04.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.347+0000 7f270ae00640 1 -- 172.21.15.52:0/3353395016 >> 172.21.15.52:0/3353395016 conn(0x7f270406a0b0 msgr2=0x7f270406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:04.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.347+0000 7f270ae00640 1 -- 172.21.15.52:0/3353395016 shutdown_connections 2024-03-12T17:07:04.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.347+0000 7f270ae00640 1 -- 172.21.15.52:0/3353395016 wait complete. 2024-03-12T17:07:04.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.350+0000 7f270ae00640 1 Processor -- start 2024-03-12T17:07:04.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.350+0000 7f270ae00640 1 -- start start 2024-03-12T17:07:04.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.351+0000 7f270ae00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26e8003f20 0x7f27040c35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:04.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.351+0000 7f270ae00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f270415ce20 0x7f2704143a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:04.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.351+0000 7f270ae00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f270415be90 0x7f270415c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:04.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.351+0000 7f270ae00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270415c850 con 0x7f26e8003f20 2024-03-12T17:07:04.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.351+0000 7f270ae00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270415e420 con 0x7f270415be90 2024-03-12T17:07:04.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.351+0000 7f270ae00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f270415d370 con 0x7f270415ce20 2024-03-12T17:07:04.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.351+0000 7f2703e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26e8003f20 0x7f27040c35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:04.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.351+0000 7f2703e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26e8003f20 0x7f27040c35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33038/0 (socket says 172.21.15.52:33038) 2024-03-12T17:07:04.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.351+0000 7f2703e00640 1 -- 172.21.15.52:0/4200145989 learned_addr learned my addr 172.21.15.52:0/4200145989 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:04.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.352+0000 7f2703400640 1 --2- 172.21.15.52:0/4200145989 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f270415ce20 0x7f2704143a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:04.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.352+0000 7f2708e00640 1 --2- 172.21.15.52:0/4200145989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f270415be90 0x7f270415c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:04.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.352+0000 7f2703e00640 1 -- 172.21.15.52:0/4200145989 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f270415ce20 msgr2=0x7f2704143a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:04.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.352+0000 7f2703e00640 1 --2- 172.21.15.52:0/4200145989 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f270415ce20 0x7f2704143a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:04.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.352+0000 7f2703e00640 1 -- 172.21.15.52:0/4200145989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f270415be90 msgr2=0x7f270415c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:04.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.352+0000 7f2703e00640 1 --2- 172.21.15.52:0/4200145989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f270415be90 0x7f270415c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:04.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.352+0000 7f2703e00640 1 -- 172.21.15.52:0/4200145989 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f270415d650 con 0x7f26e8003f20 2024-03-12T17:07:04.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.352+0000 7f2703e00640 1 --2- 172.21.15.52:0/4200145989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26e8003f20 0x7f27040c35a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f26f4011540 tx=0x7f26f4011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:04.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.352+0000 7f26fbe00640 1 -- 172.21.15.52:0/4200145989 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f4015930 con 0x7f26e8003f20 2024-03-12T17:07:04.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.352+0000 7f26fbe00640 1 -- 172.21.15.52:0/4200145989 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f4015a90 con 0x7f26e8003f20 2024-03-12T17:07:04.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.352+0000 7f270ae00640 1 -- 172.21.15.52:0/4200145989 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f270415d930 con 0x7f26e8003f20 2024-03-12T17:07:04.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.352+0000 7f26fbe00640 1 -- 172.21.15.52:0/4200145989 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f4015e60 con 0x7f26e8003f20 2024-03-12T17:07:04.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.353+0000 7f270ae00640 1 -- 172.21.15.52:0/4200145989 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27040c3ae0 con 0x7f26e8003f20 2024-03-12T17:07:04.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.354+0000 7f270ae00640 1 -- 172.21.15.52:0/4200145989 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2704073390 con 0x7f26e8003f20 2024-03-12T17:07:04.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.355+0000 7f26fbe00640 1 -- 172.21.15.52:0/4200145989 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f26f400f460 con 0x7f26e8003f20 2024-03-12T17:07:04.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.356+0000 7f26fbe00640 1 --2- 172.21.15.52:0/4200145989 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f26e406ac30 0x7f26e406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:04.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.356+0000 7f26fbe00640 1 -- 172.21.15.52:0/4200145989 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f26f4089d50 con 0x7f26e8003f20 2024-03-12T17:07:04.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.357+0000 7f2703400640 1 --2- 172.21.15.52:0/4200145989 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f26e406ac30 0x7f26e406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:04.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.358+0000 7f2703400640 1 --2- 172.21.15.52:0/4200145989 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f26e406ac30 0x7f26e406d0f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f26f0034000 tx=0x7f26f0009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:04.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.361+0000 7f26fbe00640 1 -- 172.21.15.52:0/4200145989 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f26f4054c30 con 0x7f26e8003f20 2024-03-12T17:07:04.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.561+0000 7f270ae00640 1 -- 172.21.15.52:0/4200145989 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f270406cbc0 con 0x7f26e8003f20 2024-03-12T17:07:04.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.562+0000 7f26fbe00640 1 -- 172.21.15.52:0/4200145989 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f26f4058250 con 0x7f26e8003f20 2024-03-12T17:07:04.556 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-12T17:07:04.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.565+0000 7f270ae00640 1 -- 172.21.15.52:0/4200145989 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f26e406ac30 msgr2=0x7f26e406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:04.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.565+0000 7f270ae00640 1 --2- 172.21.15.52:0/4200145989 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f26e406ac30 0x7f26e406d0f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f26f0034000 tx=0x7f26f0009030 comp rx=0 tx=0).stop 2024-03-12T17:07:04.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.565+0000 7f270ae00640 1 -- 172.21.15.52:0/4200145989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26e8003f20 msgr2=0x7f27040c35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:04.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.565+0000 7f270ae00640 1 --2- 172.21.15.52:0/4200145989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26e8003f20 0x7f27040c35a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f26f4011540 tx=0x7f26f4011a00 comp rx=0 tx=0).stop 2024-03-12T17:07:04.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.565+0000 7f270ae00640 1 -- 172.21.15.52:0/4200145989 shutdown_connections 2024-03-12T17:07:04.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.565+0000 7f270ae00640 1 --2- 172.21.15.52:0/4200145989 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f26e406ac30 0x7f26e406d0f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:04.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.565+0000 7f270ae00640 1 --2- 172.21.15.52:0/4200145989 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f270415ce20 0x7f2704143a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:04.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.565+0000 7f270ae00640 1 --2- 172.21.15.52:0/4200145989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f270415be90 0x7f270415c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:04.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.565+0000 7f270ae00640 1 --2- 172.21.15.52:0/4200145989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26e8003f20 0x7f27040c35a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:04.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.565+0000 7f270ae00640 1 -- 172.21.15.52:0/4200145989 >> 172.21.15.52:0/4200145989 conn(0x7f270406a0b0 msgr2=0x7f27040acde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:04.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.565+0000 7f270ae00640 1 -- 172.21.15.52:0/4200145989 shutdown_connections 2024-03-12T17:07:04.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.565+0000 7f270ae00640 1 -- 172.21.15.52:0/4200145989 wait complete. 2024-03-12T17:07:04.580 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T17:07:04.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.833+0000 7f38d4800640 1 Processor -- start 2024-03-12T17:07:04.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.834+0000 7f38d4800640 1 -- start start 2024-03-12T17:07:04.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.835+0000 7f38d4800640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f38cc15b560 0x7f38cc15b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:04.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.835+0000 7f38d4800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f38cc0b39b0 0x7f38cc0b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:04.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.835+0000 7f38d4800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38cc06c6f0 con 0x7f38cc15e500 2024-03-12T17:07:04.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.835+0000 7f38d4800640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38cc0b4370 con 0x7f38cc15b560 2024-03-12T17:07:04.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.835+0000 7f38d4800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38cc06c9d0 con 0x7f38cc0b39b0 2024-03-12T17:07:04.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.836+0000 7f38d1e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f38cc15b560 0x7f38cc15b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:04.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.836+0000 7f38d1e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f38cc15b560 0x7f38cc15b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33048/0 (socket says 172.21.15.52:33048) 2024-03-12T17:07:04.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.836+0000 7f38d1e00640 1 -- 172.21.15.52:0/2400878760 learned_addr learned my addr 172.21.15.52:0/2400878760 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:04.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.836+0000 7f38d1400640 1 --2- 172.21.15.52:0/2400878760 >> v2:172.21.15.102:3300/0 conn(0x7f38cc0b39b0 0x7f38cc0b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:04.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.836+0000 7f38d1e00640 1 -- 172.21.15.52:0/2400878760 >> v2:172.21.15.102:3300/0 conn(0x7f38cc0b39b0 msgr2=0x7f38cc0b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:04.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.836+0000 7f38d1e00640 1 --2- 172.21.15.52:0/2400878760 >> v2:172.21.15.102:3300/0 conn(0x7f38cc0b39b0 0x7f38cc0b3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:04.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.836+0000 7f38d1e00640 1 -- 172.21.15.52:0/2400878760 >> v1:172.21.15.102:6789/0 conn(0x7f38cc15e500 legacy=0x7f38cc15e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:04.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.836+0000 7f38d1e00640 1 -- 172.21.15.52:0/2400878760 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38cc06ccb0 con 0x7f38cc15b560 2024-03-12T17:07:04.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.838+0000 7f38d1e00640 1 --2- 172.21.15.52:0/2400878760 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38cc15b560 0x7f38cc15b940 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f38cc0ce180 tx=0x7f38b802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5fe62bbc8128ccf server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:04.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.838+0000 7f38d0a00640 1 -- 172.21.15.52:0/2400878760 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b8030860 con 0x7f38cc15b560 2024-03-12T17:07:04.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.838+0000 7f38d0a00640 1 -- 172.21.15.52:0/2400878760 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b80309c0 con 0x7f38cc15b560 2024-03-12T17:07:04.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.838+0000 7f38d0a00640 1 -- 172.21.15.52:0/2400878760 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b8030d50 con 0x7f38cc15b560 2024-03-12T17:07:04.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.839+0000 7f38d4800640 1 -- 172.21.15.52:0/2400878760 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38cc15b560 msgr2=0x7f38cc15b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:04.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.839+0000 7f38d4800640 1 --2- 172.21.15.52:0/2400878760 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38cc15b560 0x7f38cc15b940 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f38cc0ce180 tx=0x7f38b802b9a0 comp rx=0 tx=0).stop 2024-03-12T17:07:04.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.839+0000 7f38d4800640 1 -- 172.21.15.52:0/2400878760 shutdown_connections 2024-03-12T17:07:04.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.839+0000 7f38d4800640 1 --2- 172.21.15.52:0/2400878760 >> v2:172.21.15.102:3300/0 conn(0x7f38cc0b39b0 0x7f38cc0b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:04.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.839+0000 7f38d4800640 1 --2- 172.21.15.52:0/2400878760 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38cc15b560 0x7f38cc15b940 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:04.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.839+0000 7f38d4800640 1 -- 172.21.15.52:0/2400878760 >> 172.21.15.52:0/2400878760 conn(0x7f38cc06a3d0 msgr2=0x7f38cc0b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:04.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.839+0000 7f38d4800640 1 -- 172.21.15.52:0/2400878760 shutdown_connections 2024-03-12T17:07:04.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.839+0000 7f38d4800640 1 -- 172.21.15.52:0/2400878760 wait complete. 2024-03-12T17:07:04.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.841+0000 7f38d4800640 1 Processor -- start 2024-03-12T17:07:04.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.841+0000 7f38d4800640 1 -- start start 2024-03-12T17:07:04.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.842+0000 7f38d4800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38cc0b39b0 0x7f38cc13d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:04.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.842+0000 7f38d4800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f38cc15b560 0x7f38cc142d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:04.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.842+0000 7f38d4800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38cc15e500 0x7f38cc143290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:04.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.842+0000 7f38d4800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc1413d0 con 0x7f38cc15e500 2024-03-12T17:07:04.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.842+0000 7f38d4800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc1416b0 con 0x7f38cc0b39b0 2024-03-12T17:07:04.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.842+0000 7f38d4800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc141990 con 0x7f38cc15b560 2024-03-12T17:07:04.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.843+0000 7f38d2800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38cc15e500 0x7f38cc143290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:04.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.843+0000 7f38d2800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38cc15e500 0x7f38cc143290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33052/0 (socket says 172.21.15.52:33052) 2024-03-12T17:07:04.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.843+0000 7f38d2800640 1 -- 172.21.15.52:0/907392042 learned_addr learned my addr 172.21.15.52:0/907392042 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:04.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.843+0000 7f38d1400640 1 --2- 172.21.15.52:0/907392042 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f38cc15b560 0x7f38cc142d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:04.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.843+0000 7f38d1e00640 1 --2- 172.21.15.52:0/907392042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38cc0b39b0 0x7f38cc13d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:04.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.843+0000 7f38d2800640 1 -- 172.21.15.52:0/907392042 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f38cc15b560 msgr2=0x7f38cc142d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:04.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.843+0000 7f38d2800640 1 --2- 172.21.15.52:0/907392042 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f38cc15b560 0x7f38cc142d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:04.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.843+0000 7f38d2800640 1 -- 172.21.15.52:0/907392042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38cc0b39b0 msgr2=0x7f38cc13d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:04.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.843+0000 7f38d2800640 1 --2- 172.21.15.52:0/907392042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38cc0b39b0 0x7f38cc13d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:04.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.843+0000 7f38d2800640 1 -- 172.21.15.52:0/907392042 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b80094d0 con 0x7f38cc15e500 2024-03-12T17:07:04.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.843+0000 7f38d2800640 1 --2- 172.21.15.52:0/907392042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38cc15e500 0x7f38cc143290 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f38c800c6b0 tx=0x7f38c800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:04.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.844+0000 7f38c6000640 1 -- 172.21.15.52:0/907392042 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c8003b70 con 0x7f38cc15e500 2024-03-12T17:07:04.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.844+0000 7f38c6000640 1 -- 172.21.15.52:0/907392042 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c8003cf0 con 0x7f38cc15e500 2024-03-12T17:07:04.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.844+0000 7f38c6000640 1 -- 172.21.15.52:0/907392042 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c8003ea0 con 0x7f38cc15e500 2024-03-12T17:07:04.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.844+0000 7f38d4800640 1 -- 172.21.15.52:0/907392042 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38cc141c70 con 0x7f38cc15e500 2024-03-12T17:07:04.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.844+0000 7f38d4800640 1 -- 172.21.15.52:0/907392042 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38cc141010 con 0x7f38cc15e500 2024-03-12T17:07:04.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.845+0000 7f38d4800640 1 -- 172.21.15.52:0/907392042 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38cc142190 con 0x7f38cc15e500 2024-03-12T17:07:04.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.846+0000 7f38c6000640 1 -- 172.21.15.52:0/907392042 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f38c8005310 con 0x7f38cc15e500 2024-03-12T17:07:04.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.847+0000 7f38c6000640 1 --2- 172.21.15.52:0/907392042 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f38ac06ac00 0x7f38ac06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:04.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.847+0000 7f38c6000640 1 -- 172.21.15.52:0/907392042 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f38c80894b0 con 0x7f38cc15e500 2024-03-12T17:07:04.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.848+0000 7f38d1e00640 1 --2- 172.21.15.52:0/907392042 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f38ac06ac00 0x7f38ac06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:04.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.849+0000 7f38d1e00640 1 --2- 172.21.15.52:0/907392042 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f38ac06ac00 0x7f38ac06d0c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f38b8036000 tx=0x7f38b8008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:04.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:04.852+0000 7f38c6000640 1 -- 172.21.15.52:0/907392042 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f38cc142190 con 0x7f38cc15e500 2024-03-12T17:07:05.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.087+0000 7f38d4800640 1 -- 172.21.15.52:0/907392042 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38cc142190 con 0x7f38cc15e500 2024-03-12T17:07:05.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.089+0000 7f38c6000640 1 -- 172.21.15.52:0/907392042 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4733 (secure 0 0 0) 0x7f38cc142190 con 0x7f38cc15e500 2024-03-12T17:07:05.083 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:07:05.083 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2950099092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2950099092},{"type":"v1","addr":"172.21.15.52:6834","nonce":2950099092}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/846108993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":846108993},{"type":"v1","addr":"172.21.15.52:6835","nonce":846108993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6836/3002500679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3002500679},{"type":"v1","addr":"172.21.15.102:6836","nonce":3002500679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-12T17:06:25.905186+0000","modified":"2024-03-12T17:06:54.075752+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6837/3333451504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6835","nonce":3333451504},{"type":"v1","addr":"172.21.15.102:6837","nonce":3333451504}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T17:07:05.083 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 9 2024-03-12T17:07:05.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.091+0000 7f38d4800640 1 -- 172.21.15.52:0/907392042 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f38ac06ac00 msgr2=0x7f38ac06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:05.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.091+0000 7f38d4800640 1 --2- 172.21.15.52:0/907392042 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f38ac06ac00 0x7f38ac06d0c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f38b8036000 tx=0x7f38b8008030 comp rx=0 tx=0).stop 2024-03-12T17:07:05.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.091+0000 7f38d4800640 1 -- 172.21.15.52:0/907392042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38cc15e500 msgr2=0x7f38cc143290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:05.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.091+0000 7f38d4800640 1 --2- 172.21.15.52:0/907392042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38cc15e500 0x7f38cc143290 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f38c800c6b0 tx=0x7f38c800cb70 comp rx=0 tx=0).stop 2024-03-12T17:07:05.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.091+0000 7f38d4800640 1 -- 172.21.15.52:0/907392042 shutdown_connections 2024-03-12T17:07:05.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.091+0000 7f38d4800640 1 --2- 172.21.15.52:0/907392042 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f38ac06ac00 0x7f38ac06d0c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:05.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.091+0000 7f38d4800640 1 --2- 172.21.15.52:0/907392042 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f38cc15b560 0x7f38cc142d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:05.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.091+0000 7f38d4800640 1 --2- 172.21.15.52:0/907392042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38cc15e500 0x7f38cc143290 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:05.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.091+0000 7f38d4800640 1 --2- 172.21.15.52:0/907392042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38cc0b39b0 0x7f38cc13d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:05.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.091+0000 7f38d4800640 1 -- 172.21.15.52:0/907392042 >> 172.21.15.52:0/907392042 conn(0x7f38cc06a3d0 msgr2=0x7f38cc0b3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:05.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.092+0000 7f38d4800640 1 -- 172.21.15.52:0/907392042 shutdown_connections 2024-03-12T17:07:05.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.092+0000 7f38d4800640 1 -- 172.21.15.52:0/907392042 wait complete. 2024-03-12T17:07:05.105 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:07:05.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.314+0000 7f9355800640 1 Processor -- start 2024-03-12T17:07:05.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.314+0000 7f9355800640 1 -- start start 2024-03-12T17:07:05.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.315+0000 7f9355800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9350143930 0x7f93500cd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:05.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.315+0000 7f9355800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f93500cdc50 0x7f9350070020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:05.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.315+0000 7f9355800640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f935015b2b0 con 0x7f9350143460 2024-03-12T17:07:05.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.315+0000 7f9355800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93500cc6b0 con 0x7f9350143930 2024-03-12T17:07:05.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.315+0000 7f9355800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93500cc950 con 0x7f93500cdc50 2024-03-12T17:07:05.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.316+0000 7f934fe00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f9350143460 0x7f935015aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:33216/0 (socket says 172.21.15.52:33216) 2024-03-12T17:07:05.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.316+0000 7f934fe00640 1 -- 172.21.15.52:0/3928567102 learned_addr learned my addr 172.21.15.52:0/3928567102 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:05.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.316+0000 7f934ea00640 1 -- 172.21.15.52:0/3928567102 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2750922171 0 0) 0x7f935015b2b0 con 0x7f9350143460 2024-03-12T17:07:05.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.316+0000 7f934f400640 1 --2- 172.21.15.52:0/3928567102 >> v2:172.21.15.102:3300/0 conn(0x7f9350143930 0x7f93500cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:05.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.316+0000 7f9354e00640 1 --2- 172.21.15.52:0/3928567102 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f93500cdc50 0x7f9350070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:05.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.316+0000 7f934ea00640 1 -- 172.21.15.52:0/3928567102 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9334003520 con 0x7f9350143460 2024-03-12T17:07:05.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.317+0000 7f934ea00640 1 -- 172.21.15.52:0/3928567102 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3375965216 0 0) 0x7f9334003520 con 0x7f9350143460 2024-03-12T17:07:05.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.317+0000 7f934ea00640 1 -- 172.21.15.52:0/3928567102 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f93500cdc50 msgr2=0x7f9350070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:05.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.317+0000 7f934ea00640 1 --2- 172.21.15.52:0/3928567102 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f93500cdc50 0x7f9350070020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:05.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.317+0000 7f934ea00640 1 -- 172.21.15.52:0/3928567102 >> v2:172.21.15.102:3300/0 conn(0x7f9350143930 msgr2=0x7f93500cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:05.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.317+0000 7f934ea00640 1 --2- 172.21.15.52:0/3928567102 >> v2:172.21.15.102:3300/0 conn(0x7f9350143930 0x7f93500cd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:05.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.317+0000 7f934ea00640 1 -- 172.21.15.52:0/3928567102 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93500ccc30 con 0x7f9350143460 2024-03-12T17:07:05.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.317+0000 7f934ea00640 1 -- 172.21.15.52:0/3928567102 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f9340001670 con 0x7f9350143460 2024-03-12T17:07:05.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.317+0000 7f9354e00640 1 --2- 172.21.15.52:0/3928567102 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f93500cdc50 0x7f9350070020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:07:05.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.317+0000 7f934ea00640 1 -- 172.21.15.52:0/3928567102 >> v1:172.21.15.52:6789/0 conn(0x7f9350143460 legacy=0x7f935015aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:05.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.317+0000 7f934ea00640 1 --2- 172.21.15.52:0/3928567102 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9334003970 0x7f9334003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:05.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.317+0000 7f934ea00640 1 --2- 172.21.15.52:0/3928567102 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f93500cdc50 0x7f93340047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:05.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.317+0000 7f934ea00640 1 --2- 172.21.15.52:0/3928567102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9334004d00 0x7f9334005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:05.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.317+0000 7f934ea00640 1 -- 172.21.15.52:0/3928567102 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93500cc6b0 con 0x7f9334003970 2024-03-12T17:07:05.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.317+0000 7f934ea00640 1 -- 172.21.15.52:0/3928567102 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93500cc950 con 0x7f9334004d00 2024-03-12T17:07:05.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.317+0000 7f934ea00640 1 -- 172.21.15.52:0/3928567102 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f933400bac0 con 0x7f93500cdc50 2024-03-12T17:07:05.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.317+0000 7f934fe00640 1 --2- 172.21.15.52:0/3928567102 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9334003970 0x7f9334003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:05.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.317+0000 7f934f400640 1 --2- 172.21.15.52:0/3928567102 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f93500cdc50 0x7f93340047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:05.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.317+0000 7f9354e00640 1 --2- 172.21.15.52:0/3928567102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9334004d00 0x7f9334005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:05.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.317+0000 7f934fe00640 1 -- 172.21.15.52:0/3928567102 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f93500cdc50 msgr2=0x7f93340047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:05.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.317+0000 7f934fe00640 1 --2- 172.21.15.52:0/3928567102 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f93500cdc50 0x7f93340047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:05.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.317+0000 7f934fe00640 1 -- 172.21.15.52:0/3928567102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9334004d00 msgr2=0x7f9334005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:05.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.317+0000 7f934fe00640 1 --2- 172.21.15.52:0/3928567102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9334004d00 0x7f9334005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:05.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.317+0000 7f934fe00640 1 -- 172.21.15.52:0/3928567102 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9334003520 con 0x7f9334003970 2024-03-12T17:07:05.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.318+0000 7f934fe00640 1 --2- 172.21.15.52:0/3928567102 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9334003970 0x7f9334003d90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f934000c110 tx=0x7f93340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6beecc55e598c9d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:05.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.319+0000 7f934ea00640 1 -- 172.21.15.52:0/3928567102 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9340005ad0 con 0x7f9334003970 2024-03-12T17:07:05.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.319+0000 7f934ea00640 1 -- 172.21.15.52:0/3928567102 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9340005c50 con 0x7f9334003970 2024-03-12T17:07:05.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.319+0000 7f934ea00640 1 -- 172.21.15.52:0/3928567102 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934002c830 con 0x7f9334003970 2024-03-12T17:07:05.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.319+0000 7f9355800640 1 -- 172.21.15.52:0/3928567102 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9334003970 msgr2=0x7f9334003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:05.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.319+0000 7f9355800640 1 --2- 172.21.15.52:0/3928567102 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9334003970 0x7f9334003d90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f934000c110 tx=0x7f93340057f0 comp rx=0 tx=0).stop 2024-03-12T17:07:05.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.319+0000 7f9354e00640 1 -- 172.21.15.52:0/3928567102 reap_dead start 2024-03-12T17:07:05.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.319+0000 7f9355800640 1 -- 172.21.15.52:0/3928567102 shutdown_connections 2024-03-12T17:07:05.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.319+0000 7f9355800640 1 -- 172.21.15.52:0/3928567102 >> 172.21.15.52:0/3928567102 conn(0x7f9350065df0 msgr2=0x7f9350069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:05.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.319+0000 7f9355800640 1 -- 172.21.15.52:0/3928567102 shutdown_connections 2024-03-12T17:07:05.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.319+0000 7f9355800640 1 -- 172.21.15.52:0/3928567102 wait complete. 2024-03-12T17:07:05.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.321+0000 7f9355800640 1 Processor -- start 2024-03-12T17:07:05.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.321+0000 7f9355800640 1 -- start start 2024-03-12T17:07:05.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.322+0000 7f9355800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9334003970 0x7f935013f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:05.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.322+0000 7f9355800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9350141470 0x7f935013fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:05.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.323+0000 7f9355800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f935015a2d0 0x7f935006d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:05.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.323+0000 7f9355800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93501403d0 con 0x7f9350141470 2024-03-12T17:07:05.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.323+0000 7f9355800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93501406b0 con 0x7f9334003970 2024-03-12T17:07:05.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.323+0000 7f9355800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9350140990 con 0x7f935015a2d0 2024-03-12T17:07:05.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.323+0000 7f934f400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9350141470 0x7f935013fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:05.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.323+0000 7f934f400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9350141470 0x7f935013fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54110/0 (socket says 172.21.15.52:54110) 2024-03-12T17:07:05.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.323+0000 7f934f400640 1 -- 172.21.15.52:0/3922755536 learned_addr learned my addr 172.21.15.52:0/3922755536 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:05.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.323+0000 7f934fe00640 1 --2- 172.21.15.52:0/3922755536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9334003970 0x7f935013f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:05.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.323+0000 7f9354e00640 1 --2- 172.21.15.52:0/3922755536 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f935015a2d0 0x7f935006d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:05.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.323+0000 7f934f400640 1 -- 172.21.15.52:0/3922755536 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f935015a2d0 msgr2=0x7f935006d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:05.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.323+0000 7f934f400640 1 --2- 172.21.15.52:0/3922755536 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f935015a2d0 0x7f935006d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:05.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.323+0000 7f934f400640 1 -- 172.21.15.52:0/3922755536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9334003970 msgr2=0x7f935013f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:05.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.323+0000 7f934f400640 1 --2- 172.21.15.52:0/3922755536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9334003970 0x7f935013f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:05.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.323+0000 7f934f400640 1 -- 172.21.15.52:0/3922755536 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9350140c70 con 0x7f9350141470 2024-03-12T17:07:05.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.323+0000 7f934fe00640 1 --2- 172.21.15.52:0/3922755536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9334003970 0x7f935013f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:05.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.324+0000 7f934f400640 1 --2- 172.21.15.52:0/3922755536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9350141470 0x7f935013fda0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f933800e7e0 tx=0x7f933800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:05.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.324+0000 7f9347e00640 1 -- 172.21.15.52:0/3922755536 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933800c6d0 con 0x7f9350141470 2024-03-12T17:07:05.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.324+0000 7f9347e00640 1 -- 172.21.15.52:0/3922755536 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933800c830 con 0x7f9350141470 2024-03-12T17:07:05.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.324+0000 7f9347e00640 1 -- 172.21.15.52:0/3922755536 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933800cba0 con 0x7f9350141470 2024-03-12T17:07:05.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.324+0000 7f9355800640 1 -- 172.21.15.52:0/3922755536 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9350140f50 con 0x7f9350141470 2024-03-12T17:07:05.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.324+0000 7f9355800640 1 -- 172.21.15.52:0/3922755536 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f935006d740 con 0x7f9350141470 2024-03-12T17:07:05.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.325+0000 7f9355800640 1 -- 172.21.15.52:0/3922755536 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f935004e030 con 0x7f9350141470 2024-03-12T17:07:05.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.326+0000 7f9347e00640 1 -- 172.21.15.52:0/3922755536 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f933800cd00 con 0x7f9350141470 2024-03-12T17:07:05.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.327+0000 7f9347e00640 1 --2- 172.21.15.52:0/3922755536 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f932c06ac30 0x7f932c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:05.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.327+0000 7f9347e00640 1 -- 172.21.15.52:0/3922755536 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9338089390 con 0x7f9350141470 2024-03-12T17:07:05.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.327+0000 7f934fe00640 1 --2- 172.21.15.52:0/3922755536 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f932c06ac30 0x7f932c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:05.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.328+0000 7f934fe00640 1 --2- 172.21.15.52:0/3922755536 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f932c06ac30 0x7f932c06d0f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9340034000 tx=0x7f9340007000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:05.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.331+0000 7f9347e00640 1 -- 172.21.15.52:0/3922755536 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f93380541e0 con 0x7f9350141470 2024-03-12T17:07:05.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.531+0000 7f9355800640 1 -- 172.21.15.52:0/3922755536 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f935013bf40 con 0x7f9350141470 2024-03-12T17:07:05.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.534+0000 7f9347e00640 1 -- 172.21.15.52:0/3922755536 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7f9338057800 con 0x7f9350141470 2024-03-12T17:07:05.528 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:07:05.528 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":25,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:06:30.468476+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T17:06:19.218543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T17:06:23.525474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T17:06:24.911574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-12T17:06:28.017172+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.733031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.775178+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.591522+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.820838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.829252+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.582481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:07:05.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.537+0000 7f9355800640 1 -- 172.21.15.52:0/3922755536 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f932c06ac30 msgr2=0x7f932c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:05.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.537+0000 7f9355800640 1 --2- 172.21.15.52:0/3922755536 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f932c06ac30 0x7f932c06d0f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9340034000 tx=0x7f9340007000 comp rx=0 tx=0).stop 2024-03-12T17:07:05.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.537+0000 7f9355800640 1 -- 172.21.15.52:0/3922755536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9350141470 msgr2=0x7f935013fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:05.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.537+0000 7f9355800640 1 --2- 172.21.15.52:0/3922755536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9350141470 0x7f935013fda0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f933800e7e0 tx=0x7f933800eca0 comp rx=0 tx=0).stop 2024-03-12T17:07:05.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.537+0000 7f9355800640 1 -- 172.21.15.52:0/3922755536 shutdown_connections 2024-03-12T17:07:05.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.537+0000 7f9355800640 1 --2- 172.21.15.52:0/3922755536 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f932c06ac30 0x7f932c06d0f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:05.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.537+0000 7f9355800640 1 --2- 172.21.15.52:0/3922755536 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f935015a2d0 0x7f935006d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:05.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.537+0000 7f9355800640 1 --2- 172.21.15.52:0/3922755536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9350141470 0x7f935013fda0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:05.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.537+0000 7f9355800640 1 --2- 172.21.15.52:0/3922755536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9334003970 0x7f935013f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:05.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.537+0000 7f9355800640 1 -- 172.21.15.52:0/3922755536 >> 172.21.15.52:0/3922755536 conn(0x7f9350065df0 msgr2=0x7f935015cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:05.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.537+0000 7f9355800640 1 -- 172.21.15.52:0/3922755536 shutdown_connections 2024-03-12T17:07:05.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.537+0000 7f9355800640 1 -- 172.21.15.52:0/3922755536 wait complete. 2024-03-12T17:07:05.551 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T17:06:19.218543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-12T17:06:23.525474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-12T17:06:24.911574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-12T17:06:28.017172+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-12T17:07:05.551 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T17:07:05.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.778+0000 7ff37cc00640 1 Processor -- start 2024-03-12T17:07:05.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.778+0000 7ff37cc00640 1 -- start start 2024-03-12T17:07:05.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.779+0000 7ff37cc00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff37815c820 0x7ff3780cd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:05.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.779+0000 7ff37cc00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff378160e90 0x7ff3780cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:05.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.779+0000 7ff37cc00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3780ccd90 con 0x7ff3780cf5f0 2024-03-12T17:07:05.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.780+0000 7ff37cc00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3780cdd70 con 0x7ff37815c820 2024-03-12T17:07:05.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.780+0000 7ff37cc00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3780cfa60 con 0x7ff378160e90 2024-03-12T17:07:05.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.780+0000 7ff377400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff37815c820 0x7ff3780cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:05.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.780+0000 7ff377400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff37815c820 0x7ff3780cd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54120/0 (socket says 172.21.15.52:54120) 2024-03-12T17:07:05.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.780+0000 7ff377400640 1 -- 172.21.15.52:0/4543357 learned_addr learned my addr 172.21.15.52:0/4543357 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:05.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.780+0000 7ff377400640 1 -- 172.21.15.52:0/4543357 >> v2:172.21.15.102:3300/0 conn(0x7ff378160e90 msgr2=0x7ff3780cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:05.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.780+0000 7ff376a00640 1 --2- 172.21.15.52:0/4543357 >> v2:172.21.15.102:3300/0 conn(0x7ff378160e90 0x7ff3780cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:05.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.780+0000 7ff377400640 1 --2- 172.21.15.52:0/4543357 >> v2:172.21.15.102:3300/0 conn(0x7ff378160e90 0x7ff3780cf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:05.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.780+0000 7ff377400640 1 -- 172.21.15.52:0/4543357 >> v1:172.21.15.102:6789/0 conn(0x7ff3780cf5f0 legacy=0x7ff3780d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:05.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.780+0000 7ff377400640 1 -- 172.21.15.52:0/4543357 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3780b7fc0 con 0x7ff37815c820 2024-03-12T17:07:05.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.780+0000 7ff376a00640 1 --2- 172.21.15.52:0/4543357 >> v2:172.21.15.102:3300/0 conn(0x7ff378160e90 0x7ff3780cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:05.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.782+0000 7ff377400640 1 --2- 172.21.15.52:0/4543357 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff37815c820 0x7ff3780cd830 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff3780ccca0 tx=0x7ff36802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c2bef3dc048521 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:05.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.782+0000 7ff376000640 1 -- 172.21.15.52:0/4543357 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff368031870 con 0x7ff37815c820 2024-03-12T17:07:05.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.782+0000 7ff376000640 1 -- 172.21.15.52:0/4543357 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3680319d0 con 0x7ff37815c820 2024-03-12T17:07:05.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.782+0000 7ff376000640 1 -- 172.21.15.52:0/4543357 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff368031d60 con 0x7ff37815c820 2024-03-12T17:07:05.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.783+0000 7ff37cc00640 1 -- 172.21.15.52:0/4543357 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff37815c820 msgr2=0x7ff3780cd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:05.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.783+0000 7ff37cc00640 1 --2- 172.21.15.52:0/4543357 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff37815c820 0x7ff3780cd830 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff3780ccca0 tx=0x7ff36802b910 comp rx=0 tx=0).stop 2024-03-12T17:07:05.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.783+0000 7ff37cc00640 1 -- 172.21.15.52:0/4543357 shutdown_connections 2024-03-12T17:07:05.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.783+0000 7ff37cc00640 1 --2- 172.21.15.52:0/4543357 >> v2:172.21.15.102:3300/0 conn(0x7ff378160e90 0x7ff3780cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:05.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.783+0000 7ff37cc00640 1 --2- 172.21.15.52:0/4543357 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff37815c820 0x7ff3780cd830 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:05.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.783+0000 7ff37cc00640 1 -- 172.21.15.52:0/4543357 >> 172.21.15.52:0/4543357 conn(0x7ff37806b050 msgr2=0x7ff37806b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:05.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.783+0000 7ff37cc00640 1 -- 172.21.15.52:0/4543357 shutdown_connections 2024-03-12T17:07:05.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.783+0000 7ff37cc00640 1 -- 172.21.15.52:0/4543357 wait complete. 2024-03-12T17:07:05.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.785+0000 7ff37cc00640 1 Processor -- start 2024-03-12T17:07:05.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.786+0000 7ff37cc00640 1 -- start start 2024-03-12T17:07:05.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.787+0000 7ff37cc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3780cf5f0 0x7ff37814f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:05.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.787+0000 7ff37cc00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff37815c820 0x7ff3780b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:05.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.787+0000 7ff37cc00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff378160e90 0x7ff3780b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:05.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.787+0000 7ff37cc00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3780b4e10 con 0x7ff37815c820 2024-03-12T17:07:05.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.787+0000 7ff37cc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3780b2830 con 0x7ff3780cf5f0 2024-03-12T17:07:05.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.787+0000 7ff37cc00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3780c6e70 con 0x7ff378160e90 2024-03-12T17:07:05.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.787+0000 7ff376a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff37815c820 0x7ff3780b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:05.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.787+0000 7ff376a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff37815c820 0x7ff3780b48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54136/0 (socket says 172.21.15.52:54136) 2024-03-12T17:07:05.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.787+0000 7ff376a00640 1 -- 172.21.15.52:0/3633397962 learned_addr learned my addr 172.21.15.52:0/3633397962 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:05.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.788+0000 7ff376a00640 1 -- 172.21.15.52:0/3633397962 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff378160e90 msgr2=0x7ff3780b5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:05.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.788+0000 7ff376a00640 1 --2- 172.21.15.52:0/3633397962 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff378160e90 0x7ff3780b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:05.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.788+0000 7ff376a00640 1 -- 172.21.15.52:0/3633397962 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3780cf5f0 msgr2=0x7ff37814f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:05.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.788+0000 7ff377400640 1 --2- 172.21.15.52:0/3633397962 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3780cf5f0 0x7ff37814f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:05.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.788+0000 7ff376a00640 1 --2- 172.21.15.52:0/3633397962 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3780cf5f0 0x7ff37814f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:05.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.788+0000 7ff376a00640 1 -- 172.21.15.52:0/3633397962 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3680094d0 con 0x7ff37815c820 2024-03-12T17:07:05.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.788+0000 7ff376a00640 1 --2- 172.21.15.52:0/3633397962 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff37815c820 0x7ff3780b48d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff36400c7f0 tx=0x7ff36400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:05.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.788+0000 7ff36f400640 1 -- 172.21.15.52:0/3633397962 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff364007820 con 0x7ff37815c820 2024-03-12T17:07:05.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.789+0000 7ff36f400640 1 -- 172.21.15.52:0/3633397962 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff364007980 con 0x7ff37815c820 2024-03-12T17:07:05.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.789+0000 7ff37cc00640 1 -- 172.21.15.52:0/3633397962 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3780c65d0 con 0x7ff37815c820 2024-03-12T17:07:05.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.789+0000 7ff36f400640 1 -- 172.21.15.52:0/3633397962 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff364007cf0 con 0x7ff37815c820 2024-03-12T17:07:05.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.789+0000 7ff37cc00640 1 -- 172.21.15.52:0/3633397962 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3780c6b90 con 0x7ff37815c820 2024-03-12T17:07:05.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.790+0000 7ff37cc00640 1 -- 172.21.15.52:0/3633397962 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3780b2690 con 0x7ff37815c820 2024-03-12T17:07:05.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.791+0000 7ff36f400640 1 -- 172.21.15.52:0/3633397962 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff3640113f0 con 0x7ff37815c820 2024-03-12T17:07:05.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.792+0000 7ff36f400640 1 --2- 172.21.15.52:0/3633397962 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff35806ac00 0x7ff35806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:05.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.792+0000 7ff36f400640 1 -- 172.21.15.52:0/3633397962 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff364089970 con 0x7ff37815c820 2024-03-12T17:07:05.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.792+0000 7ff377400640 1 --2- 172.21.15.52:0/3633397962 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff35806ac00 0x7ff35806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:05.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.793+0000 7ff377400640 1 --2- 172.21.15.52:0/3633397962 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff35806ac00 0x7ff35806d0c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff368036000 tx=0x7ff36803c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:05.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.796+0000 7ff36f400640 1 -- 172.21.15.52:0/3633397962 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7ff3780b2690 con 0x7ff37815c820 2024-03-12T17:07:05.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.993+0000 7ff37cc00640 1 -- 172.21.15.52:0/3633397962 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff3780cd2f0 con 0x7ff37815c820 2024-03-12T17:07:05.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.994+0000 7ff36f400640 1 -- 172.21.15.52:0/3633397962 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff36401f020 con 0x7ff37815c820 2024-03-12T17:07:05.988 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-03-12T17:07:05.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.996+0000 7ff37cc00640 1 -- 172.21.15.52:0/3633397962 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff35806ac00 msgr2=0x7ff35806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:05.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.996+0000 7ff37cc00640 1 --2- 172.21.15.52:0/3633397962 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff35806ac00 0x7ff35806d0c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff368036000 tx=0x7ff36803c040 comp rx=0 tx=0).stop 2024-03-12T17:07:05.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.996+0000 7ff37cc00640 1 -- 172.21.15.52:0/3633397962 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff37815c820 msgr2=0x7ff3780b48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:05.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.996+0000 7ff37cc00640 1 --2- 172.21.15.52:0/3633397962 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff37815c820 0x7ff3780b48d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff36400c7f0 tx=0x7ff36400ccb0 comp rx=0 tx=0).stop 2024-03-12T17:07:05.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.997+0000 7ff37cc00640 1 -- 172.21.15.52:0/3633397962 shutdown_connections 2024-03-12T17:07:05.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.997+0000 7ff37cc00640 1 --2- 172.21.15.52:0/3633397962 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff35806ac00 0x7ff35806d0c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:05.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.997+0000 7ff37cc00640 1 --2- 172.21.15.52:0/3633397962 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff378160e90 0x7ff3780b5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:05.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.997+0000 7ff37cc00640 1 --2- 172.21.15.52:0/3633397962 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff37815c820 0x7ff3780b48d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:05.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.997+0000 7ff37cc00640 1 --2- 172.21.15.52:0/3633397962 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3780cf5f0 0x7ff37814f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:05.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.997+0000 7ff37cc00640 1 -- 172.21.15.52:0/3633397962 >> 172.21.15.52:0/3633397962 conn(0x7ff37806b050 msgr2=0x7ff3780b73f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:05.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.997+0000 7ff37cc00640 1 -- 172.21.15.52:0/3633397962 shutdown_connections 2024-03-12T17:07:05.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:05.997+0000 7ff37cc00640 1 -- 172.21.15.52:0/3633397962 wait complete. 2024-03-12T17:07:06.008 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-12T17:07:06.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.236+0000 7f661b200640 1 Processor -- start 2024-03-12T17:07:06.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.236+0000 7f661b200640 1 -- start start 2024-03-12T17:07:06.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.238+0000 7f661b200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f661c143460 0x7f661c15aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:06.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.238+0000 7f661b200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f661c143930 0x7f661c0cd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:06.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.238+0000 7f661b200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f661c0cc6b0 con 0x7f661c0cdc50 2024-03-12T17:07:06.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.238+0000 7f661b200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f661c0cc950 con 0x7f661c143460 2024-03-12T17:07:06.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.238+0000 7f661b200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f661c0ccc30 con 0x7f661c143930 2024-03-12T17:07:06.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.239+0000 7f6619e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f661c143460 0x7f661c15aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:06.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.239+0000 7f6619e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f661c143460 0x7f661c15aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:40046/0 (socket says 172.21.15.52:40046) 2024-03-12T17:07:06.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.239+0000 7f6619e00640 1 -- 172.21.15.52:0/626736209 learned_addr learned my addr 172.21.15.52:0/626736209 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:06.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.239+0000 7f6613e00640 1 --2- 172.21.15.52:0/626736209 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f661c143930 0x7f661c0cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:06.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.239+0000 7f6619e00640 1 -- 172.21.15.52:0/626736209 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f661c143930 msgr2=0x7f661c0cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:06.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.239+0000 7f6619e00640 1 --2- 172.21.15.52:0/626736209 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f661c143930 0x7f661c0cd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:06.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.239+0000 7f6619e00640 1 -- 172.21.15.52:0/626736209 >> v1:172.21.15.102:6789/0 conn(0x7f661c0cdc50 legacy=0x7f661c15e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:06.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.239+0000 7f6619e00640 1 -- 172.21.15.52:0/626736209 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f661c0ccf10 con 0x7f661c143460 2024-03-12T17:07:06.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.241+0000 7f6619e00640 1 --2- 172.21.15.52:0/626736209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f661c143460 0x7f661c15aba0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f660c009870 tx=0x7f660c02ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b409d8c714e995 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:06.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.241+0000 7f6619400640 1 -- 172.21.15.52:0/626736209 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c02f830 con 0x7f661c143460 2024-03-12T17:07:06.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.242+0000 7f6619400640 1 -- 172.21.15.52:0/626736209 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f660c02f990 con 0x7f661c143460 2024-03-12T17:07:06.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.242+0000 7f6619400640 1 -- 172.21.15.52:0/626736209 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c02fd20 con 0x7f661c143460 2024-03-12T17:07:06.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.242+0000 7f661b200640 1 -- 172.21.15.52:0/626736209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f661c143460 msgr2=0x7f661c15aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:06.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.242+0000 7f661b200640 1 --2- 172.21.15.52:0/626736209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f661c143460 0x7f661c15aba0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f660c009870 tx=0x7f660c02ba10 comp rx=0 tx=0).stop 2024-03-12T17:07:06.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.243+0000 7f661b200640 1 -- 172.21.15.52:0/626736209 shutdown_connections 2024-03-12T17:07:06.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.243+0000 7f661b200640 1 --2- 172.21.15.52:0/626736209 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f661c143930 0x7f661c0cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:06.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.243+0000 7f661b200640 1 --2- 172.21.15.52:0/626736209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f661c143460 0x7f661c15aba0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:06.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.243+0000 7f661b200640 1 -- 172.21.15.52:0/626736209 >> 172.21.15.52:0/626736209 conn(0x7f661c065df0 msgr2=0x7f661c069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:06.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.243+0000 7f661b200640 1 -- 172.21.15.52:0/626736209 shutdown_connections 2024-03-12T17:07:06.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.243+0000 7f661b200640 1 -- 172.21.15.52:0/626736209 wait complete. 2024-03-12T17:07:06.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.245+0000 7f661b200640 1 Processor -- start 2024-03-12T17:07:06.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.245+0000 7f661b200640 1 -- start start 2024-03-12T17:07:06.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.246+0000 7f661b200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f661c0cdc50 0x7f661c1561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:06.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.246+0000 7f661b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f661c143460 0x7f661c0ad450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:06.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.246+0000 7f661b200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f661c143930 0x7f661c0ad990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:06.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.246+0000 7f661b200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661c13b0c0 con 0x7f661c0cdc50 2024-03-12T17:07:06.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.246+0000 7f661b200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661c06c080 con 0x7f661c143460 2024-03-12T17:07:06.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.246+0000 7f661b200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f661c06cc00 con 0x7f661c143930 2024-03-12T17:07:06.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.247+0000 7f6619e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f661c0cdc50 0x7f661c1561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:06.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.247+0000 7f6619e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f661c0cdc50 0x7f661c1561b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54138/0 (socket says 172.21.15.52:54138) 2024-03-12T17:07:06.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.247+0000 7f6619e00640 1 -- 172.21.15.52:0/3434055599 learned_addr learned my addr 172.21.15.52:0/3434055599 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:06.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.247+0000 7f6613e00640 1 --2- 172.21.15.52:0/3434055599 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f661c143460 0x7f661c0ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:06.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.247+0000 7f661a800640 1 --2- 172.21.15.52:0/3434055599 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f661c143930 0x7f661c0ad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:06.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.247+0000 7f6619e00640 1 -- 172.21.15.52:0/3434055599 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f661c143930 msgr2=0x7f661c0ad990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:06.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.247+0000 7f6619e00640 1 --2- 172.21.15.52:0/3434055599 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f661c143930 0x7f661c0ad990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:06.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.247+0000 7f6619e00640 1 -- 172.21.15.52:0/3434055599 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f661c143460 msgr2=0x7f661c0ad450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:06.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.247+0000 7f6619e00640 1 --2- 172.21.15.52:0/3434055599 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f661c143460 0x7f661c0ad450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:06.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.247+0000 7f6619e00640 1 -- 172.21.15.52:0/3434055599 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f660c0094d0 con 0x7f661c0cdc50 2024-03-12T17:07:06.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.248+0000 7f6619e00640 1 --2- 172.21.15.52:0/3434055599 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f661c0cdc50 0x7f661c1561b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f660c02ba50 tx=0x7f660c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:06.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.248+0000 7f6612000640 1 -- 172.21.15.52:0/3434055599 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c037c40 con 0x7f661c0cdc50 2024-03-12T17:07:06.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.248+0000 7f6612000640 1 -- 172.21.15.52:0/3434055599 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f660c037da0 con 0x7f661c0cdc50 2024-03-12T17:07:06.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.248+0000 7f6612000640 1 -- 172.21.15.52:0/3434055599 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c0426c0 con 0x7f661c0cdc50 2024-03-12T17:07:06.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.248+0000 7f661b200640 1 -- 172.21.15.52:0/3434055599 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f661c15a230 con 0x7f661c0cdc50 2024-03-12T17:07:06.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.248+0000 7f661b200640 1 -- 172.21.15.52:0/3434055599 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f661c06c2d0 con 0x7f661c0cdc50 2024-03-12T17:07:06.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.249+0000 7f661b200640 1 -- 172.21.15.52:0/3434055599 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f661c04e030 con 0x7f661c0cdc50 2024-03-12T17:07:06.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.250+0000 7f6612000640 1 -- 172.21.15.52:0/3434055599 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f660c02d3c0 con 0x7f661c0cdc50 2024-03-12T17:07:06.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.251+0000 7f6612000640 1 --2- 172.21.15.52:0/3434055599 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f65f806ac30 0x7f65f806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:06.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.251+0000 7f6612000640 1 -- 172.21.15.52:0/3434055599 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f660c03c070 con 0x7f661c0cdc50 2024-03-12T17:07:06.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.252+0000 7f6613e00640 1 --2- 172.21.15.52:0/3434055599 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f65f806ac30 0x7f65f806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:06.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.253+0000 7f6613e00640 1 --2- 172.21.15.52:0/3434055599 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f65f806ac30 0x7f65f806d0f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f661c1f54d0 tx=0x7f6604008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:06.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.257+0000 7f6612000640 1 -- 172.21.15.52:0/3434055599 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f660c076a70 con 0x7f661c0cdc50 2024-03-12T17:07:06.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.461+0000 7f661b200640 1 -- 172.21.15.52:0/3434055599 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f661c06a860 con 0x7f661c0cdc50 2024-03-12T17:07:06.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.462+0000 7f6612000640 1 -- 172.21.15.52:0/3434055599 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f660c0470a0 con 0x7f661c0cdc50 2024-03-12T17:07:06.455 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-12T17:07:06.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.464+0000 7f661b200640 1 -- 172.21.15.52:0/3434055599 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f65f806ac30 msgr2=0x7f65f806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:06.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.464+0000 7f661b200640 1 --2- 172.21.15.52:0/3434055599 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f65f806ac30 0x7f65f806d0f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f661c1f54d0 tx=0x7f6604008040 comp rx=0 tx=0).stop 2024-03-12T17:07:06.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.464+0000 7f661b200640 1 -- 172.21.15.52:0/3434055599 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f661c0cdc50 msgr2=0x7f661c1561b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:06.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.464+0000 7f661b200640 1 --2- 172.21.15.52:0/3434055599 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f661c0cdc50 0x7f661c1561b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f660c02ba50 tx=0x7f660c004730 comp rx=0 tx=0).stop 2024-03-12T17:07:06.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.465+0000 7f661b200640 1 -- 172.21.15.52:0/3434055599 shutdown_connections 2024-03-12T17:07:06.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.465+0000 7f661b200640 1 --2- 172.21.15.52:0/3434055599 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f65f806ac30 0x7f65f806d0f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:06.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.465+0000 7f661b200640 1 --2- 172.21.15.52:0/3434055599 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f661c143930 0x7f661c0ad990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:06.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.465+0000 7f661b200640 1 --2- 172.21.15.52:0/3434055599 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f661c143460 0x7f661c0ad450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:06.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.465+0000 7f661b200640 1 --2- 172.21.15.52:0/3434055599 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f661c0cdc50 0x7f661c1561b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:06.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.465+0000 7f661b200640 1 -- 172.21.15.52:0/3434055599 >> 172.21.15.52:0/3434055599 conn(0x7f661c065df0 msgr2=0x7f661c15b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:06.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.465+0000 7f661b200640 1 -- 172.21.15.52:0/3434055599 shutdown_connections 2024-03-12T17:07:06.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.465+0000 7f661b200640 1 -- 172.21.15.52:0/3434055599 wait complete. 2024-03-12T17:07:06.479 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-12T17:07:06.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.707+0000 7f2e78e00640 1 Processor -- start 2024-03-12T17:07:06.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.707+0000 7f2e78e00640 1 -- start start 2024-03-12T17:07:06.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.709+0000 7f2e78e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2e740cd740 0x7f2e740b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:06.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.709+0000 7f2e78e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2e740d0d40 0x7f2e740cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:06.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.709+0000 7f2e78e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e740d20c0 con 0x7f2e7406b0a0 2024-03-12T17:07:06.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.709+0000 7f2e78e00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e740ce060 con 0x7f2e740d0d40 2024-03-12T17:07:06.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.709+0000 7f2e78e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e740d14b0 con 0x7f2e740cd740 2024-03-12T17:07:06.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.709+0000 7f2e6be00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2e740d0d40 0x7f2e740cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:06.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.709+0000 7f2e6be00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2e740d0d40 0x7f2e740cdb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54142/0 (socket says 172.21.15.52:54142) 2024-03-12T17:07:06.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.709+0000 7f2e6be00640 1 -- 172.21.15.52:0/115263062 learned_addr learned my addr 172.21.15.52:0/115263062 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:06.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.710+0000 7f2e71e00640 1 --2- 172.21.15.52:0/115263062 >> v2:172.21.15.102:3300/0 conn(0x7f2e740cd740 0x7f2e740b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:06.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.710+0000 7f2e6be00640 1 -- 172.21.15.52:0/115263062 >> v2:172.21.15.102:3300/0 conn(0x7f2e740cd740 msgr2=0x7f2e740b84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:06.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.710+0000 7f2e6be00640 1 --2- 172.21.15.52:0/115263062 >> v2:172.21.15.102:3300/0 conn(0x7f2e740cd740 0x7f2e740b84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:06.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.710+0000 7f2e6be00640 1 -- 172.21.15.52:0/115263062 >> v1:172.21.15.102:6789/0 conn(0x7f2e7406b0a0 legacy=0x7f2e7406b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:06.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.710+0000 7f2e6be00640 1 -- 172.21.15.52:0/115263062 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e740d1790 con 0x7f2e740d0d40 2024-03-12T17:07:06.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.711+0000 7f2e6be00640 1 --2- 172.21.15.52:0/115263062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e740d0d40 0x7f2e740cdb20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2e7408a100 tx=0x7f2e5c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc26bd6d7660794 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:06.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.711+0000 7f2e71400640 1 -- 172.21.15.52:0/115263062 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c031870 con 0x7f2e740d0d40 2024-03-12T17:07:06.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.712+0000 7f2e71400640 1 -- 172.21.15.52:0/115263062 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e5c0319d0 con 0x7f2e740d0d40 2024-03-12T17:07:06.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.712+0000 7f2e71400640 1 -- 172.21.15.52:0/115263062 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c031d60 con 0x7f2e740d0d40 2024-03-12T17:07:06.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.712+0000 7f2e78e00640 1 -- 172.21.15.52:0/115263062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e740d0d40 msgr2=0x7f2e740cdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:06.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.712+0000 7f2e78e00640 1 --2- 172.21.15.52:0/115263062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e740d0d40 0x7f2e740cdb20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2e7408a100 tx=0x7f2e5c02b910 comp rx=0 tx=0).stop 2024-03-12T17:07:06.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.712+0000 7f2e78e00640 1 -- 172.21.15.52:0/115263062 shutdown_connections 2024-03-12T17:07:06.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.712+0000 7f2e78e00640 1 --2- 172.21.15.52:0/115263062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e740d0d40 0x7f2e740cdb20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:06.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.712+0000 7f2e78e00640 1 --2- 172.21.15.52:0/115263062 >> v2:172.21.15.102:3300/0 conn(0x7f2e740cd740 0x7f2e740b84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:06.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.712+0000 7f2e78e00640 1 -- 172.21.15.52:0/115263062 >> 172.21.15.52:0/115263062 conn(0x7f2e740cf5d0 msgr2=0x7f2e740cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:06.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.712+0000 7f2e78e00640 1 -- 172.21.15.52:0/115263062 shutdown_connections 2024-03-12T17:07:06.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.712+0000 7f2e78e00640 1 -- 172.21.15.52:0/115263062 wait complete. 2024-03-12T17:07:06.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.714+0000 7f2e78e00640 1 Processor -- start 2024-03-12T17:07:06.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.714+0000 7f2e78e00640 1 -- start start 2024-03-12T17:07:06.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.715+0000 7f2e78e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e7406b0a0 0x7f2e74157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:06.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.715+0000 7f2e78e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2e740cd740 0x7f2e740cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:06.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.715+0000 7f2e78e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e740d0d40 0x7f2e740cb130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:06.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.715+0000 7f2e78e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e740ca760 con 0x7f2e740d0d40 2024-03-12T17:07:06.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.715+0000 7f2e78e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e740c9b50 con 0x7f2e7406b0a0 2024-03-12T17:07:06.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.715+0000 7f2e78e00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e740c9e30 con 0x7f2e740cd740 2024-03-12T17:07:06.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.715+0000 7f2e72800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e740d0d40 0x7f2e740cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:06.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.715+0000 7f2e72800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e740d0d40 0x7f2e740cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54158/0 (socket says 172.21.15.52:54158) 2024-03-12T17:07:06.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.716+0000 7f2e72800640 1 -- 172.21.15.52:0/2790114259 learned_addr learned my addr 172.21.15.52:0/2790114259 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:06.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.716+0000 7f2e6be00640 1 --2- 172.21.15.52:0/2790114259 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2e740cd740 0x7f2e740cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:06.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.716+0000 7f2e71e00640 1 --2- 172.21.15.52:0/2790114259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e7406b0a0 0x7f2e74157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:06.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.716+0000 7f2e72800640 1 -- 172.21.15.52:0/2790114259 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2e740cd740 msgr2=0x7f2e740cabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:06.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.716+0000 7f2e72800640 1 --2- 172.21.15.52:0/2790114259 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2e740cd740 0x7f2e740cabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:06.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.716+0000 7f2e72800640 1 -- 172.21.15.52:0/2790114259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e7406b0a0 msgr2=0x7f2e74157d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:06.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.716+0000 7f2e72800640 1 --2- 172.21.15.52:0/2790114259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e7406b0a0 0x7f2e74157d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:06.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.716+0000 7f2e72800640 1 -- 172.21.15.52:0/2790114259 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e5c0094d0 con 0x7f2e740d0d40 2024-03-12T17:07:06.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.716+0000 7f2e71e00640 1 --2- 172.21.15.52:0/2790114259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e7406b0a0 0x7f2e74157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:06.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.716+0000 7f2e72800640 1 --2- 172.21.15.52:0/2790114259 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e740d0d40 0x7f2e740cb130 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00c780 tx=0x7f2e6c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:06.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.716+0000 7f2e6a000640 1 -- 172.21.15.52:0/2790114259 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c004b60 con 0x7f2e740d0d40 2024-03-12T17:07:06.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.717+0000 7f2e6a000640 1 -- 172.21.15.52:0/2790114259 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e6c00ce90 con 0x7f2e740d0d40 2024-03-12T17:07:06.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.717+0000 7f2e6a000640 1 -- 172.21.15.52:0/2790114259 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c00a850 con 0x7f2e740d0d40 2024-03-12T17:07:06.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.717+0000 7f2e78e00640 1 -- 172.21.15.52:0/2790114259 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e740ca110 con 0x7f2e740d0d40 2024-03-12T17:07:06.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.717+0000 7f2e78e00640 1 -- 172.21.15.52:0/2790114259 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e74155830 con 0x7f2e740d0d40 2024-03-12T17:07:06.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.718+0000 7f2e78e00640 1 -- 172.21.15.52:0/2790114259 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e740ca8d0 con 0x7f2e740d0d40 2024-03-12T17:07:06.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.719+0000 7f2e6a000640 1 -- 172.21.15.52:0/2790114259 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2e6c00a9b0 con 0x7f2e740d0d40 2024-03-12T17:07:06.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.720+0000 7f2e6a000640 1 --2- 172.21.15.52:0/2790114259 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2e5006ac50 0x7f2e5006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:06.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.720+0000 7f2e6a000640 1 -- 172.21.15.52:0/2790114259 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2e6c089790 con 0x7f2e740d0d40 2024-03-12T17:07:06.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.720+0000 7f2e71e00640 1 --2- 172.21.15.52:0/2790114259 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2e5006ac50 0x7f2e5006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:06.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.721+0000 7f2e71e00640 1 --2- 172.21.15.52:0/2790114259 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2e5006ac50 0x7f2e5006d110 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2e64006fd0 tx=0x7f2e64008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:06.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.724+0000 7f2e6a000640 1 -- 172.21.15.52:0/2790114259 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f2e6c054760 con 0x7f2e740d0d40 2024-03-12T17:07:06.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.923+0000 7f2e78e00640 1 -- 172.21.15.52:0/2790114259 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2e74073390 con 0x7f2e740d0d40 2024-03-12T17:07:06.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.923+0000 7f2e6a000640 1 -- 172.21.15.52:0/2790114259 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2e6c05e030 con 0x7f2e740d0d40 2024-03-12T17:07:06.917 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-03-12T17:07:06.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.926+0000 7f2e78e00640 1 -- 172.21.15.52:0/2790114259 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2e5006ac50 msgr2=0x7f2e5006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:06.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.926+0000 7f2e78e00640 1 --2- 172.21.15.52:0/2790114259 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2e5006ac50 0x7f2e5006d110 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2e64006fd0 tx=0x7f2e64008040 comp rx=0 tx=0).stop 2024-03-12T17:07:06.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.926+0000 7f2e78e00640 1 -- 172.21.15.52:0/2790114259 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e740d0d40 msgr2=0x7f2e740cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:06.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.926+0000 7f2e78e00640 1 --2- 172.21.15.52:0/2790114259 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e740d0d40 0x7f2e740cb130 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00c780 tx=0x7f2e6c00cc40 comp rx=0 tx=0).stop 2024-03-12T17:07:06.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.927+0000 7f2e78e00640 1 -- 172.21.15.52:0/2790114259 shutdown_connections 2024-03-12T17:07:06.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.927+0000 7f2e78e00640 1 --2- 172.21.15.52:0/2790114259 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2e5006ac50 0x7f2e5006d110 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:06.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.927+0000 7f2e78e00640 1 --2- 172.21.15.52:0/2790114259 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2e740cd740 0x7f2e740cabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:06.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.927+0000 7f2e78e00640 1 --2- 172.21.15.52:0/2790114259 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e740d0d40 0x7f2e740cb130 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:06.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.927+0000 7f2e78e00640 1 --2- 172.21.15.52:0/2790114259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e7406b0a0 0x7f2e74157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:06.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.927+0000 7f2e78e00640 1 -- 172.21.15.52:0/2790114259 >> 172.21.15.52:0/2790114259 conn(0x7f2e740cf5d0 msgr2=0x7f2e740c87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:06.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.927+0000 7f2e78e00640 1 -- 172.21.15.52:0/2790114259 shutdown_connections 2024-03-12T17:07:06.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:06.927+0000 7f2e78e00640 1 -- 172.21.15.52:0/2790114259 wait complete. 2024-03-12T17:07:06.939 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-12T17:07:07.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.168+0000 7f6d37400640 1 Processor -- start 2024-03-12T17:07:07.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.168+0000 7f6d37400640 1 -- start start 2024-03-12T17:07:07.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.169+0000 7f6d37400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d300ce930 0x7f6d300add30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:07.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.170+0000 7f6d37400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f6d3006ce90 0x7f6d3006d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:07.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.170+0000 7f6d37400640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d300cdba0 con 0x7f6d300cd3e0 2024-03-12T17:07:07.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.170+0000 7f6d37400640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d3006c760 con 0x7f6d3006ce90 2024-03-12T17:07:07.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.170+0000 7f6d37400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d300b42d0 con 0x7f6d300ce930 2024-03-12T17:07:07.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.170+0000 7f6d2fe00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f6d300cd3e0 0x7f6d300ced10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:33218/0 (socket says 172.21.15.52:33218) 2024-03-12T17:07:07.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.170+0000 7f6d2fe00640 1 -- 172.21.15.52:0/1057816651 learned_addr learned my addr 172.21.15.52:0/1057816651 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:07.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.170+0000 7f6d35400640 1 --2- 172.21.15.52:0/1057816651 >> v2:172.21.15.52:3300/0 conn(0x7f6d3006ce90 0x7f6d3006d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:07.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.170+0000 7f6d34a00640 1 --2- 172.21.15.52:0/1057816651 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d300ce930 0x7f6d300add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:07.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.171+0000 7f6d35400640 1 -- 172.21.15.52:0/1057816651 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d300ce930 msgr2=0x7f6d300add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:07.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.171+0000 7f6d35400640 1 --2- 172.21.15.52:0/1057816651 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d300ce930 0x7f6d300add30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:07.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.171+0000 7f6d35400640 1 -- 172.21.15.52:0/1057816651 >> v1:172.21.15.52:6789/0 conn(0x7f6d300cd3e0 legacy=0x7f6d300ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:07.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.171+0000 7f6d35400640 1 -- 172.21.15.52:0/1057816651 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d300cde80 con 0x7f6d3006ce90 2024-03-12T17:07:07.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.172+0000 7f6d35400640 1 --2- 172.21.15.52:0/1057816651 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d3006ce90 0x7f6d3006d320 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6d280097a0 tx=0x7f6d2802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3101751569568e48 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:07.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.172+0000 7f6d2f400640 1 -- 172.21.15.52:0/1057816651 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2802d040 con 0x7f6d3006ce90 2024-03-12T17:07:07.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.172+0000 7f6d2f400640 1 -- 172.21.15.52:0/1057816651 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d28031810 con 0x7f6d3006ce90 2024-03-12T17:07:07.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.172+0000 7f6d2f400640 1 -- 172.21.15.52:0/1057816651 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d28031ba0 con 0x7f6d3006ce90 2024-03-12T17:07:07.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.173+0000 7f6d37400640 1 -- 172.21.15.52:0/1057816651 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d3006ce90 msgr2=0x7f6d3006d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:07.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.173+0000 7f6d37400640 1 --2- 172.21.15.52:0/1057816651 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d3006ce90 0x7f6d3006d320 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6d280097a0 tx=0x7f6d2802b8b0 comp rx=0 tx=0).stop 2024-03-12T17:07:07.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.173+0000 7f6d37400640 1 -- 172.21.15.52:0/1057816651 shutdown_connections 2024-03-12T17:07:07.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.173+0000 7f6d37400640 1 --2- 172.21.15.52:0/1057816651 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d3006ce90 0x7f6d3006d320 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:07.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.173+0000 7f6d37400640 1 --2- 172.21.15.52:0/1057816651 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d300ce930 0x7f6d300add30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:07.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.173+0000 7f6d37400640 1 -- 172.21.15.52:0/1057816651 >> 172.21.15.52:0/1057816651 conn(0x7f6d3006a0b0 msgr2=0x7f6d3006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:07.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.173+0000 7f6d37400640 1 -- 172.21.15.52:0/1057816651 shutdown_connections 2024-03-12T17:07:07.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.173+0000 7f6d37400640 1 -- 172.21.15.52:0/1057816651 wait complete. 2024-03-12T17:07:07.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.175+0000 7f6d37400640 1 Processor -- start 2024-03-12T17:07:07.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.175+0000 7f6d37400640 1 -- start start 2024-03-12T17:07:07.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.176+0000 7f6d37400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d3006ce90 0x7f6d300aef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:07.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.176+0000 7f6d37400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d300cd3e0 0x7f6d300b0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:07.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.176+0000 7f6d37400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d300ce930 0x7f6d3015a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:07.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.176+0000 7f6d37400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d301547a0 con 0x7f6d300ce930 2024-03-12T17:07:07.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.176+0000 7f6d37400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d300b0fe0 con 0x7f6d3006ce90 2024-03-12T17:07:07.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.176+0000 7f6d37400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d30153e40 con 0x7f6d300cd3e0 2024-03-12T17:07:07.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.176+0000 7f6d35400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d300ce930 0x7f6d3015a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:07.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.176+0000 7f6d35400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d300ce930 0x7f6d3015a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54180/0 (socket says 172.21.15.52:54180) 2024-03-12T17:07:07.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.176+0000 7f6d35400640 1 -- 172.21.15.52:0/3916261804 learned_addr learned my addr 172.21.15.52:0/3916261804 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:07.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.176+0000 7f6d34a00640 1 --2- 172.21.15.52:0/3916261804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d3006ce90 0x7f6d300aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:07.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.176+0000 7f6d2fe00640 1 --2- 172.21.15.52:0/3916261804 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d300cd3e0 0x7f6d300b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:07.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.176+0000 7f6d35400640 1 -- 172.21.15.52:0/3916261804 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d300cd3e0 msgr2=0x7f6d300b0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:07.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.176+0000 7f6d35400640 1 --2- 172.21.15.52:0/3916261804 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d300cd3e0 0x7f6d300b0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:07.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.176+0000 7f6d35400640 1 -- 172.21.15.52:0/3916261804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d3006ce90 msgr2=0x7f6d300aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:07.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.176+0000 7f6d35400640 1 --2- 172.21.15.52:0/3916261804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d3006ce90 0x7f6d300aef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:07.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.177+0000 7f6d35400640 1 -- 172.21.15.52:0/3916261804 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d28009400 con 0x7f6d300ce930 2024-03-12T17:07:07.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.177+0000 7f6d34a00640 1 --2- 172.21.15.52:0/3916261804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d3006ce90 0x7f6d300aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:07.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.177+0000 7f6d35400640 1 --2- 172.21.15.52:0/3916261804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d300ce930 0x7f6d3015a440 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6d2802b8f0 tx=0x7f6d280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:07.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.177+0000 7f6d2cc00640 1 -- 172.21.15.52:0/3916261804 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2803ace0 con 0x7f6d300ce930 2024-03-12T17:07:07.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.177+0000 7f6d2cc00640 1 -- 172.21.15.52:0/3916261804 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d2803ae60 con 0x7f6d300ce930 2024-03-12T17:07:07.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.177+0000 7f6d2cc00640 1 -- 172.21.15.52:0/3916261804 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d28044630 con 0x7f6d300ce930 2024-03-12T17:07:07.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.177+0000 7f6d37400640 1 -- 172.21.15.52:0/3916261804 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d300b0d40 con 0x7f6d300ce930 2024-03-12T17:07:07.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.177+0000 7f6d37400640 1 -- 172.21.15.52:0/3916261804 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d301543c0 con 0x7f6d300ce930 2024-03-12T17:07:07.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.179+0000 7f6d37400640 1 -- 172.21.15.52:0/3916261804 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d300b3e60 con 0x7f6d300ce930 2024-03-12T17:07:07.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.179+0000 7f6d2cc00640 1 -- 172.21.15.52:0/3916261804 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6d28044790 con 0x7f6d300ce930 2024-03-12T17:07:07.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.180+0000 7f6d2cc00640 1 --2- 172.21.15.52:0/3916261804 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f6d1006ac00 0x7f6d1006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:07.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.181+0000 7f6d2cc00640 1 -- 172.21.15.52:0/3916261804 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6d280aba30 con 0x7f6d300ce930 2024-03-12T17:07:07.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.181+0000 7f6d34a00640 1 --2- 172.21.15.52:0/3916261804 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f6d1006ac00 0x7f6d1006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:07.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.182+0000 7f6d34a00640 1 --2- 172.21.15.52:0/3916261804 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f6d1006ac00 0x7f6d1006d0c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6d20006fd0 tx=0x7f6d20008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:07.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.185+0000 7f6d2cc00640 1 -- 172.21.15.52:0/3916261804 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f6d28076ad0 con 0x7f6d300ce930 2024-03-12T17:07:07.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.387+0000 7f6d37400640 1 -- 172.21.15.52:0/3916261804 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6d300cf600 con 0x7f6d300ce930 2024-03-12T17:07:07.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.388+0000 7f6d2cc00640 1 -- 172.21.15.52:0/3916261804 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6d280350a0 con 0x7f6d300ce930 2024-03-12T17:07:07.382 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-12T17:07:07.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.391+0000 7f6d37400640 1 -- 172.21.15.52:0/3916261804 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f6d1006ac00 msgr2=0x7f6d1006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:07.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.391+0000 7f6d37400640 1 --2- 172.21.15.52:0/3916261804 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f6d1006ac00 0x7f6d1006d0c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6d20006fd0 tx=0x7f6d20008040 comp rx=0 tx=0).stop 2024-03-12T17:07:07.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.391+0000 7f6d37400640 1 -- 172.21.15.52:0/3916261804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d300ce930 msgr2=0x7f6d3015a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:07.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.391+0000 7f6d37400640 1 --2- 172.21.15.52:0/3916261804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d300ce930 0x7f6d3015a440 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6d2802b8f0 tx=0x7f6d280046c0 comp rx=0 tx=0).stop 2024-03-12T17:07:07.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.391+0000 7f6d37400640 1 -- 172.21.15.52:0/3916261804 shutdown_connections 2024-03-12T17:07:07.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.391+0000 7f6d37400640 1 --2- 172.21.15.52:0/3916261804 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f6d1006ac00 0x7f6d1006d0c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:07.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.391+0000 7f6d37400640 1 --2- 172.21.15.52:0/3916261804 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d300cd3e0 0x7f6d300b0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:07.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.391+0000 7f6d37400640 1 --2- 172.21.15.52:0/3916261804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d300ce930 0x7f6d3015a440 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:07.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.391+0000 7f6d37400640 1 --2- 172.21.15.52:0/3916261804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d3006ce90 0x7f6d300aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:07.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.391+0000 7f6d37400640 1 -- 172.21.15.52:0/3916261804 >> 172.21.15.52:0/3916261804 conn(0x7f6d3006a0b0 msgr2=0x7f6d300cbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:07.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.391+0000 7f6d37400640 1 -- 172.21.15.52:0/3916261804 shutdown_connections 2024-03-12T17:07:07.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.391+0000 7f6d37400640 1 -- 172.21.15.52:0/3916261804 wait complete. 2024-03-12T17:07:07.405 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T17:07:07.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.626+0000 7f6963200640 1 Processor -- start 2024-03-12T17:07:07.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.627+0000 7f6963200640 1 -- start start 2024-03-12T17:07:07.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.628+0000 7f6963200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f69641571d0 0x7f69640b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:07.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.628+0000 7f6963200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f69640cdc00 0x7f696415aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:07.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.628+0000 7f6963200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69641575b0 con 0x7f696415afe0 2024-03-12T17:07:07.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.628+0000 7f6963200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6964159f20 con 0x7f69640cdc00 2024-03-12T17:07:07.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.628+0000 7f6963200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f696415a200 con 0x7f69641571d0 2024-03-12T17:07:07.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.629+0000 7f6959400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f69640cdc00 0x7f696415aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:07.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.629+0000 7f6959400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f69640cdc00 0x7f696415aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54188/0 (socket says 172.21.15.52:54188) 2024-03-12T17:07:07.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.629+0000 7f6959400640 1 -- 172.21.15.52:0/1654660933 learned_addr learned my addr 172.21.15.52:0/1654660933 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:07.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.629+0000 7f6961e00640 1 --2- 172.21.15.52:0/1654660933 >> v2:172.21.15.102:3300/0 conn(0x7f69641571d0 0x7f69640b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:07.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.629+0000 7f6959400640 1 -- 172.21.15.52:0/1654660933 >> v2:172.21.15.102:3300/0 conn(0x7f69641571d0 msgr2=0x7f69640b35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:07.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.629+0000 7f6959400640 1 --2- 172.21.15.52:0/1654660933 >> v2:172.21.15.102:3300/0 conn(0x7f69641571d0 0x7f69640b35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:07.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.629+0000 7f6959400640 1 -- 172.21.15.52:0/1654660933 >> v1:172.21.15.102:6789/0 conn(0x7f696415afe0 legacy=0x7f69640cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:07.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.629+0000 7f6959400640 1 -- 172.21.15.52:0/1654660933 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f696415a4e0 con 0x7f69640cdc00 2024-03-12T17:07:07.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.630+0000 7f6959400640 1 --2- 172.21.15.52:0/1654660933 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69640cdc00 0x7f696415aaa0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f69640712e0 tx=0x7f694c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af32af03945e581 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:07.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.631+0000 7f6961400640 1 -- 172.21.15.52:0/1654660933 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694c031870 con 0x7f69640cdc00 2024-03-12T17:07:07.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.631+0000 7f6961400640 1 -- 172.21.15.52:0/1654660933 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f694c0319d0 con 0x7f69640cdc00 2024-03-12T17:07:07.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.631+0000 7f6961400640 1 -- 172.21.15.52:0/1654660933 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694c031d60 con 0x7f69640cdc00 2024-03-12T17:07:07.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.631+0000 7f6963200640 1 -- 172.21.15.52:0/1654660933 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69640cdc00 msgr2=0x7f696415aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:07.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.631+0000 7f6963200640 1 --2- 172.21.15.52:0/1654660933 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69640cdc00 0x7f696415aaa0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f69640712e0 tx=0x7f694c02b910 comp rx=0 tx=0).stop 2024-03-12T17:07:07.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.632+0000 7f6963200640 1 -- 172.21.15.52:0/1654660933 shutdown_connections 2024-03-12T17:07:07.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.632+0000 7f6963200640 1 --2- 172.21.15.52:0/1654660933 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69640cdc00 0x7f696415aaa0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:07.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.632+0000 7f6963200640 1 --2- 172.21.15.52:0/1654660933 >> v2:172.21.15.102:3300/0 conn(0x7f69641571d0 0x7f69640b35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:07.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.632+0000 7f6963200640 1 -- 172.21.15.52:0/1654660933 >> 172.21.15.52:0/1654660933 conn(0x7f6964065df0 msgr2=0x7f6964069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:07.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.632+0000 7f6963200640 1 -- 172.21.15.52:0/1654660933 shutdown_connections 2024-03-12T17:07:07.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.632+0000 7f6963200640 1 -- 172.21.15.52:0/1654660933 wait complete. 2024-03-12T17:07:07.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.633+0000 7f6963200640 1 Processor -- start 2024-03-12T17:07:07.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.634+0000 7f6963200640 1 -- start start 2024-03-12T17:07:07.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.634+0000 7f6963200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69640cdc00 0x7f696413e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:07.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.635+0000 7f6963200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f69641571d0 0x7f6964138c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:07.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.635+0000 7f6963200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f696415afe0 0x7f6964154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:07.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.635+0000 7f6963200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696406cf20 con 0x7f696415afe0 2024-03-12T17:07:07.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.635+0000 7f6963200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696406d200 con 0x7f69640cdc00 2024-03-12T17:07:07.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.635+0000 7f6963200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f696406d4e0 con 0x7f69641571d0 2024-03-12T17:07:07.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.635+0000 7f6962800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f696415afe0 0x7f6964154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:07.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.635+0000 7f6962800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f696415afe0 0x7f6964154740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54200/0 (socket says 172.21.15.52:54200) 2024-03-12T17:07:07.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.635+0000 7f6962800640 1 -- 172.21.15.52:0/2439935109 learned_addr learned my addr 172.21.15.52:0/2439935109 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:07.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.635+0000 7f6961e00640 1 --2- 172.21.15.52:0/2439935109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69640cdc00 0x7f696413e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:07.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.635+0000 7f6959400640 1 --2- 172.21.15.52:0/2439935109 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f69641571d0 0x7f6964138c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:07.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.635+0000 7f6962800640 1 -- 172.21.15.52:0/2439935109 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f69641571d0 msgr2=0x7f6964138c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:07.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.635+0000 7f6962800640 1 --2- 172.21.15.52:0/2439935109 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f69641571d0 0x7f6964138c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:07.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.635+0000 7f6962800640 1 -- 172.21.15.52:0/2439935109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69640cdc00 msgr2=0x7f696413e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:07.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.635+0000 7f6962800640 1 --2- 172.21.15.52:0/2439935109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69640cdc00 0x7f696413e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:07.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.635+0000 7f6962800640 1 -- 172.21.15.52:0/2439935109 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f694c0094d0 con 0x7f696415afe0 2024-03-12T17:07:07.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.635+0000 7f6961e00640 1 --2- 172.21.15.52:0/2439935109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69640cdc00 0x7f696413e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:07:07.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.636+0000 7f6962800640 1 --2- 172.21.15.52:0/2439935109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f696415afe0 0x7f6964154740 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f695c00be60 tx=0x7f695c00bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:07.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.636+0000 7f695aa00640 1 -- 172.21.15.52:0/2439935109 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c014440 con 0x7f696415afe0 2024-03-12T17:07:07.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.636+0000 7f695aa00640 1 -- 172.21.15.52:0/2439935109 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f695c0145a0 con 0x7f696415afe0 2024-03-12T17:07:07.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.636+0000 7f695aa00640 1 -- 172.21.15.52:0/2439935109 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c014990 con 0x7f696415afe0 2024-03-12T17:07:07.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.636+0000 7f6963200640 1 -- 172.21.15.52:0/2439935109 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6964157800 con 0x7f696415afe0 2024-03-12T17:07:07.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.636+0000 7f6963200640 1 -- 172.21.15.52:0/2439935109 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69640b13f0 con 0x7f696415afe0 2024-03-12T17:07:07.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.637+0000 7f6963200640 1 -- 172.21.15.52:0/2439935109 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f696404e030 con 0x7f696415afe0 2024-03-12T17:07:07.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.639+0000 7f695aa00640 1 -- 172.21.15.52:0/2439935109 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f695c01d3b0 con 0x7f696415afe0 2024-03-12T17:07:07.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.639+0000 7f695aa00640 1 --2- 172.21.15.52:0/2439935109 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f694006ac00 0x7f694006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:07.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.639+0000 7f695aa00640 1 -- 172.21.15.52:0/2439935109 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f695c089880 con 0x7f696415afe0 2024-03-12T17:07:07.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.640+0000 7f6961e00640 1 --2- 172.21.15.52:0/2439935109 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f694006ac00 0x7f694006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:07.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.641+0000 7f6961e00640 1 --2- 172.21.15.52:0/2439935109 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f694006ac00 0x7f694006d0c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6954006fd0 tx=0x7f6954008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:07.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.644+0000 7f695aa00640 1 -- 172.21.15.52:0/2439935109 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f695c0546d0 con 0x7f696415afe0 2024-03-12T17:07:07.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.883+0000 7f6963200640 1 -- 172.21.15.52:0/2439935109 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6964157d20 con 0x7f696415afe0 2024-03-12T17:07:07.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.885+0000 7f695aa00640 1 -- 172.21.15.52:0/2439935109 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4733 (secure 0 0 0) 0x7f695c057cf0 con 0x7f696415afe0 2024-03-12T17:07:07.879 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:07:07.879 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2950099092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2950099092},{"type":"v1","addr":"172.21.15.52:6834","nonce":2950099092}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/846108993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":846108993},{"type":"v1","addr":"172.21.15.52:6835","nonce":846108993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6836/3002500679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3002500679},{"type":"v1","addr":"172.21.15.102:6836","nonce":3002500679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-12T17:06:25.905186+0000","modified":"2024-03-12T17:06:54.075752+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6837/3333451504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6835","nonce":3333451504},{"type":"v1","addr":"172.21.15.102:6837","nonce":3333451504}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T17:07:07.879 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 9 2024-03-12T17:07:07.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.888+0000 7f6963200640 1 -- 172.21.15.52:0/2439935109 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f694006ac00 msgr2=0x7f694006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:07.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.888+0000 7f6963200640 1 --2- 172.21.15.52:0/2439935109 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f694006ac00 0x7f694006d0c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6954006fd0 tx=0x7f6954008040 comp rx=0 tx=0).stop 2024-03-12T17:07:07.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.888+0000 7f6963200640 1 -- 172.21.15.52:0/2439935109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f696415afe0 msgr2=0x7f6964154740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:07.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.888+0000 7f6963200640 1 --2- 172.21.15.52:0/2439935109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f696415afe0 0x7f6964154740 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f695c00be60 tx=0x7f695c00bf50 comp rx=0 tx=0).stop 2024-03-12T17:07:07.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.888+0000 7f6963200640 1 -- 172.21.15.52:0/2439935109 shutdown_connections 2024-03-12T17:07:07.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.888+0000 7f6963200640 1 --2- 172.21.15.52:0/2439935109 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f694006ac00 0x7f694006d0c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:07.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.888+0000 7f6963200640 1 --2- 172.21.15.52:0/2439935109 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f69641571d0 0x7f6964138c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:07.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.888+0000 7f6963200640 1 --2- 172.21.15.52:0/2439935109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f696415afe0 0x7f6964154740 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:07.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.888+0000 7f6963200640 1 --2- 172.21.15.52:0/2439935109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69640cdc00 0x7f696413e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:07.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.888+0000 7f6963200640 1 -- 172.21.15.52:0/2439935109 >> 172.21.15.52:0/2439935109 conn(0x7f6964065df0 msgr2=0x7f696415bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:07.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.888+0000 7f6963200640 1 -- 172.21.15.52:0/2439935109 shutdown_connections 2024-03-12T17:07:07.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:07.888+0000 7f6963200640 1 -- 172.21.15.52:0/2439935109 wait complete. 2024-03-12T17:07:07.902 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:07:08.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.127+0000 7f8eb2e00640 1 Processor -- start 2024-03-12T17:07:08.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.127+0000 7f8eb2e00640 1 -- start start 2024-03-12T17:07:08.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.128+0000 7f8eb2e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8eac15cda0 0x7f8eac0cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:08.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.128+0000 7f8eb2e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8eac0b4a60 0x7f8eac0b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:08.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.128+0000 7f8eb2e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eac0d1830 con 0x7f8eac0b5a30 2024-03-12T17:07:08.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.128+0000 7f8eb2e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eac0b1f20 con 0x7f8eac0b4a60 2024-03-12T17:07:08.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.128+0000 7f8eb2e00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac0d0c20 con 0x7f8eac15cda0 2024-03-12T17:07:08.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.129+0000 7f8eb0e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8eac0b4a60 0x7f8eac0b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:08.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.129+0000 7f8eabe00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8eac15cda0 0x7f8eac0cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:08.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.129+0000 7f8eabe00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8eac15cda0 0x7f8eac0cdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.52:33806/0 (socket says 172.21.15.52:33806) 2024-03-12T17:07:08.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.129+0000 7f8eabe00640 1 -- 172.21.15.52:0/1452210929 learned_addr learned my addr 172.21.15.52:0/1452210929 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:08.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.129+0000 7f8eb0e00640 1 --2- 172.21.15.52:0/1452210929 >> v2:172.21.15.102:3300/0 conn(0x7f8eac0b4a60 0x7f8eac0b5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:40142/0 (socket says 172.21.15.52:40142) 2024-03-12T17:07:08.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.129+0000 7f8eabe00640 1 -- 172.21.15.52:0/1452210929 >> v2:172.21.15.102:3300/0 conn(0x7f8eac0b4a60 msgr2=0x7f8eac0b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:08.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.129+0000 7f8eabe00640 1 --2- 172.21.15.52:0/1452210929 >> v2:172.21.15.102:3300/0 conn(0x7f8eac0b4a60 0x7f8eac0b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:08.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.129+0000 7f8eabe00640 1 -- 172.21.15.52:0/1452210929 >> v1:172.21.15.102:6789/0 conn(0x7f8eac0b5a30 legacy=0x7f8eac0b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:08.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.129+0000 7f8eabe00640 1 -- 172.21.15.52:0/1452210929 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eac0d0f00 con 0x7f8eac15cda0 2024-03-12T17:07:08.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.131+0000 7f8eabe00640 1 --2- 172.21.15.52:0/1452210929 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8eac15cda0 0x7f8eac0cdb40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c009870 tx=0x7f8e9c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19d6f7a478dd1f35 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:08.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.131+0000 7f8eaaa00640 1 -- 172.21.15.52:0/1452210929 <== mon.2 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c030840 con 0x7f8eac15cda0 2024-03-12T17:07:08.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.131+0000 7f8eaaa00640 1 -- 172.21.15.52:0/1452210929 <== mon.2 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9c0309a0 con 0x7f8eac15cda0 2024-03-12T17:07:08.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.131+0000 7f8eaaa00640 1 -- 172.21.15.52:0/1452210929 <== mon.2 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c030d30 con 0x7f8eac15cda0 2024-03-12T17:07:08.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.132+0000 7f8eb2e00640 1 -- 172.21.15.52:0/1452210929 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8eac15cda0 msgr2=0x7f8eac0cdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:08.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.132+0000 7f8eb2e00640 1 --2- 172.21.15.52:0/1452210929 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8eac15cda0 0x7f8eac0cdb40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c009870 tx=0x7f8e9c02b980 comp rx=0 tx=0).stop 2024-03-12T17:07:08.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.132+0000 7f8eb2e00640 1 -- 172.21.15.52:0/1452210929 shutdown_connections 2024-03-12T17:07:08.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.132+0000 7f8eb2e00640 1 --2- 172.21.15.52:0/1452210929 >> v2:172.21.15.102:3300/0 conn(0x7f8eac0b4a60 0x7f8eac0b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:08.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.132+0000 7f8eb2e00640 1 --2- 172.21.15.52:0/1452210929 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8eac15cda0 0x7f8eac0cdb40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:08.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.132+0000 7f8eb2e00640 1 -- 172.21.15.52:0/1452210929 >> 172.21.15.52:0/1452210929 conn(0x7f8eac0cf350 msgr2=0x7f8eac0cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:08.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.132+0000 7f8eb2e00640 1 -- 172.21.15.52:0/1452210929 shutdown_connections 2024-03-12T17:07:08.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.132+0000 7f8eb2e00640 1 -- 172.21.15.52:0/1452210929 wait complete. 2024-03-12T17:07:08.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.134+0000 7f8eb2e00640 1 Processor -- start 2024-03-12T17:07:08.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.134+0000 7f8eb2e00640 1 -- start start 2024-03-12T17:07:08.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.135+0000 7f8eb2e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8eac0b4a60 0x7f8eac15d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:08.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.135+0000 7f8eb2e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8eac0b5a30 0x7f8eac157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:08.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.135+0000 7f8eb2e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8eac15cda0 0x7f8eac0ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:08.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.135+0000 7f8eb2e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac0c9790 con 0x7f8eac0b4a60 2024-03-12T17:07:08.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.135+0000 7f8eb2e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac15d780 con 0x7f8eac0b5a30 2024-03-12T17:07:08.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.135+0000 7f8eb2e00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac158370 con 0x7f8eac15cda0 2024-03-12T17:07:08.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.135+0000 7f8eabe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8eac0b4a60 0x7f8eac15d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:08.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.135+0000 7f8eabe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8eac0b4a60 0x7f8eac15d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54212/0 (socket says 172.21.15.52:54212) 2024-03-12T17:07:08.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.135+0000 7f8eabe00640 1 -- 172.21.15.52:0/2414130062 learned_addr learned my addr 172.21.15.52:0/2414130062 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:08.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.136+0000 7f8eab400640 1 --2- 172.21.15.52:0/2414130062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8eac0b5a30 0x7f8eac157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:08.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.136+0000 7f8eb0e00640 1 --2- 172.21.15.52:0/2414130062 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8eac15cda0 0x7f8eac0ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:08.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.136+0000 7f8eabe00640 1 -- 172.21.15.52:0/2414130062 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8eac15cda0 msgr2=0x7f8eac0ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:08.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.136+0000 7f8eabe00640 1 --2- 172.21.15.52:0/2414130062 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8eac15cda0 0x7f8eac0ca9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:08.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.136+0000 7f8eabe00640 1 -- 172.21.15.52:0/2414130062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8eac0b5a30 msgr2=0x7f8eac157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:08.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.136+0000 7f8eabe00640 1 --2- 172.21.15.52:0/2414130062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8eac0b5a30 0x7f8eac157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:08.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.136+0000 7f8eabe00640 1 -- 172.21.15.52:0/2414130062 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e9c0094d0 con 0x7f8eac0b4a60 2024-03-12T17:07:08.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.136+0000 7f8eabe00640 1 --2- 172.21.15.52:0/2414130062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8eac0b4a60 0x7f8eac15d240 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c000c00 tx=0x7f8e9c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:08.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.136+0000 7f8ea3e00640 1 -- 172.21.15.52:0/2414130062 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c039a90 con 0x7f8eac0b4a60 2024-03-12T17:07:08.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.137+0000 7f8ea3e00640 1 -- 172.21.15.52:0/2414130062 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9c039bf0 con 0x7f8eac0b4a60 2024-03-12T17:07:08.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.137+0000 7f8ea3e00640 1 -- 172.21.15.52:0/2414130062 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c043410 con 0x7f8eac0b4a60 2024-03-12T17:07:08.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.137+0000 7f8eb2e00640 1 -- 172.21.15.52:0/2414130062 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eac0c8b10 con 0x7f8eac0b4a60 2024-03-12T17:07:08.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.137+0000 7f8eb2e00640 1 -- 172.21.15.52:0/2414130062 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eac0c90d0 con 0x7f8eac0b4a60 2024-03-12T17:07:08.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.138+0000 7f8eb2e00640 1 -- 172.21.15.52:0/2414130062 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eac073390 con 0x7f8eac0b4a60 2024-03-12T17:07:08.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.139+0000 7f8ea3e00640 1 -- 172.21.15.52:0/2414130062 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8e9c02f390 con 0x7f8eac0b4a60 2024-03-12T17:07:08.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.140+0000 7f8ea3e00640 1 --2- 172.21.15.52:0/2414130062 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f8e8806ac00 0x7f8e8806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:08.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.140+0000 7f8ea3e00640 1 -- 172.21.15.52:0/2414130062 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8e9c0ab9d0 con 0x7f8eac0b4a60 2024-03-12T17:07:08.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.140+0000 7f8eab400640 1 --2- 172.21.15.52:0/2414130062 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f8e8806ac00 0x7f8e8806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:08.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.141+0000 7f8eab400640 1 --2- 172.21.15.52:0/2414130062 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f8e8806ac00 0x7f8e8806d0c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8eac15bb10 tx=0x7f8e94009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:08.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.144+0000 7f8ea3e00640 1 -- 172.21.15.52:0/2414130062 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f8e9c0769d0 con 0x7f8eac0b4a60 2024-03-12T17:07:08.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.342+0000 7f8eb2e00640 1 -- 172.21.15.52:0/2414130062 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8eac0c6160 con 0x7f8eac0b4a60 2024-03-12T17:07:08.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.345+0000 7f8ea3e00640 1 -- 172.21.15.52:0/2414130062 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7f8e9c034090 con 0x7f8eac0b4a60 2024-03-12T17:07:08.339 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:07:08.339 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":25,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:06:30.468476+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T17:06:19.218543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T17:06:23.525474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T17:06:24.911574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-12T17:06:28.017172+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.733031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.775178+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.591522+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.820838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.829252+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.582481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:07:08.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.347+0000 7f8eb2e00640 1 -- 172.21.15.52:0/2414130062 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f8e8806ac00 msgr2=0x7f8e8806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:08.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.347+0000 7f8eb2e00640 1 --2- 172.21.15.52:0/2414130062 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f8e8806ac00 0x7f8e8806d0c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8eac15bb10 tx=0x7f8e94009040 comp rx=0 tx=0).stop 2024-03-12T17:07:08.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.347+0000 7f8eb2e00640 1 -- 172.21.15.52:0/2414130062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8eac0b4a60 msgr2=0x7f8eac15d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:08.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.347+0000 7f8eb2e00640 1 --2- 172.21.15.52:0/2414130062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8eac0b4a60 0x7f8eac15d240 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c000c00 tx=0x7f8e9c004730 comp rx=0 tx=0).stop 2024-03-12T17:07:08.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.347+0000 7f8eb2e00640 1 -- 172.21.15.52:0/2414130062 shutdown_connections 2024-03-12T17:07:08.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.347+0000 7f8eb2e00640 1 --2- 172.21.15.52:0/2414130062 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f8e8806ac00 0x7f8e8806d0c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:08.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.347+0000 7f8eb2e00640 1 --2- 172.21.15.52:0/2414130062 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8eac15cda0 0x7f8eac0ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:08.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.347+0000 7f8eb2e00640 1 --2- 172.21.15.52:0/2414130062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8eac0b5a30 0x7f8eac157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:08.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.347+0000 7f8eb2e00640 1 --2- 172.21.15.52:0/2414130062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8eac0b4a60 0x7f8eac15d240 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:08.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.347+0000 7f8eb2e00640 1 -- 172.21.15.52:0/2414130062 >> 172.21.15.52:0/2414130062 conn(0x7f8eac0cf350 msgr2=0x7f8eac0b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:08.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.347+0000 7f8eb2e00640 1 -- 172.21.15.52:0/2414130062 shutdown_connections 2024-03-12T17:07:08.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.348+0000 7f8eb2e00640 1 -- 172.21.15.52:0/2414130062 wait complete. 2024-03-12T17:07:08.361 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T17:07:08.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.589+0000 7ff6c1600640 1 Processor -- start 2024-03-12T17:07:08.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.589+0000 7ff6c1600640 1 -- start start 2024-03-12T17:07:08.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.590+0000 7ff6c1600640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff6bc143460 0x7ff6bc15aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:08.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.591+0000 7ff6c1600640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff6bc143930 0x7ff6bc0cd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:08.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.591+0000 7ff6c1600640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6bc0cc6b0 con 0x7ff6bc0cdc50 2024-03-12T17:07:08.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.591+0000 7ff6c1600640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6bc0cc950 con 0x7ff6bc143460 2024-03-12T17:07:08.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.591+0000 7ff6c1600640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6bc0ccc30 con 0x7ff6bc143930 2024-03-12T17:07:08.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.591+0000 7ff6c0c00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7ff6bc0cdc50 0x7ff6bc15e1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:33220/0 (socket says 172.21.15.52:33220) 2024-03-12T17:07:08.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.591+0000 7ff6c0c00640 1 -- 172.21.15.52:0/3339425480 learned_addr learned my addr 172.21.15.52:0/3339425480 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:08.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.591+0000 7ff6bbe00640 1 --2- 172.21.15.52:0/3339425480 >> v2:172.21.15.52:3300/0 conn(0x7ff6bc143460 0x7ff6bc15aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:08.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.591+0000 7ff6bb400640 1 --2- 172.21.15.52:0/3339425480 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff6bc143930 0x7ff6bc0cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:08.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.591+0000 7ff6baa00640 1 -- 172.21.15.52:0/3339425480 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2099826307 0 0) 0x7ff6bc0cc6b0 con 0x7ff6bc0cdc50 2024-03-12T17:07:08.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.592+0000 7ff6baa00640 1 -- 172.21.15.52:0/3339425480 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6a0003520 con 0x7ff6bc0cdc50 2024-03-12T17:07:08.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.592+0000 7ff6bbe00640 1 -- 172.21.15.52:0/3339425480 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff6bc143930 msgr2=0x7ff6bc0cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:08.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.592+0000 7ff6bbe00640 1 --2- 172.21.15.52:0/3339425480 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff6bc143930 0x7ff6bc0cd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:08.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.592+0000 7ff6bbe00640 1 -- 172.21.15.52:0/3339425480 >> v1:172.21.15.52:6789/0 conn(0x7ff6bc0cdc50 legacy=0x7ff6bc15e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:08.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.592+0000 7ff6bbe00640 1 -- 172.21.15.52:0/3339425480 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6bc0ccf10 con 0x7ff6bc143460 2024-03-12T17:07:08.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.593+0000 7ff6bbe00640 1 --2- 172.21.15.52:0/3339425480 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6bc143460 0x7ff6bc15aba0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0097a0 tx=0x7ff6ac02b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea4bfb229b7d01f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:08.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.593+0000 7ff6baa00640 1 -- 172.21.15.52:0/3339425480 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac030800 con 0x7ff6bc143460 2024-03-12T17:07:08.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.593+0000 7ff6baa00640 1 -- 172.21.15.52:0/3339425480 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6ac030960 con 0x7ff6bc143460 2024-03-12T17:07:08.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.593+0000 7ff6baa00640 1 -- 172.21.15.52:0/3339425480 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac030cf0 con 0x7ff6bc143460 2024-03-12T17:07:08.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.594+0000 7ff6c1600640 1 -- 172.21.15.52:0/3339425480 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6bc143460 msgr2=0x7ff6bc15aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:08.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.594+0000 7ff6c1600640 1 --2- 172.21.15.52:0/3339425480 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6bc143460 0x7ff6bc15aba0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0097a0 tx=0x7ff6ac02b940 comp rx=0 tx=0).stop 2024-03-12T17:07:08.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.594+0000 7ff6c1600640 1 -- 172.21.15.52:0/3339425480 shutdown_connections 2024-03-12T17:07:08.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.594+0000 7ff6c1600640 1 --2- 172.21.15.52:0/3339425480 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff6bc143930 0x7ff6bc0cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:08.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.594+0000 7ff6c1600640 1 --2- 172.21.15.52:0/3339425480 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6bc143460 0x7ff6bc15aba0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:08.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.594+0000 7ff6c1600640 1 -- 172.21.15.52:0/3339425480 >> 172.21.15.52:0/3339425480 conn(0x7ff6bc065df0 msgr2=0x7ff6bc069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:08.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.594+0000 7ff6c1600640 1 -- 172.21.15.52:0/3339425480 shutdown_connections 2024-03-12T17:07:08.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.594+0000 7ff6c1600640 1 -- 172.21.15.52:0/3339425480 wait complete. 2024-03-12T17:07:08.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.596+0000 7ff6c1600640 1 Processor -- start 2024-03-12T17:07:08.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.596+0000 7ff6c1600640 1 -- start start 2024-03-12T17:07:08.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.597+0000 7ff6c1600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6bc0cdc50 0x7ff6bc1f1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:08.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.597+0000 7ff6c1600640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff6bc143460 0x7ff6bc1382a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:08.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.597+0000 7ff6c1600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6bc143930 0x7ff6bc13da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:08.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.597+0000 7ff6c1600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6bc136c20 con 0x7ff6bc0cdc50 2024-03-12T17:07:08.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.597+0000 7ff6c1600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6bc1f1d70 con 0x7ff6bc143930 2024-03-12T17:07:08.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.597+0000 7ff6c1600640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6bc135fa0 con 0x7ff6bc143460 2024-03-12T17:07:08.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.597+0000 7ff6bbe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6bc0cdc50 0x7ff6bc1f1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:08.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.597+0000 7ff6bbe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6bc0cdc50 0x7ff6bc1f1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54230/0 (socket says 172.21.15.52:54230) 2024-03-12T17:07:08.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.597+0000 7ff6bbe00640 1 -- 172.21.15.52:0/183210844 learned_addr learned my addr 172.21.15.52:0/183210844 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:08.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.597+0000 7ff6bb400640 1 --2- 172.21.15.52:0/183210844 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff6bc143460 0x7ff6bc1382a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:08.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.597+0000 7ff6c0c00640 1 --2- 172.21.15.52:0/183210844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6bc143930 0x7ff6bc13da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:08.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.597+0000 7ff6bbe00640 1 -- 172.21.15.52:0/183210844 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff6bc143460 msgr2=0x7ff6bc1382a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:08.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.597+0000 7ff6bbe00640 1 --2- 172.21.15.52:0/183210844 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff6bc143460 0x7ff6bc1382a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:08.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.597+0000 7ff6bbe00640 1 -- 172.21.15.52:0/183210844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6bc143930 msgr2=0x7ff6bc13da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:08.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.597+0000 7ff6bbe00640 1 --2- 172.21.15.52:0/183210844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6bc143930 0x7ff6bc13da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:08.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.597+0000 7ff6bbe00640 1 -- 172.21.15.52:0/183210844 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ac009400 con 0x7ff6bc0cdc50 2024-03-12T17:07:08.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.598+0000 7ff6bbe00640 1 --2- 172.21.15.52:0/183210844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6bc0cdc50 0x7ff6bc1f1830 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac02bf50 tx=0x7ff6ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:08.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.598+0000 7ff6b3e00640 1 -- 172.21.15.52:0/183210844 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac03e040 con 0x7ff6bc0cdc50 2024-03-12T17:07:08.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.598+0000 7ff6b3e00640 1 -- 172.21.15.52:0/183210844 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6ac039cb0 con 0x7ff6bc0cdc50 2024-03-12T17:07:08.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.598+0000 7ff6b3e00640 1 -- 172.21.15.52:0/183210844 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac044630 con 0x7ff6bc0cdc50 2024-03-12T17:07:08.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.598+0000 7ff6c1600640 1 -- 172.21.15.52:0/183210844 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6bc136280 con 0x7ff6bc0cdc50 2024-03-12T17:07:08.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.598+0000 7ff6c1600640 1 -- 172.21.15.52:0/183210844 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6bc136840 con 0x7ff6bc0cdc50 2024-03-12T17:07:08.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.600+0000 7ff6c1600640 1 -- 172.21.15.52:0/183210844 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6bc0adf50 con 0x7ff6bc0cdc50 2024-03-12T17:07:08.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.600+0000 7ff6b3e00640 1 -- 172.21.15.52:0/183210844 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff6ac043390 con 0x7ff6bc0cdc50 2024-03-12T17:07:08.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.601+0000 7ff6b3e00640 1 --2- 172.21.15.52:0/183210844 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff69c06ac00 0x7ff69c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:08.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.601+0000 7ff6b3e00640 1 -- 172.21.15.52:0/183210844 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff6ac0ab8a0 con 0x7ff6bc0cdc50 2024-03-12T17:07:08.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.602+0000 7ff6bb400640 1 --2- 172.21.15.52:0/183210844 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff69c06ac00 0x7ff69c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:08.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.603+0000 7ff6bb400640 1 --2- 172.21.15.52:0/183210844 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff69c06ac00 0x7ff69c06d0c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8006fd0 tx=0x7ff6a8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:08.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.606+0000 7ff6b3e00640 1 -- 172.21.15.52:0/183210844 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7ff6ac076820 con 0x7ff6bc0cdc50 2024-03-12T17:07:08.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.842+0000 7ff6c1600640 1 -- 172.21.15.52:0/183210844 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6bc136d60 con 0x7ff6bc0cdc50 2024-03-12T17:07:08.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.843+0000 7ff6b3e00640 1 -- 172.21.15.52:0/183210844 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4733 (secure 0 0 0) 0x7ff6ac079e40 con 0x7ff6bc0cdc50 2024-03-12T17:07:08.837 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:07:08.837 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2950099092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2950099092},{"type":"v1","addr":"172.21.15.52:6834","nonce":2950099092}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/846108993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":846108993},{"type":"v1","addr":"172.21.15.52:6835","nonce":846108993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6836/3002500679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3002500679},{"type":"v1","addr":"172.21.15.102:6836","nonce":3002500679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-12T17:06:25.905186+0000","modified":"2024-03-12T17:06:54.075752+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6837/3333451504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6835","nonce":3333451504},{"type":"v1","addr":"172.21.15.102:6837","nonce":3333451504}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T17:07:08.838 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 9 2024-03-12T17:07:08.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.846+0000 7ff6c1600640 1 -- 172.21.15.52:0/183210844 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff69c06ac00 msgr2=0x7ff69c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:08.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.846+0000 7ff6c1600640 1 --2- 172.21.15.52:0/183210844 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff69c06ac00 0x7ff69c06d0c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8006fd0 tx=0x7ff6a8008040 comp rx=0 tx=0).stop 2024-03-12T17:07:08.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.846+0000 7ff6c1600640 1 -- 172.21.15.52:0/183210844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6bc0cdc50 msgr2=0x7ff6bc1f1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:08.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.846+0000 7ff6c1600640 1 --2- 172.21.15.52:0/183210844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6bc0cdc50 0x7ff6bc1f1830 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac02bf50 tx=0x7ff6ac004730 comp rx=0 tx=0).stop 2024-03-12T17:07:08.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.846+0000 7ff6c1600640 1 -- 172.21.15.52:0/183210844 shutdown_connections 2024-03-12T17:07:08.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.846+0000 7ff6c1600640 1 --2- 172.21.15.52:0/183210844 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff69c06ac00 0x7ff69c06d0c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:08.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.846+0000 7ff6c1600640 1 --2- 172.21.15.52:0/183210844 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff6bc143460 0x7ff6bc1382a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:08.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.846+0000 7ff6c1600640 1 --2- 172.21.15.52:0/183210844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6bc143930 0x7ff6bc13da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:08.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.846+0000 7ff6c1600640 1 --2- 172.21.15.52:0/183210844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6bc0cdc50 0x7ff6bc1f1830 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:08.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.846+0000 7ff6c1600640 1 -- 172.21.15.52:0/183210844 >> 172.21.15.52:0/183210844 conn(0x7ff6bc065df0 msgr2=0x7ff6bc1403d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:08.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.846+0000 7ff6c1600640 1 -- 172.21.15.52:0/183210844 shutdown_connections 2024-03-12T17:07:08.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:08.846+0000 7ff6c1600640 1 -- 172.21.15.52:0/183210844 wait complete. 2024-03-12T17:07:09.260 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-12T17:06:25.905186+0000', 'modified': '2024-03-12T17:06:54.075752+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4264}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4264': {'gid': 4264, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.102:6837/3333451504', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6835', 'nonce': 3333451504}, {'type': 'v1', 'addr': '172.21.15.102:6837', 'nonce': 3333451504}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-12T17:07:09.261 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-12T17:07:09.261 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-12T17:07:09.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.498+0000 7f25a9800640 1 Processor -- start 2024-03-12T17:07:09.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.498+0000 7f25a9800640 1 -- start start 2024-03-12T17:07:09.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.499+0000 7f25a9800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f25a40707e0 0x7f25a406c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:09.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.499+0000 7f25a9800640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25a415b1a0 con 0x7f25a415b630 2024-03-12T17:07:09.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.499+0000 7f25a9800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25a406d940 con 0x7f25a40ccd40 2024-03-12T17:07:09.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.499+0000 7f25a9800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a415eba0 con 0x7f25a40707e0 2024-03-12T17:07:09.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.500+0000 7f259b400640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f25a415b630 0x7f25a40700d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:33224/0 (socket says 172.21.15.52:33224) 2024-03-12T17:07:09.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.500+0000 7f259b400640 1 -- 172.21.15.52:0/3777453869 learned_addr learned my addr 172.21.15.52:0/3777453869 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:09.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.500+0000 7f25a8e00640 1 --2- 172.21.15.52:0/3777453869 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f25a40707e0 0x7f25a406c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:09.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.500+0000 7f25a3400640 1 -- 172.21.15.52:0/3777453869 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2699740072 0 0) 0x7f25a415b1a0 con 0x7f25a415b630 2024-03-12T17:07:09.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.500+0000 7f25a3400640 1 -- 172.21.15.52:0/3777453869 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2588003520 con 0x7f25a415b630 2024-03-12T17:07:09.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.501+0000 7f25a3400640 1 -- 172.21.15.52:0/3777453869 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2180714965 0 0) 0x7f2588003520 con 0x7f25a415b630 2024-03-12T17:07:09.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.501+0000 7f25a3400640 1 -- 172.21.15.52:0/3777453869 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f25a40707e0 msgr2=0x7f25a406c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:09.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.501+0000 7f25a3400640 1 --2- 172.21.15.52:0/3777453869 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f25a40707e0 0x7f25a406c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:09.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.501+0000 7f25a3400640 1 -- 172.21.15.52:0/3777453869 >> v1:172.21.15.102:6789/0 conn(0x7f25a40ccd40 legacy=0x7f25a415e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:09.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.501+0000 7f25a3400640 1 -- 172.21.15.52:0/3777453869 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a415ab50 con 0x7f25a415b630 2024-03-12T17:07:09.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.501+0000 7f25a3400640 1 -- 172.21.15.52:0/3777453869 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f258c0019f0 con 0x7f25a415b630 2024-03-12T17:07:09.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.501+0000 7f25a3400640 1 -- 172.21.15.52:0/3777453869 >> v1:172.21.15.52:6789/0 conn(0x7f25a415b630 legacy=0x7f25a40700d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:09.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.501+0000 7f25a3400640 1 --2- 172.21.15.52:0/3777453869 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25880039a0 0x7f2588003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:09.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.501+0000 7f25a3400640 1 --2- 172.21.15.52:0/3777453869 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f25a40707e0 0x7f25880047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:09.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.501+0000 7f25a3400640 1 --2- 172.21.15.52:0/3777453869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2588004d00 0x7f2588005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:09.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.501+0000 7f25a3400640 1 -- 172.21.15.52:0/3777453869 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a415eba0 con 0x7f2588004d00 2024-03-12T17:07:09.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.501+0000 7f25a3400640 1 -- 172.21.15.52:0/3777453869 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258800ba60 con 0x7f25880039a0 2024-03-12T17:07:09.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.501+0000 7f25a3400640 1 -- 172.21.15.52:0/3777453869 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258800bc20 con 0x7f25a40707e0 2024-03-12T17:07:09.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.501+0000 7f25a8e00640 1 --2- 172.21.15.52:0/3777453869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2588004d00 0x7f2588005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:09.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.501+0000 7f25a3e00640 1 --2- 172.21.15.52:0/3777453869 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25880039a0 0x7f2588003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:09.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.501+0000 7f259b400640 1 --2- 172.21.15.52:0/3777453869 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f25a40707e0 0x7f25880047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:09.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.502+0000 7f25a8e00640 1 -- 172.21.15.52:0/3777453869 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f25a40707e0 msgr2=0x7f25880047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:09.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.502+0000 7f25a8e00640 1 --2- 172.21.15.52:0/3777453869 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f25a40707e0 0x7f25880047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:09.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.502+0000 7f25a8e00640 1 -- 172.21.15.52:0/3777453869 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25880039a0 msgr2=0x7f2588003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:09.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.502+0000 7f25a8e00640 1 --2- 172.21.15.52:0/3777453869 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25880039a0 0x7f2588003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:09.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.502+0000 7f25a8e00640 1 -- 172.21.15.52:0/3777453869 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2588003520 con 0x7f2588004d00 2024-03-12T17:07:09.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.503+0000 7f25a8e00640 1 --2- 172.21.15.52:0/3777453869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2588004d00 0x7f2588005180 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f259c009ac0 tx=0x7f25880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d50e9f8e152feb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:09.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.503+0000 7f25a3400640 1 -- 172.21.15.52:0/3777453869 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c02c6b0 con 0x7f2588004d00 2024-03-12T17:07:09.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.503+0000 7f25a3400640 1 -- 172.21.15.52:0/3777453869 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259c02c810 con 0x7f2588004d00 2024-03-12T17:07:09.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.503+0000 7f25a3400640 1 -- 172.21.15.52:0/3777453869 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c02cba0 con 0x7f2588004d00 2024-03-12T17:07:09.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.504+0000 7f25a9800640 1 -- 172.21.15.52:0/3777453869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2588004d00 msgr2=0x7f2588005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:09.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.504+0000 7f25a9800640 1 --2- 172.21.15.52:0/3777453869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2588004d00 0x7f2588005180 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f259c009ac0 tx=0x7f25880057f0 comp rx=0 tx=0).stop 2024-03-12T17:07:09.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.504+0000 7f25a8e00640 1 -- 172.21.15.52:0/3777453869 reap_dead start 2024-03-12T17:07:09.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.504+0000 7f25a9800640 1 -- 172.21.15.52:0/3777453869 shutdown_connections 2024-03-12T17:07:09.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.504+0000 7f25a9800640 1 -- 172.21.15.52:0/3777453869 >> 172.21.15.52:0/3777453869 conn(0x7f25a4065df0 msgr2=0x7f25a4069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:09.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.504+0000 7f25a9800640 1 -- 172.21.15.52:0/3777453869 shutdown_connections 2024-03-12T17:07:09.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.504+0000 7f25a9800640 1 -- 172.21.15.52:0/3777453869 wait complete. 2024-03-12T17:07:09.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.506+0000 7f25a9800640 1 Processor -- start 2024-03-12T17:07:09.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.506+0000 7f25a9800640 1 -- start start 2024-03-12T17:07:09.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.507+0000 7f25a9800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2588004d00 0x7f25a40ada30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:09.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.507+0000 7f25a9800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25a4157830 0x7f25a40ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:09.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.507+0000 7f25a9800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25a4157cb0 0x7f25a40abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:09.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.507+0000 7f25a9800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a40adf70 con 0x7f25a4157830 2024-03-12T17:07:09.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.507+0000 7f25a9800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a414fde0 con 0x7f25a4157cb0 2024-03-12T17:07:09.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.507+0000 7f25a9800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a41500c0 con 0x7f2588004d00 2024-03-12T17:07:09.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.507+0000 7f259b400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25a4157830 0x7f25a40ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:09.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.507+0000 7f259b400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25a4157830 0x7f25a40ab8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54248/0 (socket says 172.21.15.52:54248) 2024-03-12T17:07:09.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.507+0000 7f259b400640 1 -- 172.21.15.52:0/968432008 learned_addr learned my addr 172.21.15.52:0/968432008 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:09.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.507+0000 7f25a3e00640 1 --2- 172.21.15.52:0/968432008 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2588004d00 0x7f25a40ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:09.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.507+0000 7f25a8e00640 1 --2- 172.21.15.52:0/968432008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25a4157cb0 0x7f25a40abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:09.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.507+0000 7f259b400640 1 -- 172.21.15.52:0/968432008 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2588004d00 msgr2=0x7f25a40ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:09.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.507+0000 7f259b400640 1 --2- 172.21.15.52:0/968432008 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2588004d00 0x7f25a40ada30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:09.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.507+0000 7f259b400640 1 -- 172.21.15.52:0/968432008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25a4157cb0 msgr2=0x7f25a40abdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:09.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.507+0000 7f259b400640 1 --2- 172.21.15.52:0/968432008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25a4157cb0 0x7f25a40abdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:09.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.507+0000 7f259b400640 1 -- 172.21.15.52:0/968432008 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a41503a0 con 0x7f25a4157830 2024-03-12T17:07:09.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.508+0000 7f25a8e00640 1 --2- 172.21.15.52:0/968432008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25a4157cb0 0x7f25a40abdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:09.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.508+0000 7f259b400640 1 --2- 172.21.15.52:0/968432008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25a4157830 0x7f25a40ab8b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f258c007a80 tx=0x7f258c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:09.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.508+0000 7f25a0c00640 1 -- 172.21.15.52:0/968432008 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c00eeb0 con 0x7f25a4157830 2024-03-12T17:07:09.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.508+0000 7f25a0c00640 1 -- 172.21.15.52:0/968432008 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258c013420 con 0x7f25a4157830 2024-03-12T17:07:09.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.508+0000 7f25a0c00640 1 -- 172.21.15.52:0/968432008 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c0137f0 con 0x7f25a4157830 2024-03-12T17:07:09.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.508+0000 7f25a9800640 1 -- 172.21.15.52:0/968432008 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a4150680 con 0x7f25a4157830 2024-03-12T17:07:09.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.508+0000 7f25a9800640 1 -- 172.21.15.52:0/968432008 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a41515c0 con 0x7f25a4157830 2024-03-12T17:07:09.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.512+0000 7f25a0c00640 1 -- 172.21.15.52:0/968432008 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f258c013950 con 0x7f25a4157830 2024-03-12T17:07:09.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.512+0000 7f25a0c00640 1 --2- 172.21.15.52:0/968432008 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f258006ac00 0x7f258006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:09.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.513+0000 7f25a0c00640 1 -- 172.21.15.52:0/968432008 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f258c089f10 con 0x7f25a4157830 2024-03-12T17:07:09.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.513+0000 7f25ad97f740 1 Processor -- start 2024-03-12T17:07:09.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.513+0000 7f25ad97f740 1 -- start start 2024-03-12T17:07:09.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.513+0000 7f25a3e00640 1 --2- 172.21.15.52:0/968432008 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f258006ac00 0x7f258006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:09.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.514+0000 7f25ad97f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626f5120200 0x5626f50d7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:09.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.514+0000 7f25ad97f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f50d6280 0x5626f50d7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:09.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.514+0000 7f25ad97f740 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5626f50d2720 0x5626f5102510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:09.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.514+0000 7f25ad97f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f50ade40 con 0x5626f50d6280 2024-03-12T17:07:09.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.514+0000 7f25ad97f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f5043090 con 0x5626f5120200 2024-03-12T17:07:09.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.514+0000 7f25ad97f740 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626f5040820 con 0x5626f50d2720 2024-03-12T17:07:09.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.514+0000 7f259b400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f50d6280 0x5626f50d7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:09.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.514+0000 7f25a3e00640 1 --2- 172.21.15.52:0/968432008 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f258006ac00 0x7f258006d0c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2594005500 tx=0x7f2594007f10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:09.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.514+0000 7f259b400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f50d6280 0x5626f50d7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54252/0 (socket says 172.21.15.52:54252) 2024-03-12T17:07:09.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.514+0000 7f259b400640 1 -- 172.21.15.52:0/2430306701 learned_addr learned my addr 172.21.15.52:0/2430306701 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:09.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.514+0000 7f25a3e00640 1 --2- 172.21.15.52:0/2430306701 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626f5120200 0x5626f50d7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:09.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.515+0000 7f259b400640 1 -- 172.21.15.52:0/2430306701 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5626f50d2720 msgr2=0x5626f5102510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:09.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.515+0000 7f259b400640 1 --2- 172.21.15.52:0/2430306701 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5626f50d2720 0x5626f5102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:09.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.515+0000 7f259b400640 1 -- 172.21.15.52:0/2430306701 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626f5120200 msgr2=0x5626f50d7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:09.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.515+0000 7f259b400640 1 --2- 172.21.15.52:0/2430306701 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626f5120200 0x5626f50d7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:09.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.515+0000 7f259b400640 1 -- 172.21.15.52:0/2430306701 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626f5003780 con 0x5626f50d6280 2024-03-12T17:07:09.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.515+0000 7f259b400640 1 --2- 172.21.15.52:0/2430306701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f50d6280 0x5626f50d7d70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f258c054f20 tx=0x7f258c023310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3efd4985d1d796a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:09.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.515+0000 7f2598c00640 1 -- 172.21.15.52:0/2430306701 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c05c850 con 0x5626f50d6280 2024-03-12T17:07:09.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.515+0000 7f2598c00640 1 -- 172.21.15.52:0/2430306701 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258c060e10 con 0x5626f50d6280 2024-03-12T17:07:09.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.515+0000 7f2598c00640 1 -- 172.21.15.52:0/2430306701 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c055130 con 0x5626f50d6280 2024-03-12T17:07:09.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.516+0000 7f25ad97f740 1 -- 172.21.15.52:0/2430306701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f50d6280 msgr2=0x5626f50d7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:09.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.516+0000 7f25ad97f740 1 --2- 172.21.15.52:0/2430306701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f50d6280 0x5626f50d7d70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f258c054f20 tx=0x7f258c023310 comp rx=0 tx=0).stop 2024-03-12T17:07:09.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.516+0000 7f25ad97f740 1 -- 172.21.15.52:0/2430306701 shutdown_connections 2024-03-12T17:07:09.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.516+0000 7f25ad97f740 1 --2- 172.21.15.52:0/2430306701 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5626f50d2720 0x5626f5102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:09.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.516+0000 7f25ad97f740 1 --2- 172.21.15.52:0/2430306701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f50d6280 0x5626f50d7d70 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:09.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.516+0000 7f25ad97f740 1 --2- 172.21.15.52:0/2430306701 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626f5120200 0x5626f50d7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:09.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.516+0000 7f25ad97f740 1 -- 172.21.15.52:0/2430306701 >> 172.21.15.52:0/2430306701 conn(0x5626f50d4990 msgr2=0x5626f511d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:09.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.516+0000 7f25ad97f740 1 -- 172.21.15.52:0/2430306701 shutdown_connections 2024-03-12T17:07:09.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.516+0000 7f25ad97f740 1 -- 172.21.15.52:0/2430306701 wait complete. 2024-03-12T17:07:09.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.517+0000 7f25ad97f740 1 -- start start 2024-03-12T17:07:09.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.517+0000 7f25ad97f740 1 Processor -- start 2024-03-12T17:07:09.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.518+0000 7f25ad97f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f50d6280 0x5626f50f9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:09.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.518+0000 7f25ad97f740 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5626f5120200 0x5626f5100460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:09.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.518+0000 7f25ad97f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626f50f9820 0x5626f50f9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:09.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.518+0000 7f25ad97f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f503c110 con 0x5626f50d6280 2024-03-12T17:07:09.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.518+0000 7f25ad97f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f4fdd8c0 con 0x5626f50f9820 2024-03-12T17:07:09.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.518+0000 7f25ad97f740 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626f4fdd560 con 0x5626f5120200 2024-03-12T17:07:09.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.518+0000 7f25ad97f740 10 client.0 _ll_register_callbacks cb 0x5626f50358d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T17:07:09.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.518+0000 7f25a3e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f50d6280 0x5626f50f9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:09.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.518+0000 7f25a3e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f50d6280 0x5626f50f9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54266/0 (socket says 172.21.15.52:54266) 2024-03-12T17:07:09.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.518+0000 7f25a3e00640 1 -- 172.21.15.52:0/3382366882 learned_addr learned my addr 172.21.15.52:0/3382366882 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:09.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.518+0000 7f25a8e00640 1 --2- 172.21.15.52:0/3382366882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626f50f9820 0x5626f50f9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:09.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.518+0000 7f259b400640 1 --2- 172.21.15.52:0/3382366882 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5626f5120200 0x5626f5100460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:09.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.518+0000 7f25a3e00640 1 -- 172.21.15.52:0/3382366882 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5626f5120200 msgr2=0x5626f5100460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:09.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.518+0000 7f25a3e00640 1 --2- 172.21.15.52:0/3382366882 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5626f5120200 0x5626f5100460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:09.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.518+0000 7f25a3e00640 1 -- 172.21.15.52:0/3382366882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626f50f9820 msgr2=0x5626f50f9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:09.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.518+0000 7f25a3e00640 1 --2- 172.21.15.52:0/3382366882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626f50f9820 0x5626f50f9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:09.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.518+0000 7f25a3e00640 1 -- 172.21.15.52:0/3382366882 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a41515c0 con 0x5626f50d6280 2024-03-12T17:07:09.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.518+0000 7f25a8e00640 1 --2- 172.21.15.52:0/3382366882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626f50f9820 0x5626f50f9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:09.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.519+0000 7f25a3e00640 1 --2- 172.21.15.52:0/3382366882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f50d6280 0x5626f50f9110 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2594001cd0 tx=0x7f2594001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b4ec36e92fd378 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:09.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.519+0000 7f2591600640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-03-12T17:07:09.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.519+0000 7f2591600640 1 -- 172.21.15.52:0/3382366882 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2594027070 con 0x5626f50d6280 2024-03-12T17:07:09.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.519+0000 7f2591600640 1 -- 172.21.15.52:0/3382366882 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f259401c070 con 0x5626f50d6280 2024-03-12T17:07:09.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.519+0000 7f25ad97f740 1 -- 172.21.15.52:0/3382366882 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626f5066660 con 0x5626f50d6280 2024-03-12T17:07:09.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.519+0000 7f2591600640 1 -- 172.21.15.52:0/3382366882 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f259402d410 con 0x5626f50d6280 2024-03-12T17:07:09.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.519+0000 7f25ad97f740 10 client.4857 fetch_fsmap learned FSMap version 9 2024-03-12T17:07:09.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.519+0000 7f25ad97f740 1 -- 172.21.15.52:0/3382366882 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x5626f503a790 con 0x5626f50d6280 2024-03-12T17:07:09.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.520+0000 7f2591600640 1 -- 172.21.15.52:0/3382366882 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2594031020 con 0x5626f50d6280 2024-03-12T17:07:09.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.520+0000 7f2591600640 1 -- 172.21.15.52:0/3382366882 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 9) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f259402a020 con 0x5626f50d6280 2024-03-12T17:07:09.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.520+0000 7f25ad97f740 10 client.4857 fetch_fsmap finished waiting for FSMap version 9 2024-03-12T17:07:09.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.520+0000 7f25ad97f740 10 client.4857 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-12T17:07:09.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.520+0000 7f25ad97f740 20 client.4857 populate_metadata read hostname 'smithi052' 2024-03-12T17:07:09.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.520+0000 7f25ad97f740 1 --2- 172.21.15.52:0/3382366882 >> [v2:172.21.15.102:6835/3333451504,v1:172.21.15.102:6837/3333451504] conn(0x5626f5138950 0x5626f513ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:09.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.520+0000 7f25ad97f740 4 client.4857 mds_command: new command op to 4264 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T17:07:09.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.520+0000 7f25ad97f740 1 -- 172.21.15.52:0/3382366882 --> [v2:172.21.15.102:6835/3333451504,v1:172.21.15.102:6837/3333451504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626f502ee50 con 0x5626f5138950 2024-03-12T17:07:09.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.521+0000 7f259b400640 1 --2- 172.21.15.52:0/3382366882 >> [v2:172.21.15.102:6835/3333451504,v1:172.21.15.102:6837/3333451504] conn(0x5626f5138950 0x5626f513ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:09.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.521+0000 7f259b400640 1 --2- 172.21.15.52:0/3382366882 >> [v2:172.21.15.102:6835/3333451504,v1:172.21.15.102:6837/3333451504] conn(0x5626f5138950 0x5626f513ad40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b13361bd83e0fc3b server_cookie=6385b249aee2d413 in_seq=0 out_seq=0 2024-03-12T17:07:09.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.521+0000 7f2591600640 10 client.4857 ms_handle_connect on v2:172.21.15.102:6835/3333451504 2024-03-12T17:07:09.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.523+0000 7f2591600640 1 -- 172.21.15.52:0/3382366882 <== mds.0 v2:172.21.15.102:6835/3333451504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5626f502ee50 con 0x5626f5138950 2024-03-12T17:07:09.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.523+0000 7f2591600640 10 client.4857 handle_command_reply: tid=0 2024-03-12T17:07:09.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.523+0000 7f25ad97f740 1 client.4857 shutdown 2024-03-12T17:07:09.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.523+0000 7f25ad97f740 1 -- 172.21.15.52:0/3382366882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f50d6280 msgr2=0x5626f50f9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:09.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.523+0000 7f25ad97f740 1 --2- 172.21.15.52:0/3382366882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f50d6280 0x5626f50f9110 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2594001cd0 tx=0x7f2594001230 comp rx=0 tx=0).stop 2024-03-12T17:07:09.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.523+0000 7f25ad97f740 1 -- 172.21.15.52:0/3382366882 shutdown_connections 2024-03-12T17:07:09.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.523+0000 7f25ad97f740 1 --2- 172.21.15.52:0/3382366882 >> [v2:172.21.15.102:6835/3333451504,v1:172.21.15.102:6837/3333451504] conn(0x5626f5138950 0x5626f513ad40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:09.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.523+0000 7f25ad97f740 1 --2- 172.21.15.52:0/3382366882 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5626f5120200 0x5626f5100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:09.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.523+0000 7f25ad97f740 1 --2- 172.21.15.52:0/3382366882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626f50f9820 0x5626f50f9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:09.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.523+0000 7f25ad97f740 1 --2- 172.21.15.52:0/3382366882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f50d6280 0x5626f50f9110 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:09.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.523+0000 7f2591600640 0 client.4857 ms_handle_reset on v2:172.21.15.102:6835/3333451504 2024-03-12T17:07:09.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.523+0000 7f25ad97f740 1 -- 172.21.15.52:0/3382366882 >> 172.21.15.52:0/3382366882 conn(0x5626f50d2720 msgr2=0x5626f50d1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:09.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.524+0000 7f25ad97f740 1 -- 172.21.15.52:0/3382366882 shutdown_connections 2024-03-12T17:07:09.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.524+0000 7f25ad97f740 1 -- 172.21.15.52:0/3382366882 wait complete. 2024-03-12T17:07:09.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.524+0000 7f25ad97f740 20 client.4857 trim_cache size 0 max 16384 2024-03-12T17:07:09.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.540+0000 7f25ad97f740 1 Processor -- start 2024-03-12T17:07:09.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.541+0000 7f25ad97f740 1 -- start start 2024-03-12T17:07:09.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.541+0000 7f25ad97f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626f50d6280 0x5626f50ff020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:09.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.541+0000 7f25ad97f740 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5626f50f9820 0x5626f50d3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:09.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.541+0000 7f25ad97f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f5120200 0x5626f50f9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:09.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.541+0000 7f25ad97f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f4da3f20 con 0x5626f5120200 2024-03-12T17:07:09.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.541+0000 7f25ad97f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f511bf80 con 0x5626f50d6280 2024-03-12T17:07:09.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.541+0000 7f25ad97f740 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626f51063f0 con 0x5626f50f9820 2024-03-12T17:07:09.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.542+0000 7f25a8e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f5120200 0x5626f50f9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:09.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.542+0000 7f25a8e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f5120200 0x5626f50f9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54274/0 (socket says 172.21.15.52:54274) 2024-03-12T17:07:09.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.542+0000 7f25a8e00640 1 -- 172.21.15.52:0/261537751 learned_addr learned my addr 172.21.15.52:0/261537751 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:09.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.542+0000 7f25a3e00640 1 --2- 172.21.15.52:0/261537751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626f50d6280 0x5626f50ff020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:09.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.542+0000 7f259b400640 1 --2- 172.21.15.52:0/261537751 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5626f50f9820 0x5626f50d3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:09.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.542+0000 7f25a8e00640 1 -- 172.21.15.52:0/261537751 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5626f50f9820 msgr2=0x5626f50d3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:09.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.542+0000 7f25a8e00640 1 --2- 172.21.15.52:0/261537751 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5626f50f9820 0x5626f50d3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:09.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.542+0000 7f25a8e00640 1 -- 172.21.15.52:0/261537751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626f50d6280 msgr2=0x5626f50ff020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:09.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.542+0000 7f25a8e00640 1 --2- 172.21.15.52:0/261537751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626f50d6280 0x5626f50ff020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:09.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.542+0000 7f25a8e00640 1 -- 172.21.15.52:0/261537751 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626f4fdcad0 con 0x5626f5120200 2024-03-12T17:07:09.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.543+0000 7f25a8e00640 1 --2- 172.21.15.52:0/261537751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f5120200 0x5626f50f9d40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f259c006f80 tx=0x7f259c02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3781fdd992fa3b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:09.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.543+0000 7f2599600640 1 -- 172.21.15.52:0/261537751 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c004240 con 0x5626f5120200 2024-03-12T17:07:09.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.543+0000 7f2599600640 1 -- 172.21.15.52:0/261537751 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259c0043a0 con 0x5626f5120200 2024-03-12T17:07:09.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.543+0000 7f2599600640 1 -- 172.21.15.52:0/261537751 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c02cb80 con 0x5626f5120200 2024-03-12T17:07:09.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.543+0000 7f25ad97f740 1 -- 172.21.15.52:0/261537751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f5120200 msgr2=0x5626f50f9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:09.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.543+0000 7f25ad97f740 1 --2- 172.21.15.52:0/261537751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f5120200 0x5626f50f9d40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f259c006f80 tx=0x7f259c02e070 comp rx=0 tx=0).stop 2024-03-12T17:07:09.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.544+0000 7f25ad97f740 1 -- 172.21.15.52:0/261537751 shutdown_connections 2024-03-12T17:07:09.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.544+0000 7f25ad97f740 1 --2- 172.21.15.52:0/261537751 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5626f50f9820 0x5626f50d3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:09.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.544+0000 7f25ad97f740 1 --2- 172.21.15.52:0/261537751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f5120200 0x5626f50f9d40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:09.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.544+0000 7f25ad97f740 1 --2- 172.21.15.52:0/261537751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626f50d6280 0x5626f50ff020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:09.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.544+0000 7f25ad97f740 1 -- 172.21.15.52:0/261537751 >> 172.21.15.52:0/261537751 conn(0x5626f50d2720 msgr2=0x5626f50f8050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:09.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.544+0000 7f25ad97f740 1 -- 172.21.15.52:0/261537751 shutdown_connections 2024-03-12T17:07:09.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.544+0000 7f25ad97f740 1 -- 172.21.15.52:0/261537751 wait complete. 2024-03-12T17:07:09.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.544+0000 7f25ad97f740 1 -- start start 2024-03-12T17:07:09.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.545+0000 7f25ad97f740 1 Processor -- start 2024-03-12T17:07:09.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.545+0000 7f25ad97f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f50d6280 0x5626f511f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:09.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.545+0000 7f25ad97f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626f50f9820 0x5626f512fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:09.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.545+0000 7f25ad97f740 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5626f5120200 0x5626f50d1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:09.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.545+0000 7f25ad97f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f511f600 con 0x5626f50d6280 2024-03-12T17:07:09.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.545+0000 7f25ad97f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f5124180 con 0x5626f50f9820 2024-03-12T17:07:09.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.545+0000 7f25ad97f740 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626f50d8e90 con 0x5626f5120200 2024-03-12T17:07:09.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.546+0000 7f25ad97f740 10 client.0 _ll_register_callbacks cb 0x5626f50fff60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T17:07:09.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.546+0000 7f25a3e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f50d6280 0x5626f511f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:09.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.546+0000 7f25a3e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f50d6280 0x5626f511f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54276/0 (socket says 172.21.15.52:54276) 2024-03-12T17:07:09.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.546+0000 7f25a3e00640 1 -- 172.21.15.52:0/68815824 learned_addr learned my addr 172.21.15.52:0/68815824 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:09.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.546+0000 7f259b400640 1 --2- 172.21.15.52:0/68815824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626f50f9820 0x5626f512fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:09.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.546+0000 7f25a8e00640 1 --2- 172.21.15.52:0/68815824 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5626f5120200 0x5626f50d1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:09.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.546+0000 7f25a3e00640 1 -- 172.21.15.52:0/68815824 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5626f5120200 msgr2=0x5626f50d1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:09.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.546+0000 7f25a3e00640 1 --2- 172.21.15.52:0/68815824 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5626f5120200 0x5626f50d1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:09.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.546+0000 7f25a3e00640 1 -- 172.21.15.52:0/68815824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626f50f9820 msgr2=0x5626f512fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:09.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.546+0000 7f25a3e00640 1 --2- 172.21.15.52:0/68815824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626f50f9820 0x5626f512fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:09.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.546+0000 7f25a3e00640 1 -- 172.21.15.52:0/68815824 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2588003520 con 0x5626f50d6280 2024-03-12T17:07:09.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.546+0000 7f25a8e00640 1 --2- 172.21.15.52:0/68815824 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5626f5120200 0x5626f50d1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:09.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.547+0000 7f25a3e00640 1 --2- 172.21.15.52:0/68815824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f50d6280 0x5626f511f0c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2594009010 tx=0x7f2594021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e1fdf34648beec server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:09.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.547+0000 7f2591600640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-03-12T17:07:09.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.547+0000 7f2591600640 1 -- 172.21.15.52:0/68815824 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2594027040 con 0x5626f50d6280 2024-03-12T17:07:09.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.547+0000 7f2591600640 1 -- 172.21.15.52:0/68815824 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2594032080 con 0x5626f50d6280 2024-03-12T17:07:09.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.547+0000 7f25ad97f740 1 -- 172.21.15.52:0/68815824 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f258c060e10 con 0x5626f50d6280 2024-03-12T17:07:09.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.547+0000 7f2591600640 1 -- 172.21.15.52:0/68815824 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2594022400 con 0x5626f50d6280 2024-03-12T17:07:09.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.547+0000 7f25ad97f740 10 client.4863 fetch_fsmap learned FSMap version 9 2024-03-12T17:07:09.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.547+0000 7f25ad97f740 1 -- 172.21.15.52:0/68815824 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x5626f5001fe0 con 0x5626f50d6280 2024-03-12T17:07:09.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.547+0000 7f2591600640 1 -- 172.21.15.52:0/68815824 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258c060e10 con 0x5626f50d6280 2024-03-12T17:07:09.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.547+0000 7f2591600640 1 -- 172.21.15.52:0/68815824 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 9) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f259402a040 con 0x5626f50d6280 2024-03-12T17:07:09.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.547+0000 7f25ad97f740 10 client.4863 fetch_fsmap finished waiting for FSMap version 9 2024-03-12T17:07:09.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.547+0000 7f25ad97f740 10 client.4863 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-12T17:07:09.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.547+0000 7f25ad97f740 20 client.4863 populate_metadata read hostname 'smithi052' 2024-03-12T17:07:09.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.547+0000 7f25ad97f740 1 --2- 172.21.15.52:0/68815824 >> [v2:172.21.15.102:6835/3333451504,v1:172.21.15.102:6837/3333451504] conn(0x5626f51300a0 0x5626f50d1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:09.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.548+0000 7f25ad97f740 4 client.4863 mds_command: new command op to 4264 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-12T17:07:09.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.548+0000 7f25ad97f740 1 -- 172.21.15.52:0/68815824 --> [v2:172.21.15.102:6835/3333451504,v1:172.21.15.102:6837/3333451504] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5626f4d303c0 con 0x5626f51300a0 2024-03-12T17:07:09.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.548+0000 7f259b400640 1 --2- 172.21.15.52:0/68815824 >> [v2:172.21.15.102:6835/3333451504,v1:172.21.15.102:6837/3333451504] conn(0x5626f51300a0 0x5626f50d1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:09.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.549+0000 7f259b400640 1 --2- 172.21.15.52:0/68815824 >> [v2:172.21.15.102:6835/3333451504,v1:172.21.15.102:6837/3333451504] conn(0x5626f51300a0 0x5626f50d1d60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1732f113c6452c5 server_cookie=6fa647ab5cbf34d8 in_seq=0 out_seq=0 2024-03-12T17:07:09.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.549+0000 7f2591600640 10 client.4863 ms_handle_connect on v2:172.21.15.102:6835/3333451504 2024-03-12T17:07:09.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.549+0000 7f2591600640 1 -- 172.21.15.52:0/68815824 <== mds.0 v2:172.21.15.102:6835/3333451504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x5626f4d303c0 con 0x5626f51300a0 2024-03-12T17:07:09.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.549+0000 7f2591600640 10 client.4863 handle_command_reply: tid=0 2024-03-12T17:07:09.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.549+0000 7f25ad97f740 1 client.4863 shutdown 2024-03-12T17:07:09.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.550+0000 7f25ad97f740 1 -- 172.21.15.52:0/68815824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f50d6280 msgr2=0x5626f511f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:09.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.550+0000 7f25ad97f740 1 --2- 172.21.15.52:0/68815824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f50d6280 0x5626f511f0c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2594009010 tx=0x7f2594021000 comp rx=0 tx=0).stop 2024-03-12T17:07:09.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.550+0000 7f25ad97f740 1 -- 172.21.15.52:0/68815824 shutdown_connections 2024-03-12T17:07:09.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.550+0000 7f25ad97f740 1 --2- 172.21.15.52:0/68815824 >> [v2:172.21.15.102:6835/3333451504,v1:172.21.15.102:6837/3333451504] conn(0x5626f51300a0 0x5626f50d1d60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:09.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.550+0000 7f25ad97f740 1 --2- 172.21.15.52:0/68815824 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5626f5120200 0x5626f50d1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:09.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.550+0000 7f25ad97f740 1 --2- 172.21.15.52:0/68815824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626f50f9820 0x5626f512fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:09.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.550+0000 7f2591600640 0 client.4863 ms_handle_reset on v2:172.21.15.102:6835/3333451504 2024-03-12T17:07:09.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.550+0000 7f25ad97f740 1 --2- 172.21.15.52:0/68815824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f50d6280 0x5626f511f0c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:09.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.550+0000 7f25ad97f740 1 -- 172.21.15.52:0/68815824 >> 172.21.15.52:0/68815824 conn(0x5626f50d2720 msgr2=0x5626f50f8050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:09.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.550+0000 7f25ad97f740 1 -- 172.21.15.52:0/68815824 shutdown_connections 2024-03-12T17:07:09.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.550+0000 7f25ad97f740 1 -- 172.21.15.52:0/68815824 wait complete. 2024-03-12T17:07:09.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.550+0000 7f25ad97f740 20 client.4863 trim_cache size 0 max 16384 2024-03-12T17:07:09.549 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-03-12T17:07:09.549 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3", 2024-03-12T17:07:09.549 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 0, 2024-03-12T17:07:09.549 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 4264, 2024-03-12T17:07:09.549 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-03-12T17:07:09.549 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-03-12T17:07:09.549 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-03-12T17:07:09.549 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 43.638600087999997, 2024-03-12T17:07:09.549 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 9, 2024-03-12T17:07:09.549 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 25, 2024-03-12T17:07:09.549 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 25, 2024-03-12T17:07:09.550 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 51.179385060999998 2024-03-12T17:07:09.550 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-03-12T17:07:09.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.551+0000 7f25a9800640 1 -- 172.21.15.52:0/968432008 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f258006ac00 msgr2=0x7f258006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:09.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.551+0000 7f25a9800640 1 --2- 172.21.15.52:0/968432008 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f258006ac00 0x7f258006d0c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2594005500 tx=0x7f2594007f10 comp rx=0 tx=0).stop 2024-03-12T17:07:09.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.551+0000 7f25a9800640 1 -- 172.21.15.52:0/968432008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25a4157830 msgr2=0x7f25a40ab8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:09.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.551+0000 7f25a9800640 1 --2- 172.21.15.52:0/968432008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25a4157830 0x7f25a40ab8b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f258c007a80 tx=0x7f258c00e530 comp rx=0 tx=0).stop 2024-03-12T17:07:09.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.551+0000 7f25a9800640 1 -- 172.21.15.52:0/968432008 shutdown_connections 2024-03-12T17:07:09.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.551+0000 7f25a9800640 1 --2- 172.21.15.52:0/968432008 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f258006ac00 0x7f258006d0c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:09.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.551+0000 7f25a9800640 1 --2- 172.21.15.52:0/968432008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25a4157cb0 0x7f25a40abdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:09.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.551+0000 7f25a9800640 1 --2- 172.21.15.52:0/968432008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25a4157830 0x7f25a40ab8b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:09.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.551+0000 7f25a9800640 1 --2- 172.21.15.52:0/968432008 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2588004d00 0x7f25a40ada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:09.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.551+0000 7f25a9800640 1 -- 172.21.15.52:0/968432008 >> 172.21.15.52:0/968432008 conn(0x7f25a4065df0 msgr2=0x7f25a40ae3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:09.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.551+0000 7f25a9800640 1 -- 172.21.15.52:0/968432008 shutdown_connections 2024-03-12T17:07:09.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:09.551+0000 7f25a9800640 1 -- 172.21.15.52:0/968432008 wait complete. 2024-03-12T17:07:09.563 INFO:teuthology.run_tasks:Running task kclient... 2024-03-12T17:07:09.577 INFO:tasks.kclient:Mounting kernel clients... 2024-03-12T17:07:09.578 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-03-12T17:07:09.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:09.578 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-03-12T17:07:09.635 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-03-12T17:07:09.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:09.637 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-12T17:07:09.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:09.743 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link delete ceph-brx 2024-03-12T17:07:09.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:09.873 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-03-12T17:07:09.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:09.917 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link delete ceph-brx 2024-03-12T17:07:10.060 INFO:teuthology.orchestra.run.smithi052.stderr:Cannot find device "ceph-brx" 2024-03-12T17:07:10.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T17:07:10.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:10.065 DEBUG:teuthology.orchestra.run.smithi102:> ip netns list 2024-03-12T17:07:10.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:10.121 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link delete ceph-brx 2024-03-12T17:07:10.213 INFO:teuthology.orchestra.run.smithi102.stderr:Cannot find device "ceph-brx" 2024-03-12T17:07:10.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T17:07:10.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:10.218 DEBUG:teuthology.orchestra.run.smithi102:> ip netns list 2024-03-12T17:07:10.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:10.270 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link delete ceph-brx 2024-03-12T17:07:10.364 INFO:teuthology.orchestra.run.smithi102.stderr:Cannot find device "ceph-brx" 2024-03-12T17:07:10.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T17:07:10.369 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-03-12T17:07:10.369 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-12T17:07:10.369 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-12T17:07:10.369 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-12T17:07:10.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.601+0000 7f1ac4a00640 1 Processor -- start 2024-03-12T17:07:10.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.602+0000 7f1ac4a00640 1 -- start start 2024-03-12T17:07:10.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.603+0000 7f1ac4a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1ac0156e30 0x7f1ac006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:10.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.603+0000 7f1ac4a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ac00cb540 0x7f1ac00b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:10.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.603+0000 7f1ac4a00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ac00cc8c0 con 0x7f1ac00b3310 2024-03-12T17:07:10.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.603+0000 7f1ac4a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ac006d630 con 0x7f1ac0156e30 2024-03-12T17:07:10.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.603+0000 7f1ac4a00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac015ce60 con 0x7f1ac00cb540 2024-03-12T17:07:10.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.604+0000 7f1abde00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f1ac00b3310 0x7f1ac00cae30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:33240/0 (socket says 172.21.15.52:33240) 2024-03-12T17:07:10.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.604+0000 7f1abde00640 1 -- 172.21.15.52:0/3910360982 learned_addr learned my addr 172.21.15.52:0/3910360982 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:10.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.604+0000 7f1abe800640 1 --2- 172.21.15.52:0/3910360982 >> v2:172.21.15.102:3300/0 conn(0x7f1ac0156e30 0x7f1ac006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:10.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.604+0000 7f1abf200640 1 --2- 172.21.15.52:0/3910360982 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ac00cb540 0x7f1ac00b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:10.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.604+0000 7f1abd400640 1 -- 172.21.15.52:0/3910360982 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4261963756 0 0) 0x7f1ac00cc8c0 con 0x7f1ac00b3310 2024-03-12T17:07:10.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.604+0000 7f1abd400640 1 -- 172.21.15.52:0/3910360982 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1aa4003520 con 0x7f1ac00b3310 2024-03-12T17:07:10.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.605+0000 7f1abd400640 1 -- 172.21.15.52:0/3910360982 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3262966123 0 0) 0x7f1aa4003520 con 0x7f1ac00b3310 2024-03-12T17:07:10.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.605+0000 7f1abd400640 1 -- 172.21.15.52:0/3910360982 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ac00cb540 msgr2=0x7f1ac00b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:10.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.605+0000 7f1abd400640 1 --2- 172.21.15.52:0/3910360982 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ac00cb540 0x7f1ac00b1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:10.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.605+0000 7f1abd400640 1 -- 172.21.15.52:0/3910360982 >> v2:172.21.15.102:3300/0 conn(0x7f1ac0156e30 msgr2=0x7f1ac006d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:10.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.605+0000 7f1abd400640 1 --2- 172.21.15.52:0/3910360982 >> v2:172.21.15.102:3300/0 conn(0x7f1ac0156e30 0x7f1ac006d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:10.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.605+0000 7f1abd400640 1 -- 172.21.15.52:0/3910360982 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac015d120 con 0x7f1ac00b3310 2024-03-12T17:07:10.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.605+0000 7f1abd400640 1 -- 172.21.15.52:0/3910360982 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f1aac0019f0 con 0x7f1ac00b3310 2024-03-12T17:07:10.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.605+0000 7f1abd400640 1 -- 172.21.15.52:0/3910360982 >> v1:172.21.15.52:6789/0 conn(0x7f1ac00b3310 legacy=0x7f1ac00cae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:10.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.605+0000 7f1abd400640 1 --2- 172.21.15.52:0/3910360982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1aa4003970 0x7f1aa4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:10.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.605+0000 7f1abd400640 1 --2- 172.21.15.52:0/3910360982 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1aa40047c0 0x7f1aa4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:10.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.605+0000 7f1abd400640 1 --2- 172.21.15.52:0/3910360982 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ac00cb540 0x7f1aa4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:10.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.605+0000 7f1abd400640 1 -- 172.21.15.52:0/3910360982 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac006d630 con 0x7f1aa4003970 2024-03-12T17:07:10.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.605+0000 7f1abd400640 1 -- 172.21.15.52:0/3910360982 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac015ce60 con 0x7f1aa40047c0 2024-03-12T17:07:10.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.605+0000 7f1abd400640 1 -- 172.21.15.52:0/3910360982 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa400b980 con 0x7f1ac00cb540 2024-03-12T17:07:10.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.605+0000 7f1abe800640 1 --2- 172.21.15.52:0/3910360982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1aa4003970 0x7f1aa4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:10.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.605+0000 7f1abe800640 1 -- 172.21.15.52:0/3910360982 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ac00cb540 msgr2=0x7f1aa4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:10.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.605+0000 7f1abf200640 1 --2- 172.21.15.52:0/3910360982 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ac00cb540 0x7f1aa4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:10.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.606+0000 7f1abe800640 1 --2- 172.21.15.52:0/3910360982 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ac00cb540 0x7f1aa4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:10.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.606+0000 7f1abde00640 1 --2- 172.21.15.52:0/3910360982 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1aa40047c0 0x7f1aa4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:10.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.606+0000 7f1abe800640 1 -- 172.21.15.52:0/3910360982 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1aa40047c0 msgr2=0x7f1aa4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:10.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.606+0000 7f1abe800640 1 --2- 172.21.15.52:0/3910360982 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1aa40047c0 0x7f1aa4004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:10.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.606+0000 7f1abe800640 1 -- 172.21.15.52:0/3910360982 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa4003520 con 0x7f1aa4003970 2024-03-12T17:07:10.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.607+0000 7f1abe800640 1 --2- 172.21.15.52:0/3910360982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1aa4003970 0x7f1aa4003d90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0009ac0 tx=0x7f1aa40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a1abf27dba168b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:10.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.607+0000 7f1abd400640 1 -- 172.21.15.52:0/3910360982 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab002c710 con 0x7f1aa4003970 2024-03-12T17:07:10.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.607+0000 7f1abd400640 1 -- 172.21.15.52:0/3910360982 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab002c870 con 0x7f1aa4003970 2024-03-12T17:07:10.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.607+0000 7f1abd400640 1 -- 172.21.15.52:0/3910360982 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab002cc00 con 0x7f1aa4003970 2024-03-12T17:07:10.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.608+0000 7f1ac4a00640 1 -- 172.21.15.52:0/3910360982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1aa4003970 msgr2=0x7f1aa4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:10.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.608+0000 7f1ac4a00640 1 --2- 172.21.15.52:0/3910360982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1aa4003970 0x7f1aa4003d90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0009ac0 tx=0x7f1aa40057f0 comp rx=0 tx=0).stop 2024-03-12T17:07:10.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.608+0000 7f1abf200640 1 -- 172.21.15.52:0/3910360982 reap_dead start 2024-03-12T17:07:10.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.608+0000 7f1ac4a00640 1 -- 172.21.15.52:0/3910360982 shutdown_connections 2024-03-12T17:07:10.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.608+0000 7f1ac4a00640 1 -- 172.21.15.52:0/3910360982 >> 172.21.15.52:0/3910360982 conn(0x7f1ac0065df0 msgr2=0x7f1ac0069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:10.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.608+0000 7f1ac4a00640 1 -- 172.21.15.52:0/3910360982 shutdown_connections 2024-03-12T17:07:10.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.608+0000 7f1ac4a00640 1 -- 172.21.15.52:0/3910360982 wait complete. 2024-03-12T17:07:10.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.610+0000 7f1ac4a00640 1 Processor -- start 2024-03-12T17:07:10.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.610+0000 7f1ac4a00640 1 -- start start 2024-03-12T17:07:10.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.611+0000 7f1ac4a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1aa4003970 0x7f1ac00afc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:10.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.611+0000 7f1ac4a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ac00aee10 0x7f1ac015bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:10.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.611+0000 7f1ac4a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ac015f050 0x7f1ac015f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:10.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.611+0000 7f1ac4a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac015fa20 con 0x7f1aa4003970 2024-03-12T17:07:10.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.611+0000 7f1ac4a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac015b790 con 0x7f1ac00aee10 2024-03-12T17:07:10.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.611+0000 7f1ac4a00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac00b01d0 con 0x7f1ac015f050 2024-03-12T17:07:10.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.611+0000 7f1abe800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1aa4003970 0x7f1ac00afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:10.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.611+0000 7f1abe800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1aa4003970 0x7f1ac00afc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54306/0 (socket says 172.21.15.52:54306) 2024-03-12T17:07:10.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.611+0000 7f1abe800640 1 -- 172.21.15.52:0/1413374855 learned_addr learned my addr 172.21.15.52:0/1413374855 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:10.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.611+0000 7f1abde00640 1 --2- 172.21.15.52:0/1413374855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ac00aee10 0x7f1ac015bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:10.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.611+0000 7f1abf200640 1 --2- 172.21.15.52:0/1413374855 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ac015f050 0x7f1ac015f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:10.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.612+0000 7f1abe800640 1 -- 172.21.15.52:0/1413374855 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ac015f050 msgr2=0x7f1ac015f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:10.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.612+0000 7f1abe800640 1 --2- 172.21.15.52:0/1413374855 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ac015f050 0x7f1ac015f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:10.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.612+0000 7f1abe800640 1 -- 172.21.15.52:0/1413374855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ac00aee10 msgr2=0x7f1ac015bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:10.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.612+0000 7f1abe800640 1 --2- 172.21.15.52:0/1413374855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ac00aee10 0x7f1ac015bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:10.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.612+0000 7f1abe800640 1 -- 172.21.15.52:0/1413374855 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac015ab80 con 0x7f1aa4003970 2024-03-12T17:07:10.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.612+0000 7f1abde00640 1 --2- 172.21.15.52:0/1413374855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ac00aee10 0x7f1ac015bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:10.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.612+0000 7f1abf200640 1 --2- 172.21.15.52:0/1413374855 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ac015f050 0x7f1ac015f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:10.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.612+0000 7f1abe800640 1 --2- 172.21.15.52:0/1413374855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1aa4003970 0x7f1ac00afc90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0037040 tx=0x7f1ab0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:10.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.612+0000 7f1ab6a00640 1 -- 172.21.15.52:0/1413374855 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0003810 con 0x7f1aa4003970 2024-03-12T17:07:10.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.612+0000 7f1ab6a00640 1 -- 172.21.15.52:0/1413374855 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab002de90 con 0x7f1aa4003970 2024-03-12T17:07:10.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.612+0000 7f1ab6a00640 1 -- 172.21.15.52:0/1413374855 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0003b70 con 0x7f1aa4003970 2024-03-12T17:07:10.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.612+0000 7f1ac4a00640 1 -- 172.21.15.52:0/1413374855 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac015ae60 con 0x7f1aa4003970 2024-03-12T17:07:10.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.612+0000 7f1ac4a00640 1 -- 172.21.15.52:0/1413374855 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac015b420 con 0x7f1aa4003970 2024-03-12T17:07:10.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.614+0000 7f1ac4a00640 1 -- 172.21.15.52:0/1413374855 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac015fa20 con 0x7f1aa4003970 2024-03-12T17:07:10.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.614+0000 7f1ab6a00640 1 -- 172.21.15.52:0/1413374855 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1ab0003cd0 con 0x7f1aa4003970 2024-03-12T17:07:10.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.615+0000 7f1ab6a00640 1 --2- 172.21.15.52:0/1413374855 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1aa006ac30 0x7f1aa006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:10.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.615+0000 7f1ab6a00640 1 -- 172.21.15.52:0/1413374855 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1ab00ab510 con 0x7f1aa4003970 2024-03-12T17:07:10.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.616+0000 7f1abde00640 1 --2- 172.21.15.52:0/1413374855 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1aa006ac30 0x7f1aa006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:10.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.617+0000 7f1abde00640 1 --2- 172.21.15.52:0/1413374855 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1aa006ac30 0x7f1aa006d0f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1aac001c60 tx=0x7f1aac009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:10.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.620+0000 7f1ab6a00640 1 -- 172.21.15.52:0/1413374855 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f1ab00765d0 con 0x7f1aa4003970 2024-03-12T17:07:10.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.856+0000 7f1ac4a00640 1 -- 172.21.15.52:0/1413374855 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1ac015b5b0 con 0x7f1aa4003970 2024-03-12T17:07:10.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.857+0000 7f1ab6a00640 1 -- 172.21.15.52:0/1413374855 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f1ac015b5b0 con 0x7f1aa4003970 2024-03-12T17:07:10.851 INFO:teuthology.orchestra.run.smithi052.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-12T17:07:10.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.860+0000 7f1ac4a00640 1 -- 172.21.15.52:0/1413374855 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1aa006ac30 msgr2=0x7f1aa006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:10.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.860+0000 7f1ac4a00640 1 --2- 172.21.15.52:0/1413374855 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1aa006ac30 0x7f1aa006d0f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1aac001c60 tx=0x7f1aac009040 comp rx=0 tx=0).stop 2024-03-12T17:07:10.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.860+0000 7f1ac4a00640 1 -- 172.21.15.52:0/1413374855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1aa4003970 msgr2=0x7f1ac00afc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:10.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.860+0000 7f1ac4a00640 1 --2- 172.21.15.52:0/1413374855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1aa4003970 0x7f1ac00afc90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0037040 tx=0x7f1ab0004730 comp rx=0 tx=0).stop 2024-03-12T17:07:10.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.860+0000 7f1ac4a00640 1 -- 172.21.15.52:0/1413374855 shutdown_connections 2024-03-12T17:07:10.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.860+0000 7f1ac4a00640 1 --2- 172.21.15.52:0/1413374855 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1aa006ac30 0x7f1aa006d0f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:10.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.860+0000 7f1ac4a00640 1 --2- 172.21.15.52:0/1413374855 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ac015f050 0x7f1ac015f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:10.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.860+0000 7f1ac4a00640 1 --2- 172.21.15.52:0/1413374855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ac00aee10 0x7f1ac015bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:10.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.860+0000 7f1ac4a00640 1 --2- 172.21.15.52:0/1413374855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1aa4003970 0x7f1ac00afc90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:10.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.860+0000 7f1ac4a00640 1 -- 172.21.15.52:0/1413374855 >> 172.21.15.52:0/1413374855 conn(0x7f1ac0065df0 msgr2=0x7f1ac0069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:10.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.860+0000 7f1ac4a00640 1 -- 172.21.15.52:0/1413374855 shutdown_connections 2024-03-12T17:07:10.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:10.860+0000 7f1ac4a00640 1 -- 172.21.15.52:0/1413374855 wait complete. 2024-03-12T17:07:10.873 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-12T17:07:10.873 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-12T17:07:10.873 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi052.front.sepia.ceph.com 2024-03-12T17:07:10.873 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-12T17:07:10.873 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-12T17:07:10.873 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-12T17:07:10.873 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-12T17:07:10.873 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-12T17:07:10.873 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-12T17:07:10.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:10.874 DEBUG:teuthology.orchestra.run.smithi052:> ip addr 2024-03-12T17:07:10.925 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-12T17:07:10.925 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-12T17:07:10.925 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-03-12T17:07:10.925 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:07:10.925 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-03-12T17:07:10.925 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:07:10.926 INFO:teuthology.orchestra.run.smithi052.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-12T17:07:10.926 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d2 brd ff:ff:ff:ff:ff:ff 2024-03-12T17:07:10.926 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f0 2024-03-12T17:07:10.926 INFO:teuthology.orchestra.run.smithi052.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-12T17:07:10.926 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d3 brd ff:ff:ff:ff:ff:ff 2024-03-12T17:07:10.926 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f1 2024-03-12T17:07:10.926 INFO:teuthology.orchestra.run.smithi052.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-12T17:07:10.926 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:de brd ff:ff:ff:ff:ff:ff 2024-03-12T17:07:10.926 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f0 2024-03-12T17:07:10.926 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.21.15.52/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-12T17:07:10.926 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft 42563sec preferred_lft 42563sec 2024-03-12T17:07:10.927 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::ec4:7aff:febd:15de/64 scope link noprefixroute 2024-03-12T17:07:10.927 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:07:10.927 INFO:teuthology.orchestra.run.smithi052.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-12T17:07:10.927 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:df brd ff:ff:ff:ff:ff:ff 2024-03-12T17:07:10.927 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f1 2024-03-12T17:07:10.928 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-12T17:07:10.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:10.928 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec bash -c ' 2024-03-12T17:07:10.928 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-03-12T17:07:10.929 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link add name ceph-brx type bridge 2024-03-12T17:07:10.929 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip addr flush dev ceph-brx 2024-03-12T17:07:10.929 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set ceph-brx up 2024-03-12T17:07:10.929 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-12T17:07:10.929 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-03-12T17:07:11.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:11.160 DEBUG:teuthology.orchestra.run.smithi052:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-12T17:07:11.244 INFO:teuthology.orchestra.run.smithi052.stdout:1 2024-03-12T17:07:11.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:11.248 DEBUG:teuthology.orchestra.run.smithi052:> route 2024-03-12T17:07:11.300 INFO:teuthology.orchestra.run.smithi052.stdout:Kernel IP routing table 2024-03-12T17:07:11.300 INFO:teuthology.orchestra.run.smithi052.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-12T17:07:11.300 INFO:teuthology.orchestra.run.smithi052.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-12T17:07:11.300 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-12T17:07:11.300 INFO:teuthology.orchestra.run.smithi052.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-12T17:07:11.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:11.300 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec bash -c ' 2024-03-12T17:07:11.301 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-03-12T17:07:11.301 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-12T17:07:11.301 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-12T17:07:11.301 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-12T17:07:11.301 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-03-12T17:07:11.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:11.489 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-03-12T17:07:11.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:11.542 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list-id 2024-03-12T17:07:11.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:11.594 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec bash -c ' 2024-03-12T17:07:11.595 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-03-12T17:07:11.595 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-12T17:07:11.595 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-12T17:07:11.595 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-03-12T17:07:11.744 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-12T17:07:11.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:11.744 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec bash -c ' 2024-03-12T17:07:11.744 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-03-12T17:07:11.744 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-12T17:07:11.745 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-12T17:07:11.745 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-12T17:07:11.745 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-12T17:07:11.745 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-12T17:07:11.745 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-03-12T17:07:12.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:12.012 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec bash -c ' 2024-03-12T17:07:12.012 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-03-12T17:07:12.012 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set brx.0 up 2024-03-12T17:07:12.012 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set dev brx.0 master ceph-brx 2024-03-12T17:07:12.012 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-03-12T17:07:12.169 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-12T17:07:12.169 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-12T17:07:12.217 INFO:teuthology.orchestra.run.smithi052.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-12T17:07:12.218 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-12T17:07:12.218 DEBUG:teuthology.orchestra.run.smithi052:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-12T17:07:12.267 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-12T17:07:12.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:12.267 DEBUG:teuthology.orchestra.run.smithi052:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-12T17:07:12.408 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-12T17:07:12.408 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-12T17:07:13.074 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-12T17:07:13.074 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-12T17:07:13.074 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: resolved to: "172.21.15.52:3300,172.21.15.102:3300,172.21.15.102:3301" 2024-03-12T17:07:13.074 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: trying mount with new device syntax: 0@524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3.cephfs=/ 2024-03-12T17:07:13.074 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.52:3300/172.21.15.102:3300/172.21.15.102:3301" will pass to kernel 2024-03-12T17:07:13.078 INFO:teuthology.orchestra.run.smithi052.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-12T17:07:13.078 INFO:teuthology.orchestra.run.smithi052.stdout: You just mounted a file system that supports labels which does not 2024-03-12T17:07:13.078 INFO:teuthology.orchestra.run.smithi052.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-12T17:07:13.078 INFO:teuthology.orchestra.run.smithi052.stdout: applications will generate AVC messages and not be allowed access to 2024-03-12T17:07:13.078 INFO:teuthology.orchestra.run.smithi052.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-12T17:07:13.082 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-12T17:07:13.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:13.082 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-12T17:07:13.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:13.163 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-12T17:07:13.216 INFO:teuthology.orchestra.run.smithi052.stdout:client4872 2024-03-12T17:07:13.217 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:07:13.217 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-12T17:07:13.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:13.267 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-12T17:07:13.375 INFO:teuthology.orchestra.run.smithi052.stdout:client4872 2024-03-12T17:07:13.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:13.376 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-12T17:07:13.439 INFO:teuthology.orchestra.run.smithi052.stdout:client4872 2024-03-12T17:07:13.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:13.440 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3.client4872/status') 2024-03-12T17:07:13.535 INFO:teuthology.orchestra.run.smithi052.stdout:instance: client.4872 (3)192.168.0.1:0/2903735896 2024-03-12T17:07:13.536 INFO:teuthology.orchestra.run.smithi052.stdout:blocklisted: false 2024-03-12T17:07:13.536 INFO:teuthology.orchestra.run.smithi052.stderr:0+1 records in 2024-03-12T17:07:13.536 INFO:teuthology.orchestra.run.smithi052.stderr:0+1 records out 2024-03-12T17:07:13.537 INFO:teuthology.orchestra.run.smithi052.stderr:69 bytes copied, 9.0655e-05 s, 761 kB/s 2024-03-12T17:07:13.540 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-03-12T17:07:13.541 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-12T17:07:13.541 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-12T17:07:13.541 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-12T17:07:13.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.761+0000 7f72a7600640 1 Processor -- start 2024-03-12T17:07:13.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.761+0000 7f72a7600640 1 -- start start 2024-03-12T17:07:13.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.763+0000 7f72a7600640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f72a00b39b0 0x7f72a00b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:13.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.763+0000 7f72a7600640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f72a015c400 0x7f72a015c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:13.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.763+0000 7f72a7600640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72a006c6f0 con 0x7f72a015b560 2024-03-12T17:07:13.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.763+0000 7f72a7600640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72a00b4370 con 0x7f72a00b39b0 2024-03-12T17:07:13.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.763+0000 7f72a7600640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a006c9d0 con 0x7f72a015c400 2024-03-12T17:07:13.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.763+0000 7f729fe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f72a00b39b0 0x7f72a00b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:13.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.763+0000 7f72a4c00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f72a015b560 0x7f72a015b940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:33242/0 (socket says 172.21.15.52:33242) 2024-03-12T17:07:13.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.763+0000 7f72a4c00640 1 -- 172.21.15.52:0/3098626937 learned_addr learned my addr 172.21.15.52:0/3098626937 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:13.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.764+0000 7f72a5600640 1 --2- 172.21.15.52:0/3098626937 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f72a015c400 0x7f72a015c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:13.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.764+0000 7f729fe00640 1 -- 172.21.15.52:0/3098626937 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f72a015c400 msgr2=0x7f72a015c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:13.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.764+0000 7f729fe00640 1 --2- 172.21.15.52:0/3098626937 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f72a015c400 0x7f72a015c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:13.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.764+0000 7f729fe00640 1 -- 172.21.15.52:0/3098626937 >> v1:172.21.15.52:6789/0 conn(0x7f72a015b560 legacy=0x7f72a015b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:13.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.764+0000 7f729fe00640 1 -- 172.21.15.52:0/3098626937 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a006ccb0 con 0x7f72a00b39b0 2024-03-12T17:07:13.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.764+0000 7f729f400640 1 -- 172.21.15.52:0/3098626937 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2021056594 0 0) 0x7f72a006c6f0 con 0x7f72a015b560 2024-03-12T17:07:13.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.764+0000 7f72a5600640 1 --2- 172.21.15.52:0/3098626937 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f72a015c400 0x7f72a015c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:13.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.765+0000 7f729fe00640 1 --2- 172.21.15.52:0/3098626937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72a00b39b0 0x7f72a00b3e30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f728c0097a0 tx=0x7f728c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d8a488d6d1b1a02 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:13.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.765+0000 7f729f400640 1 -- 172.21.15.52:0/3098626937 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c02d040 con 0x7f72a00b39b0 2024-03-12T17:07:13.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.765+0000 7f729f400640 1 -- 172.21.15.52:0/3098626937 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728c031810 con 0x7f72a00b39b0 2024-03-12T17:07:13.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.765+0000 7f729f400640 1 -- 172.21.15.52:0/3098626937 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c031ba0 con 0x7f72a00b39b0 2024-03-12T17:07:13.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.766+0000 7f72a7600640 1 -- 172.21.15.52:0/3098626937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72a00b39b0 msgr2=0x7f72a00b3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:13.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.766+0000 7f72a7600640 1 --2- 172.21.15.52:0/3098626937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72a00b39b0 0x7f72a00b3e30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f728c0097a0 tx=0x7f728c02b8b0 comp rx=0 tx=0).stop 2024-03-12T17:07:13.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.766+0000 7f72a7600640 1 -- 172.21.15.52:0/3098626937 shutdown_connections 2024-03-12T17:07:13.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.766+0000 7f72a7600640 1 --2- 172.21.15.52:0/3098626937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72a00b39b0 0x7f72a00b3e30 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:13.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.766+0000 7f72a7600640 1 --2- 172.21.15.52:0/3098626937 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f72a015c400 0x7f72a015c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:13.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.766+0000 7f72a7600640 1 -- 172.21.15.52:0/3098626937 >> 172.21.15.52:0/3098626937 conn(0x7f72a006a3d0 msgr2=0x7f72a00b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:13.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.766+0000 7f72a7600640 1 -- 172.21.15.52:0/3098626937 shutdown_connections 2024-03-12T17:07:13.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.766+0000 7f72a7600640 1 -- 172.21.15.52:0/3098626937 wait complete. 2024-03-12T17:07:13.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.768+0000 7f72a7600640 1 Processor -- start 2024-03-12T17:07:13.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.768+0000 7f72a7600640 1 -- start start 2024-03-12T17:07:13.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.769+0000 7f72a7600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72a00b39b0 0x7f72a0149d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:13.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.769+0000 7f72a7600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72a015b560 0x7f72a01f6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:13.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.769+0000 7f72a7600640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f72a015c400 0x7f72a00af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:13.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.769+0000 7f72a7600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a01f6680 con 0x7f72a00b39b0 2024-03-12T17:07:13.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.769+0000 7f72a7600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a00b1940 con 0x7f72a015b560 2024-03-12T17:07:13.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.769+0000 7f72a7600640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a00b1c20 con 0x7f72a015c400 2024-03-12T17:07:13.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.769+0000 7f72a4c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72a00b39b0 0x7f72a0149d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:13.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.769+0000 7f72a4c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72a00b39b0 0x7f72a0149d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54318/0 (socket says 172.21.15.52:54318) 2024-03-12T17:07:13.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.769+0000 7f72a4c00640 1 -- 172.21.15.52:0/3924049551 learned_addr learned my addr 172.21.15.52:0/3924049551 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:13.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.770+0000 7f72a5600640 1 --2- 172.21.15.52:0/3924049551 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f72a015c400 0x7f72a00af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:13.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.770+0000 7f729fe00640 1 --2- 172.21.15.52:0/3924049551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72a015b560 0x7f72a01f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:13.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.770+0000 7f72a4c00640 1 -- 172.21.15.52:0/3924049551 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f72a015c400 msgr2=0x7f72a00af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:13.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.770+0000 7f72a4c00640 1 --2- 172.21.15.52:0/3924049551 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f72a015c400 0x7f72a00af360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:13.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.770+0000 7f72a4c00640 1 -- 172.21.15.52:0/3924049551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72a015b560 msgr2=0x7f72a01f6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:13.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.770+0000 7f72a4c00640 1 --2- 172.21.15.52:0/3924049551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72a015b560 0x7f72a01f6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:13.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.770+0000 7f72a4c00640 1 -- 172.21.15.52:0/3924049551 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f728c009400 con 0x7f72a00b39b0 2024-03-12T17:07:13.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.770+0000 7f729fe00640 1 --2- 172.21.15.52:0/3924049551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72a015b560 0x7f72a01f6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:13.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.770+0000 7f72a4c00640 1 --2- 172.21.15.52:0/3924049551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72a00b39b0 0x7f72a0149d60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f729400c4d0 tx=0x7f729400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:13.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.771+0000 7f729cc00640 1 -- 172.21.15.52:0/3924049551 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72940074e0 con 0x7f72a00b39b0 2024-03-12T17:07:13.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.771+0000 7f729cc00640 1 -- 172.21.15.52:0/3924049551 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7294007660 con 0x7f72a00b39b0 2024-03-12T17:07:13.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.771+0000 7f72a7600640 1 -- 172.21.15.52:0/3924049551 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a00b1f00 con 0x7f72a00b39b0 2024-03-12T17:07:13.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.771+0000 7f729cc00640 1 -- 172.21.15.52:0/3924049551 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7294004a20 con 0x7f72a00b39b0 2024-03-12T17:07:13.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.771+0000 7f72a7600640 1 -- 172.21.15.52:0/3924049551 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a00b24c0 con 0x7f72a00b39b0 2024-03-12T17:07:13.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.772+0000 7f72a7600640 1 -- 172.21.15.52:0/3924049551 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72a006ac20 con 0x7f72a00b39b0 2024-03-12T17:07:13.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.773+0000 7f729cc00640 1 -- 172.21.15.52:0/3924049551 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7294004b80 con 0x7f72a00b39b0 2024-03-12T17:07:13.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.774+0000 7f729cc00640 1 --2- 172.21.15.52:0/3924049551 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f728006ac00 0x7f728006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:13.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.774+0000 7f729cc00640 1 -- 172.21.15.52:0/3924049551 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7294088cc0 con 0x7f72a00b39b0 2024-03-12T17:07:13.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.775+0000 7f729fe00640 1 --2- 172.21.15.52:0/3924049551 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f728006ac00 0x7f728006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:13.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.776+0000 7f729fe00640 1 --2- 172.21.15.52:0/3924049551 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f728006ac00 0x7f728006d0c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f728c036000 tx=0x7f728c03c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:13.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:13.779+0000 7f729cc00640 1 -- 172.21.15.52:0/3924049551 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f729408f080 con 0x7f72a00b39b0 2024-03-12T17:07:14.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:14.022+0000 7f72a7600640 1 -- 172.21.15.52:0/3924049551 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f72a0073390 con 0x7f72a00b39b0 2024-03-12T17:07:14.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:14.023+0000 7f729cc00640 1 -- 172.21.15.52:0/3924049551 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f7294018020 con 0x7f72a00b39b0 2024-03-12T17:07:14.017 INFO:teuthology.orchestra.run.smithi052.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-12T17:07:14.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:14.025+0000 7f72a7600640 1 -- 172.21.15.52:0/3924049551 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f728006ac00 msgr2=0x7f728006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:14.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:14.025+0000 7f72a7600640 1 --2- 172.21.15.52:0/3924049551 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f728006ac00 0x7f728006d0c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f728c036000 tx=0x7f728c03c040 comp rx=0 tx=0).stop 2024-03-12T17:07:14.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:14.025+0000 7f72a7600640 1 -- 172.21.15.52:0/3924049551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72a00b39b0 msgr2=0x7f72a0149d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:14.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:14.025+0000 7f72a7600640 1 --2- 172.21.15.52:0/3924049551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72a00b39b0 0x7f72a0149d60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f729400c4d0 tx=0x7f729400c990 comp rx=0 tx=0).stop 2024-03-12T17:07:14.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:14.026+0000 7f72a7600640 1 -- 172.21.15.52:0/3924049551 shutdown_connections 2024-03-12T17:07:14.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:14.026+0000 7f72a7600640 1 --2- 172.21.15.52:0/3924049551 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f728006ac00 0x7f728006d0c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:14.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:14.026+0000 7f72a7600640 1 --2- 172.21.15.52:0/3924049551 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f72a015c400 0x7f72a00af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:14.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:14.026+0000 7f72a7600640 1 --2- 172.21.15.52:0/3924049551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72a015b560 0x7f72a01f6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:14.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:14.026+0000 7f72a7600640 1 --2- 172.21.15.52:0/3924049551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72a00b39b0 0x7f72a0149d60 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:14.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:14.026+0000 7f72a7600640 1 -- 172.21.15.52:0/3924049551 >> 172.21.15.52:0/3924049551 conn(0x7f72a006a3d0 msgr2=0x7f72a015e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:14.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:14.026+0000 7f72a7600640 1 -- 172.21.15.52:0/3924049551 shutdown_connections 2024-03-12T17:07:14.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:14.026+0000 7f72a7600640 1 -- 172.21.15.52:0/3924049551 wait complete. 2024-03-12T17:07:14.040 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-12T17:07:14.040 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-12T17:07:14.040 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi052.front.sepia.ceph.com 2024-03-12T17:07:14.040 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-03-12T17:07:14.040 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-12T17:07:14.040 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-12T17:07:14.040 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-12T17:07:14.040 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-12T17:07:14.040 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-03-12T17:07:14.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:14.041 DEBUG:teuthology.orchestra.run.smithi052:> ip addr 2024-03-12T17:07:14.093 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-12T17:07:14.093 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-12T17:07:14.094 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-03-12T17:07:14.094 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:07:14.094 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-03-12T17:07:14.094 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:07:14.094 INFO:teuthology.orchestra.run.smithi052.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-12T17:07:14.094 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d2 brd ff:ff:ff:ff:ff:ff 2024-03-12T17:07:14.094 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f0 2024-03-12T17:07:14.094 INFO:teuthology.orchestra.run.smithi052.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-12T17:07:14.094 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d3 brd ff:ff:ff:ff:ff:ff 2024-03-12T17:07:14.094 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f1 2024-03-12T17:07:14.094 INFO:teuthology.orchestra.run.smithi052.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-12T17:07:14.094 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:de brd ff:ff:ff:ff:ff:ff 2024-03-12T17:07:14.094 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f0 2024-03-12T17:07:14.095 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.21.15.52/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-12T17:07:14.095 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft 42560sec preferred_lft 42560sec 2024-03-12T17:07:14.095 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::ec4:7aff:febd:15de/64 scope link noprefixroute 2024-03-12T17:07:14.095 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:07:14.095 INFO:teuthology.orchestra.run.smithi052.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-12T17:07:14.095 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:df brd ff:ff:ff:ff:ff:ff 2024-03-12T17:07:14.095 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f1 2024-03-12T17:07:14.095 INFO:teuthology.orchestra.run.smithi052.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-12T17:07:14.095 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 72:aa:0f:4e:37:01 brd ff:ff:ff:ff:ff:ff 2024-03-12T17:07:14.095 INFO:teuthology.orchestra.run.smithi052.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-12T17:07:14.095 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:07:14.095 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::1896:69ff:fe28:1273/64 scope link 2024-03-12T17:07:14.095 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:07:14.095 INFO:teuthology.orchestra.run.smithi052.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-12T17:07:14.096 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 72:aa:0f:4e:37:01 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-12T17:07:14.096 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::70aa:fff:fe4e:3701/64 scope link 2024-03-12T17:07:14.096 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:07:14.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:14.097 DEBUG:teuthology.orchestra.run.smithi052:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-12T17:07:14.191 INFO:teuthology.orchestra.run.smithi052.stdout:1 2024-03-12T17:07:14.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:14.196 DEBUG:teuthology.orchestra.run.smithi052:> route 2024-03-12T17:07:14.247 INFO:teuthology.orchestra.run.smithi052.stdout:Kernel IP routing table 2024-03-12T17:07:14.247 INFO:teuthology.orchestra.run.smithi052.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-12T17:07:14.248 INFO:teuthology.orchestra.run.smithi052.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-12T17:07:14.248 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-12T17:07:14.248 INFO:teuthology.orchestra.run.smithi052.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-12T17:07:14.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:14.248 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec bash -c ' 2024-03-12T17:07:14.248 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-03-12T17:07:14.248 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-12T17:07:14.248 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-12T17:07:14.248 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-12T17:07:14.248 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-03-12T17:07:14.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:14.438 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-03-12T17:07:14.492 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-12T17:07:14.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:14.493 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list-id 2024-03-12T17:07:14.549 INFO:teuthology.orchestra.run.smithi052.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-03-12T17:07:14.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:14.550 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec bash -c ' 2024-03-12T17:07:14.550 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-03-12T17:07:14.550 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-12T17:07:14.550 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-03-12T17:07:14.550 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-03-12T17:07:14.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:14.700 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-12T17:07:14.797 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-12T17:07:14.797 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-12T17:07:14.797 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-03-12T17:07:14.797 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:07:14.797 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-03-12T17:07:14.798 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:07:14.798 INFO:teuthology.orchestra.run.smithi052.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-12T17:07:14.798 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 02:5b:29:e4:bc:5f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-12T17:07:14.798 INFO:teuthology.orchestra.run.smithi052.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-12T17:07:14.798 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:07:14.798 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::5b:29ff:fee4:bc5f/64 scope link 2024-03-12T17:07:14.798 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:07:14.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:14.802 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-12T17:07:14.898 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-12T17:07:14.898 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-12T17:07:14.898 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-03-12T17:07:14.898 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:07:14.898 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-03-12T17:07:14.898 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:07:14.898 INFO:teuthology.orchestra.run.smithi052.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-12T17:07:14.898 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 02:5b:29:e4:bc:5f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-12T17:07:14.898 INFO:teuthology.orchestra.run.smithi052.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-12T17:07:14.898 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:07:14.899 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::5b:29ff:fee4:bc5f/64 scope link 2024-03-12T17:07:14.899 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:07:14.902 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-03-12T17:07:14.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:14.902 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec bash -c ' 2024-03-12T17:07:14.902 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-03-12T17:07:14.902 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-03-12T17:07:14.902 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-12T17:07:14.903 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-03-12T17:07:14.903 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-03-12T17:07:14.903 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-03-12T17:07:14.903 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-03-12T17:07:15.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:15.198 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec bash -c ' 2024-03-12T17:07:15.198 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-03-12T17:07:15.198 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set brx.1 up 2024-03-12T17:07:15.198 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set dev brx.1 master ceph-brx 2024-03-12T17:07:15.198 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-03-12T17:07:15.370 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-12T17:07:15.370 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-03-12T17:07:15.421 INFO:teuthology.orchestra.run.smithi052.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-03-12T17:07:15.421 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-12T17:07:15.421 DEBUG:teuthology.orchestra.run.smithi052:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-03-12T17:07:15.471 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-03-12T17:07:15.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:15.472 DEBUG:teuthology.orchestra.run.smithi052:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-12T17:07:15.609 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-12T17:07:15.610 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-12T17:07:15.645 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-12T17:07:15.645 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-12T17:07:15.645 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: resolved to: "172.21.15.52:3300,172.21.15.102:3300,172.21.15.102:3301" 2024-03-12T17:07:15.645 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: trying mount with new device syntax: 2@524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3.cephfs=/ 2024-03-12T17:07:15.645 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.52:3300/172.21.15.102:3300/172.21.15.102:3301" will pass to kernel 2024-03-12T17:07:15.647 INFO:teuthology.orchestra.run.smithi052.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-03-12T17:07:15.648 INFO:teuthology.orchestra.run.smithi052.stdout: You just mounted a file system that supports labels which does not 2024-03-12T17:07:15.648 INFO:teuthology.orchestra.run.smithi052.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-12T17:07:15.648 INFO:teuthology.orchestra.run.smithi052.stdout: applications will generate AVC messages and not be allowed access to 2024-03-12T17:07:15.648 INFO:teuthology.orchestra.run.smithi052.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-12T17:07:15.651 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-12T17:07:15.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:15.652 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-03-12T17:07:15.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:15.746 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-12T17:07:15.807 INFO:teuthology.orchestra.run.smithi052.stdout:client4884 2024-03-12T17:07:15.808 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:07:15.808 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-12T17:07:15.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:15.860 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-12T17:07:15.913 INFO:teuthology.orchestra.run.smithi052.stdout:client4884 2024-03-12T17:07:15.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:15.914 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-12T17:07:15.974 INFO:teuthology.orchestra.run.smithi052.stdout:client4884 2024-03-12T17:07:15.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:15.974 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3.client4884/status') 2024-03-12T17:07:16.066 INFO:teuthology.orchestra.run.smithi052.stdout:instance: client.4884 (3)192.168.0.2:0/893521411 2024-03-12T17:07:16.066 INFO:teuthology.orchestra.run.smithi052.stdout:blocklisted: false 2024-03-12T17:07:16.066 INFO:teuthology.orchestra.run.smithi052.stderr:0+1 records in 2024-03-12T17:07:16.066 INFO:teuthology.orchestra.run.smithi052.stderr:0+1 records out 2024-03-12T17:07:16.067 INFO:teuthology.orchestra.run.smithi052.stderr:68 bytes copied, 7.8249e-05 s, 869 kB/s 2024-03-12T17:07:16.070 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-03-12T17:07:16.070 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-12T17:07:16.070 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-12T17:07:16.071 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-12T17:07:16.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.295+0000 7f0046e00640 1 Processor -- start 2024-03-12T17:07:16.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.295+0000 7f0046e00640 1 -- start start 2024-03-12T17:07:16.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.297+0000 7f0046e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f0040143930 0x7f00400cd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:16.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.297+0000 7f0046e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f00400cdc50 0x7f0040070020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:16.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.297+0000 7f0046e00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f004015b2b0 con 0x7f0040143460 2024-03-12T17:07:16.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.297+0000 7f0046e00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00400cc6b0 con 0x7f0040143930 2024-03-12T17:07:16.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.297+0000 7f0046e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00400cc950 con 0x7f00400cdc50 2024-03-12T17:07:16.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.297+0000 7f0045000640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f0040143930 0x7f00400cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:16.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.297+0000 7f0045000640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f0040143930 0x7f00400cd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49172/0 (socket says 172.21.15.52:49172) 2024-03-12T17:07:16.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.297+0000 7f0045000640 1 -- 172.21.15.52:0/842365751 learned_addr learned my addr 172.21.15.52:0/842365751 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:16.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.297+0000 7f0045a00640 1 --1- 172.21.15.52:0/842365751 >> v1:172.21.15.52:6789/0 conn(0x7f0040143460 0x7f004015aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:57318/0 (socket says 172.21.15.52:57318) 2024-03-12T17:07:16.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.297+0000 7f0046400640 1 --2- 172.21.15.52:0/842365751 >> v2:172.21.15.102:3300/0 conn(0x7f00400cdc50 0x7f0040070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:16.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.298+0000 7f003fe00640 1 -- 172.21.15.52:0/842365751 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4105273534 0 0) 0x7f004015b2b0 con 0x7f0040143460 2024-03-12T17:07:16.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.298+0000 7f003fe00640 1 -- 172.21.15.52:0/842365751 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0024003520 con 0x7f0040143460 2024-03-12T17:07:16.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.298+0000 7f0045000640 1 -- 172.21.15.52:0/842365751 >> v2:172.21.15.102:3300/0 conn(0x7f00400cdc50 msgr2=0x7f0040070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:16.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.298+0000 7f0045000640 1 --2- 172.21.15.52:0/842365751 >> v2:172.21.15.102:3300/0 conn(0x7f00400cdc50 0x7f0040070020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:16.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.298+0000 7f0045000640 1 -- 172.21.15.52:0/842365751 >> v1:172.21.15.52:6789/0 conn(0x7f0040143460 legacy=0x7f004015aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:16.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.298+0000 7f0045000640 1 -- 172.21.15.52:0/842365751 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00400ccc30 con 0x7f0040143930 2024-03-12T17:07:16.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.299+0000 7f0045000640 1 --2- 172.21.15.52:0/842365751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0040143930 0x7f00400cd710 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f00400b1a50 tx=0x7f002c02c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=816b3e0449be4973 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:16.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.299+0000 7f003fe00640 1 -- 172.21.15.52:0/842365751 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002c02cbb0 con 0x7f0040143930 2024-03-12T17:07:16.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.299+0000 7f003fe00640 1 -- 172.21.15.52:0/842365751 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f002c02cd10 con 0x7f0040143930 2024-03-12T17:07:16.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.299+0000 7f003fe00640 1 -- 172.21.15.52:0/842365751 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002c034610 con 0x7f0040143930 2024-03-12T17:07:16.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.300+0000 7f0046e00640 1 -- 172.21.15.52:0/842365751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0040143930 msgr2=0x7f00400cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:16.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.300+0000 7f0046e00640 1 --2- 172.21.15.52:0/842365751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0040143930 0x7f00400cd710 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f00400b1a50 tx=0x7f002c02c010 comp rx=0 tx=0).stop 2024-03-12T17:07:16.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.300+0000 7f0046e00640 1 -- 172.21.15.52:0/842365751 shutdown_connections 2024-03-12T17:07:16.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.300+0000 7f0046e00640 1 --2- 172.21.15.52:0/842365751 >> v2:172.21.15.102:3300/0 conn(0x7f00400cdc50 0x7f0040070020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:16.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.300+0000 7f0046e00640 1 --2- 172.21.15.52:0/842365751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0040143930 0x7f00400cd710 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:16.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.300+0000 7f0046e00640 1 -- 172.21.15.52:0/842365751 >> 172.21.15.52:0/842365751 conn(0x7f0040065df0 msgr2=0x7f0040069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:16.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.300+0000 7f0046e00640 1 -- 172.21.15.52:0/842365751 shutdown_connections 2024-03-12T17:07:16.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.300+0000 7f0046e00640 1 -- 172.21.15.52:0/842365751 wait complete. 2024-03-12T17:07:16.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.302+0000 7f0046e00640 1 Processor -- start 2024-03-12T17:07:16.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.302+0000 7f0046e00640 1 -- start start 2024-03-12T17:07:16.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.303+0000 7f0046e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00400cdc50 0x7f00401f2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:16.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.303+0000 7f0046e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0040143460 0x7f00400acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:16.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.303+0000 7f0046e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0040143930 0x7f004013bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:16.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.303+0000 7f0046e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00400c3c40 con 0x7f0040143460 2024-03-12T17:07:16.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.303+0000 7f0046e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00400b1df0 con 0x7f00400cdc50 2024-03-12T17:07:16.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.303+0000 7f0046e00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00401f1fb0 con 0x7f0040143930 2024-03-12T17:07:16.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.303+0000 7f0045000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0040143460 0x7f00400acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:16.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.304+0000 7f0045000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0040143460 0x7f00400acdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49180/0 (socket says 172.21.15.52:49180) 2024-03-12T17:07:16.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.304+0000 7f0045000640 1 -- 172.21.15.52:0/336691514 learned_addr learned my addr 172.21.15.52:0/336691514 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:16.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.304+0000 7f0045a00640 1 --2- 172.21.15.52:0/336691514 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00400cdc50 0x7f00401f2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:16.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.304+0000 7f0046400640 1 --2- 172.21.15.52:0/336691514 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0040143930 0x7f004013bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:16.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.304+0000 7f0045000640 1 -- 172.21.15.52:0/336691514 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0040143930 msgr2=0x7f004013bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:16.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.304+0000 7f0045000640 1 --2- 172.21.15.52:0/336691514 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0040143930 0x7f004013bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:16.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.304+0000 7f0045000640 1 -- 172.21.15.52:0/336691514 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00400cdc50 msgr2=0x7f00401f2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:16.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.304+0000 7f0045000640 1 --2- 172.21.15.52:0/336691514 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00400cdc50 0x7f00401f2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:16.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.304+0000 7f0045000640 1 -- 172.21.15.52:0/336691514 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f002c009b30 con 0x7f0040143460 2024-03-12T17:07:16.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.304+0000 7f0046400640 1 --2- 172.21.15.52:0/336691514 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0040143930 0x7f004013bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:16.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.304+0000 7f0045000640 1 --2- 172.21.15.52:0/336691514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0040143460 0x7f00400acdc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f002c000f80 tx=0x7f002c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:16.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.305+0000 7f003d600640 1 -- 172.21.15.52:0/336691514 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002c040670 con 0x7f0040143460 2024-03-12T17:07:16.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.305+0000 7f003d600640 1 -- 172.21.15.52:0/336691514 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f002c0407d0 con 0x7f0040143460 2024-03-12T17:07:16.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.305+0000 7f003d600640 1 -- 172.21.15.52:0/336691514 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002c040b80 con 0x7f0040143460 2024-03-12T17:07:16.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.305+0000 7f0046e00640 1 -- 172.21.15.52:0/336691514 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00401f2290 con 0x7f0040143460 2024-03-12T17:07:16.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.305+0000 7f0046e00640 1 -- 172.21.15.52:0/336691514 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00400b20d0 con 0x7f0040143460 2024-03-12T17:07:16.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.306+0000 7f0046e00640 1 -- 172.21.15.52:0/336691514 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00400adf50 con 0x7f0040143460 2024-03-12T17:07:16.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.307+0000 7f003d600640 1 -- 172.21.15.52:0/336691514 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f002c040ce0 con 0x7f0040143460 2024-03-12T17:07:16.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.308+0000 7f003d600640 1 --2- 172.21.15.52:0/336691514 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f002006ac00 0x7f002006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:16.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.308+0000 7f003d600640 1 -- 172.21.15.52:0/336691514 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f002c0ac6c0 con 0x7f0040143460 2024-03-12T17:07:16.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.309+0000 7f0045a00640 1 --2- 172.21.15.52:0/336691514 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f002006ac00 0x7f002006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:16.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.310+0000 7f0045a00640 1 --2- 172.21.15.52:0/336691514 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f002006ac00 0x7f002006d0c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f00300011e0 tx=0x7f0030007b70 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:16.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.314+0000 7f003d600640 1 -- 172.21.15.52:0/336691514 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f002c0775f0 con 0x7f0040143460 2024-03-12T17:07:16.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.553+0000 7f0046e00640 1 -- 172.21.15.52:0/336691514 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f00401f2750 con 0x7f0040143460 2024-03-12T17:07:16.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.554+0000 7f003d600640 1 -- 172.21.15.52:0/336691514 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f002c009510 con 0x7f0040143460 2024-03-12T17:07:16.548 INFO:teuthology.orchestra.run.smithi052.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-12T17:07:16.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.557+0000 7f0046e00640 1 -- 172.21.15.52:0/336691514 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f002006ac00 msgr2=0x7f002006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:16.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.557+0000 7f0046e00640 1 --2- 172.21.15.52:0/336691514 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f002006ac00 0x7f002006d0c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f00300011e0 tx=0x7f0030007b70 comp rx=0 tx=0).stop 2024-03-12T17:07:16.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.557+0000 7f0046e00640 1 -- 172.21.15.52:0/336691514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0040143460 msgr2=0x7f00400acdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:16.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.557+0000 7f0046e00640 1 --2- 172.21.15.52:0/336691514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0040143460 0x7f00400acdc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f002c000f80 tx=0x7f002c004a40 comp rx=0 tx=0).stop 2024-03-12T17:07:16.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.557+0000 7f0046e00640 1 -- 172.21.15.52:0/336691514 shutdown_connections 2024-03-12T17:07:16.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.557+0000 7f0046e00640 1 --2- 172.21.15.52:0/336691514 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f002006ac00 0x7f002006d0c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:16.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.557+0000 7f0046e00640 1 --2- 172.21.15.52:0/336691514 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0040143930 0x7f004013bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:16.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.557+0000 7f0046e00640 1 --2- 172.21.15.52:0/336691514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0040143460 0x7f00400acdc0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:16.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.557+0000 7f0046e00640 1 --2- 172.21.15.52:0/336691514 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00400cdc50 0x7f00401f2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:16.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.557+0000 7f0046e00640 1 -- 172.21.15.52:0/336691514 >> 172.21.15.52:0/336691514 conn(0x7f0040065df0 msgr2=0x7f00400aed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:16.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.557+0000 7f0046e00640 1 -- 172.21.15.52:0/336691514 shutdown_connections 2024-03-12T17:07:16.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:16.557+0000 7f0046e00640 1 -- 172.21.15.52:0/336691514 wait complete. 2024-03-12T17:07:16.570 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-12T17:07:16.570 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-12T17:07:16.570 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi102.front.sepia.ceph.com 2024-03-12T17:07:16.570 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-12T17:07:16.570 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-12T17:07:16.570 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-12T17:07:16.570 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-12T17:07:16.570 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-12T17:07:16.570 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-12T17:07:16.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:16.571 DEBUG:teuthology.orchestra.run.smithi102:> ip addr 2024-03-12T17:07:16.625 INFO:teuthology.orchestra.run.smithi102.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-12T17:07:16.625 INFO:teuthology.orchestra.run.smithi102.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-12T17:07:16.626 INFO:teuthology.orchestra.run.smithi102.stdout: inet 127.0.0.1/8 scope host lo 2024-03-12T17:07:16.626 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:07:16.626 INFO:teuthology.orchestra.run.smithi102.stdout: inet6 ::1/128 scope host 2024-03-12T17:07:16.626 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:07:16.626 INFO:teuthology.orchestra.run.smithi102.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-12T17:07:16.626 INFO:teuthology.orchestra.run.smithi102.stdout: link/ether 0c:c4:7a:d8:d9:dc brd ff:ff:ff:ff:ff:ff 2024-03-12T17:07:16.626 INFO:teuthology.orchestra.run.smithi102.stdout: altname enp7s0f0 2024-03-12T17:07:16.626 INFO:teuthology.orchestra.run.smithi102.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-12T17:07:16.626 INFO:teuthology.orchestra.run.smithi102.stdout: link/ether 0c:c4:7a:d8:d9:dd brd ff:ff:ff:ff:ff:ff 2024-03-12T17:07:16.626 INFO:teuthology.orchestra.run.smithi102.stdout: altname enp7s0f1 2024-03-12T17:07:16.626 INFO:teuthology.orchestra.run.smithi102.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-12T17:07:16.626 INFO:teuthology.orchestra.run.smithi102.stdout: link/ether 0c:c4:7a:8f:d7:68 brd ff:ff:ff:ff:ff:ff 2024-03-12T17:07:16.629 INFO:teuthology.orchestra.run.smithi102.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-12T17:07:16.629 INFO:teuthology.orchestra.run.smithi102.stdout: link/ether 0c:c4:7a:8f:d7:69 brd ff:ff:ff:ff:ff:ff 2024-03-12T17:07:16.629 INFO:teuthology.orchestra.run.smithi102.stdout: inet 172.21.15.102/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-12T17:07:16.629 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft 42556sec preferred_lft 42556sec 2024-03-12T17:07:16.629 INFO:teuthology.orchestra.run.smithi102.stdout: inet6 fe80::ec4:7aff:fe8f:d769/64 scope link 2024-03-12T17:07:16.630 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:07:16.631 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-12T17:07:16.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:16.631 DEBUG:teuthology.orchestra.run.smithi102:> (cd / && exec bash -c ' 2024-03-12T17:07:16.631 DEBUG:teuthology.orchestra.run.smithi102:> set -e 2024-03-12T17:07:16.631 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link add name ceph-brx type bridge 2024-03-12T17:07:16.631 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip addr flush dev ceph-brx 2024-03-12T17:07:16.631 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link set ceph-brx up 2024-03-12T17:07:16.631 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-12T17:07:16.632 DEBUG:teuthology.orchestra.run.smithi102:> ') 2024-03-12T17:07:17.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:17.085 DEBUG:teuthology.orchestra.run.smithi102:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-12T17:07:17.156 INFO:teuthology.orchestra.run.smithi102.stdout:1 2024-03-12T17:07:17.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:17.160 DEBUG:teuthology.orchestra.run.smithi102:> route 2024-03-12T17:07:17.224 INFO:teuthology.orchestra.run.smithi102.stdout:Kernel IP routing table 2024-03-12T17:07:17.224 INFO:teuthology.orchestra.run.smithi102.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-12T17:07:17.224 INFO:teuthology.orchestra.run.smithi102.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-12T17:07:17.224 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-12T17:07:17.224 INFO:teuthology.orchestra.run.smithi102.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-12T17:07:17.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:17.225 DEBUG:teuthology.orchestra.run.smithi102:> (cd / && exec bash -c ' 2024-03-12T17:07:17.225 DEBUG:teuthology.orchestra.run.smithi102:> set -e 2024-03-12T17:07:17.225 DEBUG:teuthology.orchestra.run.smithi102:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-12T17:07:17.225 DEBUG:teuthology.orchestra.run.smithi102:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-12T17:07:17.225 DEBUG:teuthology.orchestra.run.smithi102:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-12T17:07:17.225 DEBUG:teuthology.orchestra.run.smithi102:> ') 2024-03-12T17:07:17.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:17.943 DEBUG:teuthology.orchestra.run.smithi102:> ip netns list 2024-03-12T17:07:17.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:17.984 DEBUG:teuthology.orchestra.run.smithi102:> ip netns list-id 2024-03-12T17:07:18.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:18.071 DEBUG:teuthology.orchestra.run.smithi102:> (cd / && exec bash -c ' 2024-03-12T17:07:18.071 DEBUG:teuthology.orchestra.run.smithi102:> set -e 2024-03-12T17:07:18.071 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-12T17:07:18.071 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-12T17:07:18.072 DEBUG:teuthology.orchestra.run.smithi102:> ') 2024-03-12T17:07:18.231 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-12T17:07:18.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:18.231 DEBUG:teuthology.orchestra.run.smithi102:> (cd / && exec bash -c ' 2024-03-12T17:07:18.231 DEBUG:teuthology.orchestra.run.smithi102:> set -e 2024-03-12T17:07:18.232 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-12T17:07:18.232 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-12T17:07:18.232 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-12T17:07:18.232 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-12T17:07:18.232 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-12T17:07:18.232 DEBUG:teuthology.orchestra.run.smithi102:> ') 2024-03-12T17:07:18.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:18.491 DEBUG:teuthology.orchestra.run.smithi102:> (cd / && exec bash -c ' 2024-03-12T17:07:18.491 DEBUG:teuthology.orchestra.run.smithi102:> set -e 2024-03-12T17:07:18.491 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link set brx.0 up 2024-03-12T17:07:18.491 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link set dev brx.0 master ceph-brx 2024-03-12T17:07:18.491 DEBUG:teuthology.orchestra.run.smithi102:> ') 2024-03-12T17:07:18.643 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-12T17:07:18.643 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-12T17:07:18.684 INFO:teuthology.orchestra.run.smithi102.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-12T17:07:18.684 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-12T17:07:18.684 DEBUG:teuthology.orchestra.run.smithi102:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-12T17:07:18.775 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-03-12T17:07:18.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:18.775 DEBUG:teuthology.orchestra.run.smithi102:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-12T17:07:18.893 INFO:teuthology.orchestra.run.smithi102.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-12T17:07:18.894 INFO:teuthology.orchestra.run.smithi102.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-12T17:07:19.548 INFO:teuthology.orchestra.run.smithi102.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-12T17:07:19.548 INFO:teuthology.orchestra.run.smithi102.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-12T17:07:19.548 INFO:teuthology.orchestra.run.smithi102.stdout:mount.ceph: resolved to: "172.21.15.52:3300,172.21.15.102:3300,172.21.15.102:3301" 2024-03-12T17:07:19.548 INFO:teuthology.orchestra.run.smithi102.stdout:mount.ceph: trying mount with new device syntax: 1@524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3.cephfs=/ 2024-03-12T17:07:19.548 INFO:teuthology.orchestra.run.smithi102.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.52:3300/172.21.15.102:3300/172.21.15.102:3301" will pass to kernel 2024-03-12T17:07:19.551 INFO:teuthology.orchestra.run.smithi102.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-03-12T17:07:19.551 INFO:teuthology.orchestra.run.smithi102.stdout: You just mounted a file system that supports labels which does not 2024-03-12T17:07:19.552 INFO:teuthology.orchestra.run.smithi102.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-12T17:07:19.552 INFO:teuthology.orchestra.run.smithi102.stdout: applications will generate AVC messages and not be allowed access to 2024-03-12T17:07:19.552 INFO:teuthology.orchestra.run.smithi102.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-12T17:07:19.556 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-12T17:07:19.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:19.556 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-12T17:07:19.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:19.636 DEBUG:teuthology.orchestra.run.smithi102:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-12T17:07:19.688 INFO:teuthology.orchestra.run.smithi102.stdout:client4532 2024-03-12T17:07:19.689 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T17:07:19.689 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-12T17:07:19.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:19.739 DEBUG:teuthology.orchestra.run.smithi102:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-12T17:07:19.839 INFO:teuthology.orchestra.run.smithi102.stdout:client4532 2024-03-12T17:07:19.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:19.840 DEBUG:teuthology.orchestra.run.smithi102:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-12T17:07:19.897 INFO:teuthology.orchestra.run.smithi102.stdout:client4532 2024-03-12T17:07:19.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:19.897 DEBUG:teuthology.orchestra.run.smithi102:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3.client4532/status') 2024-03-12T17:07:19.988 INFO:teuthology.orchestra.run.smithi102.stdout:instance: client.4532 (3)192.168.0.1:0/1197806529 2024-03-12T17:07:19.988 INFO:teuthology.orchestra.run.smithi102.stdout:blocklisted: false 2024-03-12T17:07:19.988 INFO:teuthology.orchestra.run.smithi102.stderr:0+1 records in 2024-03-12T17:07:19.988 INFO:teuthology.orchestra.run.smithi102.stderr:0+1 records out 2024-03-12T17:07:19.988 INFO:teuthology.orchestra.run.smithi102.stderr:69 bytes copied, 9.8401e-05 s, 701 kB/s 2024-03-12T17:07:19.992 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-03-12T17:07:19.992 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-12T17:07:19.992 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-12T17:07:19.992 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-12T17:07:20.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.231+0000 7f737ec00640 1 Processor -- start 2024-03-12T17:07:20.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.231+0000 7f737ec00640 1 -- start start 2024-03-12T17:07:20.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.233+0000 7f737ec00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f73781571d0 0x7f73780b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:20.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.233+0000 7f737ec00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f73780cdc00 0x7f737815aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:20.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.233+0000 7f737ec00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73781575b0 con 0x7f737815afe0 2024-03-12T17:07:20.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.233+0000 7f737ec00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7378159f20 con 0x7f73780cdc00 2024-03-12T17:07:20.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.233+0000 7f737ec00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f737815a200 con 0x7f73781571d0 2024-03-12T17:07:20.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.233+0000 7f737ce00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f73780cdc00 0x7f737815aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:20.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.233+0000 7f737ce00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f73780cdc00 0x7f737815aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49192/0 (socket says 172.21.15.52:49192) 2024-03-12T17:07:20.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.233+0000 7f737ce00640 1 -- 172.21.15.52:0/1505483182 learned_addr learned my addr 172.21.15.52:0/1505483182 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:20.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.233+0000 7f737d800640 1 --2- 172.21.15.52:0/1505483182 >> v2:172.21.15.102:3300/0 conn(0x7f73781571d0 0x7f73780b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:20.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.234+0000 7f737ce00640 1 -- 172.21.15.52:0/1505483182 >> v2:172.21.15.102:3300/0 conn(0x7f73781571d0 msgr2=0x7f73780b35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:20.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.234+0000 7f737ce00640 1 --2- 172.21.15.52:0/1505483182 >> v2:172.21.15.102:3300/0 conn(0x7f73781571d0 0x7f73780b35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:20.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.234+0000 7f737ce00640 1 -- 172.21.15.52:0/1505483182 >> v1:172.21.15.102:6789/0 conn(0x7f737815afe0 legacy=0x7f73780cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:20.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.234+0000 7f737ce00640 1 -- 172.21.15.52:0/1505483182 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f737815a4e0 con 0x7f73780cdc00 2024-03-12T17:07:20.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.235+0000 7f737ce00640 1 --2- 172.21.15.52:0/1505483182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73780cdc00 0x7f737815aaa0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f73780712e0 tx=0x7f736402e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf30dc73452e6be server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:20.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.235+0000 7f7377e00640 1 -- 172.21.15.52:0/1505483182 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736402ee90 con 0x7f73780cdc00 2024-03-12T17:07:20.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.235+0000 7f7377e00640 1 -- 172.21.15.52:0/1505483182 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7364036440 con 0x7f73780cdc00 2024-03-12T17:07:20.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.235+0000 7f7377e00640 1 -- 172.21.15.52:0/1505483182 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73640367d0 con 0x7f73780cdc00 2024-03-12T17:07:20.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.236+0000 7f737ec00640 1 -- 172.21.15.52:0/1505483182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73780cdc00 msgr2=0x7f737815aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:20.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.236+0000 7f737ec00640 1 --2- 172.21.15.52:0/1505483182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73780cdc00 0x7f737815aaa0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f73780712e0 tx=0x7f736402e0e0 comp rx=0 tx=0).stop 2024-03-12T17:07:20.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.236+0000 7f737ec00640 1 -- 172.21.15.52:0/1505483182 shutdown_connections 2024-03-12T17:07:20.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.236+0000 7f737ec00640 1 --2- 172.21.15.52:0/1505483182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73780cdc00 0x7f737815aaa0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:20.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.236+0000 7f737ec00640 1 --2- 172.21.15.52:0/1505483182 >> v2:172.21.15.102:3300/0 conn(0x7f73781571d0 0x7f73780b35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:20.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.236+0000 7f737ec00640 1 -- 172.21.15.52:0/1505483182 >> 172.21.15.52:0/1505483182 conn(0x7f7378065df0 msgr2=0x7f7378069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:20.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.236+0000 7f737ec00640 1 -- 172.21.15.52:0/1505483182 shutdown_connections 2024-03-12T17:07:20.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.236+0000 7f737ec00640 1 -- 172.21.15.52:0/1505483182 wait complete. 2024-03-12T17:07:20.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.238+0000 7f737ec00640 1 Processor -- start 2024-03-12T17:07:20.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.238+0000 7f737ec00640 1 -- start start 2024-03-12T17:07:20.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.239+0000 7f737ec00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73780cdc00 0x7f737813cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:20.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.239+0000 7f737ec00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73781571d0 0x7f737813ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:20.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.239+0000 7f737ec00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f737815afe0 0x7f737813f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:20.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.239+0000 7f737ec00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73780ae140 con 0x7f73781571d0 2024-03-12T17:07:20.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.239+0000 7f737ec00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73780ae420 con 0x7f73780cdc00 2024-03-12T17:07:20.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.239+0000 7f737ec00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73780ae700 con 0x7f737815afe0 2024-03-12T17:07:20.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.239+0000 7f737ce00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73781571d0 0x7f737813ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:20.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.239+0000 7f737ce00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73781571d0 0x7f737813ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49202/0 (socket says 172.21.15.52:49202) 2024-03-12T17:07:20.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.239+0000 7f737ce00640 1 -- 172.21.15.52:0/559810149 learned_addr learned my addr 172.21.15.52:0/559810149 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:20.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.239+0000 7f737d800640 1 --2- 172.21.15.52:0/559810149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73780cdc00 0x7f737813cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:20.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.239+0000 7f737e200640 1 --2- 172.21.15.52:0/559810149 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f737815afe0 0x7f737813f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:20.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.239+0000 7f737ce00640 1 -- 172.21.15.52:0/559810149 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f737815afe0 msgr2=0x7f737813f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:20.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.239+0000 7f737ce00640 1 --2- 172.21.15.52:0/559810149 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f737815afe0 0x7f737813f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:20.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.239+0000 7f737ce00640 1 -- 172.21.15.52:0/559810149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73780cdc00 msgr2=0x7f737813cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:20.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.239+0000 7f737ce00640 1 --2- 172.21.15.52:0/559810149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73780cdc00 0x7f737813cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:20.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.239+0000 7f737ce00640 1 -- 172.21.15.52:0/559810149 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f736400b770 con 0x7f73781571d0 2024-03-12T17:07:20.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.240+0000 7f737e200640 1 --2- 172.21.15.52:0/559810149 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f737815afe0 0x7f737813f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:20.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.240+0000 7f737ce00640 1 --2- 172.21.15.52:0/559810149 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73781571d0 0x7f737813ebd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f736402e120 tx=0x7f73640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:20.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.240+0000 7f7375600640 1 -- 172.21.15.52:0/559810149 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73640348d0 con 0x7f73781571d0 2024-03-12T17:07:20.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.240+0000 7f7375600640 1 -- 172.21.15.52:0/559810149 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7364034a30 con 0x7f73781571d0 2024-03-12T17:07:20.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.240+0000 7f7375600640 1 -- 172.21.15.52:0/559810149 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7364034de0 con 0x7f73781571d0 2024-03-12T17:07:20.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.240+0000 7f737ec00640 1 -- 172.21.15.52:0/559810149 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73780ae9e0 con 0x7f73781571d0 2024-03-12T17:07:20.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.240+0000 7f737ec00640 1 -- 172.21.15.52:0/559810149 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f737813fd90 con 0x7f73781571d0 2024-03-12T17:07:20.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.242+0000 7f737ec00640 1 -- 172.21.15.52:0/559810149 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f737804e030 con 0x7f73781571d0 2024-03-12T17:07:20.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.243+0000 7f7375600640 1 -- 172.21.15.52:0/559810149 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f736403d040 con 0x7f73781571d0 2024-03-12T17:07:20.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.243+0000 7f7375600640 1 --2- 172.21.15.52:0/559810149 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f735806ac00 0x7f735806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:20.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.243+0000 7f7375600640 1 -- 172.21.15.52:0/559810149 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f73640ae4c0 con 0x7f73781571d0 2024-03-12T17:07:20.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.244+0000 7f737d800640 1 --2- 172.21.15.52:0/559810149 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f735806ac00 0x7f735806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:20.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.245+0000 7f737d800640 1 --2- 172.21.15.52:0/559810149 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f735806ac00 0x7f735806d0c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7368009a60 tx=0x7f7368003990 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:20.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.248+0000 7f7375600640 1 -- 172.21.15.52:0/559810149 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f7364079250 con 0x7f73781571d0 2024-03-12T17:07:20.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.485+0000 7f737ec00640 1 -- 172.21.15.52:0/559810149 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f73780cc8b0 con 0x7f73781571d0 2024-03-12T17:07:20.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.486+0000 7f7375600640 1 -- 172.21.15.52:0/559810149 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f736407c870 con 0x7f73781571d0 2024-03-12T17:07:20.480 INFO:teuthology.orchestra.run.smithi052.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-12T17:07:20.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.488+0000 7f737ec00640 1 -- 172.21.15.52:0/559810149 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f735806ac00 msgr2=0x7f735806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:20.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.488+0000 7f737ec00640 1 --2- 172.21.15.52:0/559810149 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f735806ac00 0x7f735806d0c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7368009a60 tx=0x7f7368003990 comp rx=0 tx=0).stop 2024-03-12T17:07:20.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.488+0000 7f737ec00640 1 -- 172.21.15.52:0/559810149 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73781571d0 msgr2=0x7f737813ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:20.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.488+0000 7f737ec00640 1 --2- 172.21.15.52:0/559810149 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73781571d0 0x7f737813ebd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f736402e120 tx=0x7f73640046c0 comp rx=0 tx=0).stop 2024-03-12T17:07:20.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.488+0000 7f737ec00640 1 -- 172.21.15.52:0/559810149 shutdown_connections 2024-03-12T17:07:20.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.488+0000 7f737ec00640 1 --2- 172.21.15.52:0/559810149 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f735806ac00 0x7f735806d0c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:20.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.488+0000 7f737ec00640 1 --2- 172.21.15.52:0/559810149 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f737815afe0 0x7f737813f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:20.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.488+0000 7f737ec00640 1 --2- 172.21.15.52:0/559810149 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73781571d0 0x7f737813ebd0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:20.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.488+0000 7f737ec00640 1 --2- 172.21.15.52:0/559810149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73780cdc00 0x7f737813cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:20.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.488+0000 7f737ec00640 1 -- 172.21.15.52:0/559810149 >> 172.21.15.52:0/559810149 conn(0x7f7378065df0 msgr2=0x7f73780cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:20.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.489+0000 7f737ec00640 1 -- 172.21.15.52:0/559810149 shutdown_connections 2024-03-12T17:07:20.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:20.489+0000 7f737ec00640 1 -- 172.21.15.52:0/559810149 wait complete. 2024-03-12T17:07:20.501 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-12T17:07:20.501 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-12T17:07:20.501 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi102.front.sepia.ceph.com 2024-03-12T17:07:20.501 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-03-12T17:07:20.501 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-12T17:07:20.501 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-12T17:07:20.501 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-12T17:07:20.502 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-12T17:07:20.502 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-03-12T17:07:20.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:20.502 DEBUG:teuthology.orchestra.run.smithi102:> ip addr 2024-03-12T17:07:20.560 INFO:teuthology.orchestra.run.smithi102.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-12T17:07:20.560 INFO:teuthology.orchestra.run.smithi102.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-12T17:07:20.560 INFO:teuthology.orchestra.run.smithi102.stdout: inet 127.0.0.1/8 scope host lo 2024-03-12T17:07:20.560 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:07:20.560 INFO:teuthology.orchestra.run.smithi102.stdout: inet6 ::1/128 scope host 2024-03-12T17:07:20.561 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:07:20.561 INFO:teuthology.orchestra.run.smithi102.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-12T17:07:20.561 INFO:teuthology.orchestra.run.smithi102.stdout: link/ether 0c:c4:7a:d8:d9:dc brd ff:ff:ff:ff:ff:ff 2024-03-12T17:07:20.561 INFO:teuthology.orchestra.run.smithi102.stdout: altname enp7s0f0 2024-03-12T17:07:20.561 INFO:teuthology.orchestra.run.smithi102.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-12T17:07:20.561 INFO:teuthology.orchestra.run.smithi102.stdout: link/ether 0c:c4:7a:d8:d9:dd brd ff:ff:ff:ff:ff:ff 2024-03-12T17:07:20.562 INFO:teuthology.orchestra.run.smithi102.stdout: altname enp7s0f1 2024-03-12T17:07:20.562 INFO:teuthology.orchestra.run.smithi102.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-12T17:07:20.562 INFO:teuthology.orchestra.run.smithi102.stdout: link/ether 0c:c4:7a:8f:d7:68 brd ff:ff:ff:ff:ff:ff 2024-03-12T17:07:20.562 INFO:teuthology.orchestra.run.smithi102.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-12T17:07:20.562 INFO:teuthology.orchestra.run.smithi102.stdout: link/ether 0c:c4:7a:8f:d7:69 brd ff:ff:ff:ff:ff:ff 2024-03-12T17:07:20.562 INFO:teuthology.orchestra.run.smithi102.stdout: inet 172.21.15.102/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-12T17:07:20.562 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft 42552sec preferred_lft 42552sec 2024-03-12T17:07:20.562 INFO:teuthology.orchestra.run.smithi102.stdout: inet6 fe80::ec4:7aff:fe8f:d769/64 scope link 2024-03-12T17:07:20.562 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:07:20.562 INFO:teuthology.orchestra.run.smithi102.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-12T17:07:20.562 INFO:teuthology.orchestra.run.smithi102.stdout: link/ether 4e:3d:ab:1c:16:27 brd ff:ff:ff:ff:ff:ff 2024-03-12T17:07:20.563 INFO:teuthology.orchestra.run.smithi102.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-12T17:07:20.563 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:07:20.563 INFO:teuthology.orchestra.run.smithi102.stdout: inet6 fe80::70a9:99ff:fe90:9225/64 scope link 2024-03-12T17:07:20.563 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:07:20.563 INFO:teuthology.orchestra.run.smithi102.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-12T17:07:20.563 INFO:teuthology.orchestra.run.smithi102.stdout: link/ether 4e:3d:ab:1c:16:27 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-12T17:07:20.563 INFO:teuthology.orchestra.run.smithi102.stdout: inet6 fe80::4c3d:abff:fe1c:1627/64 scope link 2024-03-12T17:07:20.563 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:07:20.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:20.565 DEBUG:teuthology.orchestra.run.smithi102:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-12T17:07:20.654 INFO:teuthology.orchestra.run.smithi102.stdout:1 2024-03-12T17:07:20.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:20.658 DEBUG:teuthology.orchestra.run.smithi102:> route 2024-03-12T17:07:20.713 INFO:teuthology.orchestra.run.smithi102.stdout:Kernel IP routing table 2024-03-12T17:07:20.714 INFO:teuthology.orchestra.run.smithi102.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-12T17:07:20.714 INFO:teuthology.orchestra.run.smithi102.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-12T17:07:20.714 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-12T17:07:20.714 INFO:teuthology.orchestra.run.smithi102.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-12T17:07:20.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:20.714 DEBUG:teuthology.orchestra.run.smithi102:> (cd / && exec bash -c ' 2024-03-12T17:07:20.714 DEBUG:teuthology.orchestra.run.smithi102:> set -e 2024-03-12T17:07:20.714 DEBUG:teuthology.orchestra.run.smithi102:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-12T17:07:20.714 DEBUG:teuthology.orchestra.run.smithi102:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-12T17:07:20.717 DEBUG:teuthology.orchestra.run.smithi102:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-12T17:07:20.718 DEBUG:teuthology.orchestra.run.smithi102:> ') 2024-03-12T17:07:20.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:20.896 DEBUG:teuthology.orchestra.run.smithi102:> ip netns list 2024-03-12T17:07:20.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-12T17:07:20.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:20.948 DEBUG:teuthology.orchestra.run.smithi102:> ip netns list-id 2024-03-12T17:07:20.999 INFO:teuthology.orchestra.run.smithi102.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-03-12T17:07:21.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:21.000 DEBUG:teuthology.orchestra.run.smithi102:> (cd / && exec bash -c ' 2024-03-12T17:07:21.000 DEBUG:teuthology.orchestra.run.smithi102:> set -e 2024-03-12T17:07:21.000 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-12T17:07:21.000 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-03-12T17:07:21.000 DEBUG:teuthology.orchestra.run.smithi102:> ') 2024-03-12T17:07:21.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:21.131 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-12T17:07:21.214 INFO:teuthology.orchestra.run.smithi102.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-12T17:07:21.215 INFO:teuthology.orchestra.run.smithi102.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-12T17:07:21.215 INFO:teuthology.orchestra.run.smithi102.stdout: inet 127.0.0.1/8 scope host lo 2024-03-12T17:07:21.215 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:07:21.215 INFO:teuthology.orchestra.run.smithi102.stdout: inet6 ::1/128 scope host 2024-03-12T17:07:21.215 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:07:21.215 INFO:teuthology.orchestra.run.smithi102.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-12T17:07:21.215 INFO:teuthology.orchestra.run.smithi102.stdout: link/ether 12:33:e4:7f:22:9f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-12T17:07:21.215 INFO:teuthology.orchestra.run.smithi102.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-12T17:07:21.215 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:07:21.215 INFO:teuthology.orchestra.run.smithi102.stdout: inet6 fe80::1033:e4ff:fe7f:229f/64 scope link 2024-03-12T17:07:21.215 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:07:21.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:21.219 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-12T17:07:21.308 INFO:teuthology.orchestra.run.smithi102.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-12T17:07:21.309 INFO:teuthology.orchestra.run.smithi102.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-12T17:07:21.309 INFO:teuthology.orchestra.run.smithi102.stdout: inet 127.0.0.1/8 scope host lo 2024-03-12T17:07:21.309 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:07:21.309 INFO:teuthology.orchestra.run.smithi102.stdout: inet6 ::1/128 scope host 2024-03-12T17:07:21.309 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:07:21.309 INFO:teuthology.orchestra.run.smithi102.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-12T17:07:21.309 INFO:teuthology.orchestra.run.smithi102.stdout: link/ether 12:33:e4:7f:22:9f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-12T17:07:21.309 INFO:teuthology.orchestra.run.smithi102.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-12T17:07:21.309 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:07:21.309 INFO:teuthology.orchestra.run.smithi102.stdout: inet6 fe80::1033:e4ff:fe7f:229f/64 scope link 2024-03-12T17:07:21.309 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:07:21.313 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-03-12T17:07:21.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:21.313 DEBUG:teuthology.orchestra.run.smithi102:> (cd / && exec bash -c ' 2024-03-12T17:07:21.314 DEBUG:teuthology.orchestra.run.smithi102:> set -e 2024-03-12T17:07:21.314 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-03-12T17:07:21.314 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-12T17:07:21.314 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-03-12T17:07:21.314 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-03-12T17:07:21.314 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-03-12T17:07:21.314 DEBUG:teuthology.orchestra.run.smithi102:> ') 2024-03-12T17:07:21.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:21.576 DEBUG:teuthology.orchestra.run.smithi102:> (cd / && exec bash -c ' 2024-03-12T17:07:21.576 DEBUG:teuthology.orchestra.run.smithi102:> set -e 2024-03-12T17:07:21.576 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link set brx.1 up 2024-03-12T17:07:21.576 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link set dev brx.1 master ceph-brx 2024-03-12T17:07:21.576 DEBUG:teuthology.orchestra.run.smithi102:> ') 2024-03-12T17:07:21.733 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-12T17:07:21.733 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-03-12T17:07:21.776 INFO:teuthology.orchestra.run.smithi102.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-03-12T17:07:21.776 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-12T17:07:21.777 DEBUG:teuthology.orchestra.run.smithi102:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-03-12T17:07:21.870 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-03-12T17:07:21.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:21.870 DEBUG:teuthology.orchestra.run.smithi102:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-12T17:07:22.002 INFO:teuthology.orchestra.run.smithi102.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-12T17:07:22.003 INFO:teuthology.orchestra.run.smithi102.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-12T17:07:22.035 INFO:teuthology.orchestra.run.smithi102.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-12T17:07:22.035 INFO:teuthology.orchestra.run.smithi102.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-12T17:07:22.035 INFO:teuthology.orchestra.run.smithi102.stdout:mount.ceph: resolved to: "172.21.15.52:3300,172.21.15.102:3300,172.21.15.102:3301" 2024-03-12T17:07:22.035 INFO:teuthology.orchestra.run.smithi102.stdout:mount.ceph: trying mount with new device syntax: 3@524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3.cephfs=/ 2024-03-12T17:07:22.035 INFO:teuthology.orchestra.run.smithi102.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.52:3300/172.21.15.102:3300/172.21.15.102:3301" will pass to kernel 2024-03-12T17:07:22.037 INFO:teuthology.orchestra.run.smithi102.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-03-12T17:07:22.038 INFO:teuthology.orchestra.run.smithi102.stdout: You just mounted a file system that supports labels which does not 2024-03-12T17:07:22.038 INFO:teuthology.orchestra.run.smithi102.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-12T17:07:22.038 INFO:teuthology.orchestra.run.smithi102.stdout: applications will generate AVC messages and not be allowed access to 2024-03-12T17:07:22.038 INFO:teuthology.orchestra.run.smithi102.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-12T17:07:22.041 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-12T17:07:22.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:22.041 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-03-12T17:07:22.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:22.117 DEBUG:teuthology.orchestra.run.smithi102:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-12T17:07:22.170 INFO:teuthology.orchestra.run.smithi102.stdout:client4535 2024-03-12T17:07:22.171 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T17:07:22.171 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-12T17:07:22.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:22.218 DEBUG:teuthology.orchestra.run.smithi102:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-12T17:07:22.319 INFO:teuthology.orchestra.run.smithi102.stdout:client4535 2024-03-12T17:07:22.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:22.319 DEBUG:teuthology.orchestra.run.smithi102:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-12T17:07:22.381 INFO:teuthology.orchestra.run.smithi102.stdout:client4535 2024-03-12T17:07:22.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:22.382 DEBUG:teuthology.orchestra.run.smithi102:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3.client4535/status') 2024-03-12T17:07:22.472 INFO:teuthology.orchestra.run.smithi102.stdout:instance: client.4535 (3)192.168.0.2:0/2811741634 2024-03-12T17:07:22.473 INFO:teuthology.orchestra.run.smithi102.stdout:blocklisted: false 2024-03-12T17:07:22.473 INFO:teuthology.orchestra.run.smithi102.stderr:0+1 records in 2024-03-12T17:07:22.473 INFO:teuthology.orchestra.run.smithi102.stderr:0+1 records out 2024-03-12T17:07:22.473 INFO:teuthology.orchestra.run.smithi102.stderr:69 bytes copied, 9.8929e-05 s, 697 kB/s 2024-03-12T17:07:22.477 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-03-12T17:07:22.515 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:07:22.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.757+0000 7fe37a400640 1 Processor -- start 2024-03-12T17:07:22.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.757+0000 7fe37a400640 1 -- start start 2024-03-12T17:07:22.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.758+0000 7fe37a400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe3740b5a30 0x7fe3740b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:22.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.758+0000 7fe37a400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe3740b4350 0x7fe3740b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:22.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.758+0000 7fe37a400640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3740d1830 con 0x7fe37415cda0 2024-03-12T17:07:22.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.758+0000 7fe37a400640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3740b4d20 con 0x7fe3740b5a30 2024-03-12T17:07:22.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.758+0000 7fe37a400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3740b1f20 con 0x7fe3740b4350 2024-03-12T17:07:22.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.759+0000 7fe373400640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fe37415cda0 0x7fe3740cdb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:57320/0 (socket says 172.21.15.52:57320) 2024-03-12T17:07:22.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.759+0000 7fe373400640 1 -- 172.21.15.52:0/399553398 learned_addr learned my addr 172.21.15.52:0/399553398 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:22.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.759+0000 7fe36aa00640 1 --2- 172.21.15.52:0/399553398 >> v2:172.21.15.52:3300/0 conn(0x7fe3740b5a30 0x7fe3740b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:22.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.759+0000 7fe373e00640 1 --2- 172.21.15.52:0/399553398 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe3740b4350 0x7fe3740b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:22.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.759+0000 7fe372a00640 1 -- 172.21.15.52:0/399553398 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1967326532 0 0) 0x7fe3740d1830 con 0x7fe37415cda0 2024-03-12T17:07:22.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.759+0000 7fe372a00640 1 -- 172.21.15.52:0/399553398 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe358003520 con 0x7fe37415cda0 2024-03-12T17:07:22.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.759+0000 7fe36aa00640 1 -- 172.21.15.52:0/399553398 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe3740b4350 msgr2=0x7fe3740b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:22.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.759+0000 7fe36aa00640 1 --2- 172.21.15.52:0/399553398 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe3740b4350 0x7fe3740b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:22.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.759+0000 7fe36aa00640 1 -- 172.21.15.52:0/399553398 >> v1:172.21.15.52:6789/0 conn(0x7fe37415cda0 legacy=0x7fe3740cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:22.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.759+0000 7fe36aa00640 1 -- 172.21.15.52:0/399553398 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3740d0c20 con 0x7fe3740b5a30 2024-03-12T17:07:22.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.761+0000 7fe36aa00640 1 --2- 172.21.15.52:0/399553398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3740b5a30 0x7fe3740b5e90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe35c0097a0 tx=0x7fe35c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0ea1a902f6f48e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:22.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.761+0000 7fe372a00640 1 -- 172.21.15.52:0/399553398 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c02d040 con 0x7fe3740b5a30 2024-03-12T17:07:22.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.761+0000 7fe372a00640 1 -- 172.21.15.52:0/399553398 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35c031810 con 0x7fe3740b5a30 2024-03-12T17:07:22.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.761+0000 7fe372a00640 1 -- 172.21.15.52:0/399553398 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c031ba0 con 0x7fe3740b5a30 2024-03-12T17:07:22.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.761+0000 7fe37a400640 1 -- 172.21.15.52:0/399553398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3740b5a30 msgr2=0x7fe3740b5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:22.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.761+0000 7fe37a400640 1 --2- 172.21.15.52:0/399553398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3740b5a30 0x7fe3740b5e90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe35c0097a0 tx=0x7fe35c02b8b0 comp rx=0 tx=0).stop 2024-03-12T17:07:22.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.762+0000 7fe37a400640 1 -- 172.21.15.52:0/399553398 shutdown_connections 2024-03-12T17:07:22.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.762+0000 7fe37a400640 1 --2- 172.21.15.52:0/399553398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3740b5a30 0x7fe3740b5e90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:22.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.762+0000 7fe37a400640 1 --2- 172.21.15.52:0/399553398 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe3740b4350 0x7fe3740b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:22.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.762+0000 7fe37a400640 1 -- 172.21.15.52:0/399553398 >> 172.21.15.52:0/399553398 conn(0x7fe3740cf350 msgr2=0x7fe3740cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:22.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.762+0000 7fe37a400640 1 -- 172.21.15.52:0/399553398 shutdown_connections 2024-03-12T17:07:22.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.762+0000 7fe37a400640 1 -- 172.21.15.52:0/399553398 wait complete. 2024-03-12T17:07:22.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.763+0000 7fe37a400640 1 Processor -- start 2024-03-12T17:07:22.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.764+0000 7fe37a400640 1 -- start start 2024-03-12T17:07:22.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.765+0000 7fe37a400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3740b4350 0x7fe374159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:22.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.765+0000 7fe37a400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3740b5a30 0x7fe37415ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:22.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.765+0000 7fe37a400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe37415cda0 0x7fe3740cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:22.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.765+0000 7fe37a400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3740c7010 con 0x7fe3740b4350 2024-03-12T17:07:22.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.765+0000 7fe37a400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3740c6440 con 0x7fe3740b5a30 2024-03-12T17:07:22.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.765+0000 7fe37a400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3740c66e0 con 0x7fe37415cda0 2024-03-12T17:07:22.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.765+0000 7fe373400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3740b4350 0x7fe374159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:22.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.765+0000 7fe373400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3740b4350 0x7fe374159ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49218/0 (socket says 172.21.15.52:49218) 2024-03-12T17:07:22.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.765+0000 7fe373400640 1 -- 172.21.15.52:0/1065749783 learned_addr learned my addr 172.21.15.52:0/1065749783 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:22.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.765+0000 7fe373e00640 1 --2- 172.21.15.52:0/1065749783 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe37415cda0 0x7fe3740cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:22.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.765+0000 7fe36aa00640 1 --2- 172.21.15.52:0/1065749783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3740b5a30 0x7fe37415ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:22.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.765+0000 7fe373400640 1 -- 172.21.15.52:0/1065749783 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe37415cda0 msgr2=0x7fe3740cb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:22.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.765+0000 7fe373400640 1 --2- 172.21.15.52:0/1065749783 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe37415cda0 0x7fe3740cb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:22.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.765+0000 7fe373400640 1 -- 172.21.15.52:0/1065749783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3740b5a30 msgr2=0x7fe37415ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:22.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.765+0000 7fe373400640 1 --2- 172.21.15.52:0/1065749783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3740b5a30 0x7fe37415ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:22.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.765+0000 7fe373400640 1 -- 172.21.15.52:0/1065749783 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c009400 con 0x7fe3740b4350 2024-03-12T17:07:22.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.765+0000 7fe36aa00640 1 --2- 172.21.15.52:0/1065749783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3740b5a30 0x7fe37415ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:22.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.766+0000 7fe373400640 1 --2- 172.21.15.52:0/1065749783 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3740b4350 0x7fe374159ee0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe36400c410 tx=0x7fe36400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:22.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.766+0000 7fe36be00640 1 -- 172.21.15.52:0/1065749783 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe364005ce0 con 0x7fe3740b4350 2024-03-12T17:07:22.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.766+0000 7fe36be00640 1 -- 172.21.15.52:0/1065749783 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe364005e40 con 0x7fe3740b4350 2024-03-12T17:07:22.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.766+0000 7fe36be00640 1 -- 172.21.15.52:0/1065749783 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3640072f0 con 0x7fe3740b4350 2024-03-12T17:07:22.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.766+0000 7fe37a400640 1 -- 172.21.15.52:0/1065749783 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3740c69c0 con 0x7fe3740b4350 2024-03-12T17:07:22.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.766+0000 7fe37a400640 1 -- 172.21.15.52:0/1065749783 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe37415a420 con 0x7fe3740b4350 2024-03-12T17:07:22.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.768+0000 7fe37a400640 1 -- 172.21.15.52:0/1065749783 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe374073390 con 0x7fe3740b4350 2024-03-12T17:07:22.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.769+0000 7fe36be00640 1 -- 172.21.15.52:0/1065749783 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe364007510 con 0x7fe3740b4350 2024-03-12T17:07:22.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.769+0000 7fe36be00640 1 --2- 172.21.15.52:0/1065749783 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe35006ac00 0x7fe35006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:22.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.770+0000 7fe36be00640 1 -- 172.21.15.52:0/1065749783 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe364088d80 con 0x7fe3740b4350 2024-03-12T17:07:22.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.770+0000 7fe36aa00640 1 --2- 172.21.15.52:0/1065749783 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe35006ac00 0x7fe35006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:22.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.772+0000 7fe36aa00640 1 --2- 172.21.15.52:0/1065749783 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe35006ac00 0x7fe35006d0c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe35c036000 tx=0x7fe35c03c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:22.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.775+0000 7fe36be00640 1 -- 172.21.15.52:0/1065749783 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fe364053c60 con 0x7fe3740b4350 2024-03-12T17:07:22.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.974+0000 7fe37a400640 1 -- 172.21.15.52:0/1065749783 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3740d29a0 con 0x7fe3740b4350 2024-03-12T17:07:22.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.977+0000 7fe36be00640 1 -- 172.21.15.52:0/1065749783 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18207 (secure 0 0 0) 0x7fe364017090 con 0x7fe3740b4350 2024-03-12T17:07:22.971 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:07:22.972 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":27,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:07:12.969614+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T17:06:19.218543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T17:06:23.525474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T17:06:24.911574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-12T17:06:28.017172+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.733031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.775178+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.591522+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.820838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.829252+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.582481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.18","mappings":[{"from":3,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:07:22.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.980+0000 7fe37a400640 1 -- 172.21.15.52:0/1065749783 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe35006ac00 msgr2=0x7fe35006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:22.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.980+0000 7fe37a400640 1 --2- 172.21.15.52:0/1065749783 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe35006ac00 0x7fe35006d0c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe35c036000 tx=0x7fe35c03c040 comp rx=0 tx=0).stop 2024-03-12T17:07:22.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.980+0000 7fe37a400640 1 -- 172.21.15.52:0/1065749783 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3740b4350 msgr2=0x7fe374159ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:22.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.980+0000 7fe37a400640 1 --2- 172.21.15.52:0/1065749783 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3740b4350 0x7fe374159ee0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe36400c410 tx=0x7fe36400c8d0 comp rx=0 tx=0).stop 2024-03-12T17:07:22.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.980+0000 7fe37a400640 1 -- 172.21.15.52:0/1065749783 shutdown_connections 2024-03-12T17:07:22.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.980+0000 7fe37a400640 1 --2- 172.21.15.52:0/1065749783 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe35006ac00 0x7fe35006d0c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:22.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.980+0000 7fe37a400640 1 --2- 172.21.15.52:0/1065749783 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe37415cda0 0x7fe3740cb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:22.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.980+0000 7fe37a400640 1 --2- 172.21.15.52:0/1065749783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3740b5a30 0x7fe37415ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:22.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.980+0000 7fe37a400640 1 --2- 172.21.15.52:0/1065749783 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3740b4350 0x7fe374159ee0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:22.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.980+0000 7fe37a400640 1 -- 172.21.15.52:0/1065749783 >> 172.21.15.52:0/1065749783 conn(0x7fe3740cf350 msgr2=0x7fe3740b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:22.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.980+0000 7fe37a400640 1 -- 172.21.15.52:0/1065749783 shutdown_connections 2024-03-12T17:07:22.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:22.980+0000 7fe37a400640 1 -- 172.21.15.52:0/1065749783 wait complete. 2024-03-12T17:07:22.995 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T17:06:19.218543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-12T17:06:23.525474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-12T17:06:24.911574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-12T17:06:28.017172+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-12T17:07:22.996 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T17:07:23.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.220+0000 7fc040800640 1 Processor -- start 2024-03-12T17:07:23.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.220+0000 7fc040800640 1 -- start start 2024-03-12T17:07:23.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.222+0000 7fc040800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc03815dd40 0x7fc038071c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:23.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.222+0000 7fc040800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc0380cc070 0x7fc038069c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:23.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.222+0000 7fc040800640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc03815e540 con 0x7fc0380cc4b0 2024-03-12T17:07:23.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.222+0000 7fc040800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0380af0e0 con 0x7fc03815dd40 2024-03-12T17:07:23.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.222+0000 7fc040800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0380cea20 con 0x7fc0380cc070 2024-03-12T17:07:23.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.222+0000 7fc03e800640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fc0380cc4b0 0x7fc03815e9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:57328/0 (socket says 172.21.15.52:57328) 2024-03-12T17:07:23.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.223+0000 7fc03e800640 1 -- 172.21.15.52:0/3301712120 learned_addr learned my addr 172.21.15.52:0/3301712120 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:23.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.223+0000 7fc03de00640 1 --2- 172.21.15.52:0/3301712120 >> v2:172.21.15.102:3300/0 conn(0x7fc03815dd40 0x7fc038071c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:23.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.223+0000 7fc03d400640 1 --2- 172.21.15.52:0/3301712120 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc0380cc070 0x7fc038069c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:23.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.223+0000 7fc03ca00640 1 -- 172.21.15.52:0/3301712120 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 449763153 0 0) 0x7fc03815e540 con 0x7fc0380cc4b0 2024-03-12T17:07:23.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.223+0000 7fc03ca00640 1 -- 172.21.15.52:0/3301712120 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc020003520 con 0x7fc0380cc4b0 2024-03-12T17:07:23.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.224+0000 7fc03ca00640 1 -- 172.21.15.52:0/3301712120 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 414434305 0 0) 0x7fc020003520 con 0x7fc0380cc4b0 2024-03-12T17:07:23.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.224+0000 7fc03ca00640 1 -- 172.21.15.52:0/3301712120 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc0380cc070 msgr2=0x7fc038069c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:23.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.224+0000 7fc03ca00640 1 --2- 172.21.15.52:0/3301712120 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc0380cc070 0x7fc038069c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:23.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.224+0000 7fc03ca00640 1 -- 172.21.15.52:0/3301712120 >> v2:172.21.15.102:3300/0 conn(0x7fc03815dd40 msgr2=0x7fc038071c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:23.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.224+0000 7fc03ca00640 1 --2- 172.21.15.52:0/3301712120 >> v2:172.21.15.102:3300/0 conn(0x7fc03815dd40 0x7fc038071c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:23.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.224+0000 7fc03ca00640 1 -- 172.21.15.52:0/3301712120 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0380ced00 con 0x7fc0380cc4b0 2024-03-12T17:07:23.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.224+0000 7fc03ca00640 1 -- 172.21.15.52:0/3301712120 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7fc0340019f0 con 0x7fc0380cc4b0 2024-03-12T17:07:23.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.224+0000 7fc03ca00640 1 -- 172.21.15.52:0/3301712120 >> v1:172.21.15.52:6789/0 conn(0x7fc0380cc4b0 legacy=0x7fc03815e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:23.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.224+0000 7fc03ca00640 1 --2- 172.21.15.52:0/3301712120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc020003970 0x7fc020003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:23.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.224+0000 7fc03ca00640 1 --2- 172.21.15.52:0/3301712120 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc0380cc070 0x7fc0200047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:23.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.224+0000 7fc03ca00640 1 --2- 172.21.15.52:0/3301712120 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc020004d00 0x7fc020005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:23.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.224+0000 7fc03ca00640 1 -- 172.21.15.52:0/3301712120 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0380af0e0 con 0x7fc020003970 2024-03-12T17:07:23.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.224+0000 7fc03ca00640 1 -- 172.21.15.52:0/3301712120 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0380cea20 con 0x7fc020004d00 2024-03-12T17:07:23.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.224+0000 7fc03ca00640 1 -- 172.21.15.52:0/3301712120 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc02000b9b0 con 0x7fc0380cc070 2024-03-12T17:07:23.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.224+0000 7fc03de00640 1 --2- 172.21.15.52:0/3301712120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc020003970 0x7fc020003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:23.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.224+0000 7fc03d400640 1 --2- 172.21.15.52:0/3301712120 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc0380cc070 0x7fc0200047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:23.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.224+0000 7fc03e800640 1 --2- 172.21.15.52:0/3301712120 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc020004d00 0x7fc020005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:23.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.225+0000 7fc03de00640 1 -- 172.21.15.52:0/3301712120 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc0380cc070 msgr2=0x7fc0200047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:23.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.225+0000 7fc03de00640 1 --2- 172.21.15.52:0/3301712120 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc0380cc070 0x7fc0200047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:23.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.225+0000 7fc03de00640 1 -- 172.21.15.52:0/3301712120 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc020004d00 msgr2=0x7fc020005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:23.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.225+0000 7fc03de00640 1 --2- 172.21.15.52:0/3301712120 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc020004d00 0x7fc020005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:23.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.225+0000 7fc03de00640 1 -- 172.21.15.52:0/3301712120 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc020003520 con 0x7fc020003970 2024-03-12T17:07:23.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.226+0000 7fc03de00640 1 --2- 172.21.15.52:0/3301712120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc020003970 0x7fc020003d90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009ac0 tx=0x7fc0200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d139d7e7143a18b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:23.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.227+0000 7fc03ca00640 1 -- 172.21.15.52:0/3301712120 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c02c710 con 0x7fc020003970 2024-03-12T17:07:23.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.227+0000 7fc03ca00640 1 -- 172.21.15.52:0/3301712120 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02c02c870 con 0x7fc020003970 2024-03-12T17:07:23.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.227+0000 7fc03ca00640 1 -- 172.21.15.52:0/3301712120 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c02cc00 con 0x7fc020003970 2024-03-12T17:07:23.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.227+0000 7fc040800640 1 -- 172.21.15.52:0/3301712120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc020003970 msgr2=0x7fc020003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:23.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.227+0000 7fc040800640 1 --2- 172.21.15.52:0/3301712120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc020003970 0x7fc020003d90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009ac0 tx=0x7fc0200057f0 comp rx=0 tx=0).stop 2024-03-12T17:07:23.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.227+0000 7fc03e800640 1 -- 172.21.15.52:0/3301712120 reap_dead start 2024-03-12T17:07:23.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.228+0000 7fc040800640 1 -- 172.21.15.52:0/3301712120 shutdown_connections 2024-03-12T17:07:23.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.228+0000 7fc040800640 1 -- 172.21.15.52:0/3301712120 >> 172.21.15.52:0/3301712120 conn(0x7fc03806d440 msgr2=0x7fc03806d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:23.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.228+0000 7fc040800640 1 -- 172.21.15.52:0/3301712120 shutdown_connections 2024-03-12T17:07:23.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.228+0000 7fc040800640 1 -- 172.21.15.52:0/3301712120 wait complete. 2024-03-12T17:07:23.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.230+0000 7fc040800640 1 Processor -- start 2024-03-12T17:07:23.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.230+0000 7fc040800640 1 -- start start 2024-03-12T17:07:23.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.231+0000 7fc040800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc020003970 0x7fc038155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:23.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.231+0000 7fc040800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc03814cc10 0x7fc038153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:23.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.231+0000 7fc040800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc0380cc970 0x7fc0380ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:23.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.231+0000 7fc040800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0380b27e0 con 0x7fc03814cc10 2024-03-12T17:07:23.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.231+0000 7fc040800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0380cd330 con 0x7fc020003970 2024-03-12T17:07:23.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.231+0000 7fc040800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc03814d090 con 0x7fc0380cc970 2024-03-12T17:07:23.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.231+0000 7fc03d400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc03814cc10 0x7fc038153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:23.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.232+0000 7fc03d400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc03814cc10 0x7fc038153e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49238/0 (socket says 172.21.15.52:49238) 2024-03-12T17:07:23.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.232+0000 7fc03d400640 1 -- 172.21.15.52:0/1442664458 learned_addr learned my addr 172.21.15.52:0/1442664458 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:23.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.232+0000 7fc03e800640 1 --2- 172.21.15.52:0/1442664458 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc0380cc970 0x7fc0380ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:23.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.232+0000 7fc03de00640 1 --2- 172.21.15.52:0/1442664458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc020003970 0x7fc038155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:23.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.232+0000 7fc03d400640 1 -- 172.21.15.52:0/1442664458 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc0380cc970 msgr2=0x7fc0380ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:23.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.232+0000 7fc03d400640 1 --2- 172.21.15.52:0/1442664458 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc0380cc970 0x7fc0380ccdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:23.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.232+0000 7fc03d400640 1 -- 172.21.15.52:0/1442664458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc020003970 msgr2=0x7fc038155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:23.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.232+0000 7fc03d400640 1 --2- 172.21.15.52:0/1442664458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc020003970 0x7fc038155d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:23.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.232+0000 7fc03d400640 1 -- 172.21.15.52:0/1442664458 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0381546b0 con 0x7fc03814cc10 2024-03-12T17:07:23.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.232+0000 7fc03d400640 1 --2- 172.21.15.52:0/1442664458 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc03814cc10 0x7fc038153e40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc024011600 tx=0x7fc024011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:23.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.233+0000 7fc032000640 1 -- 172.21.15.52:0/1442664458 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02400f920 con 0x7fc03814cc10 2024-03-12T17:07:23.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.233+0000 7fc032000640 1 -- 172.21.15.52:0/1442664458 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02400fa80 con 0x7fc03814cc10 2024-03-12T17:07:23.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.233+0000 7fc032000640 1 -- 172.21.15.52:0/1442664458 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02400fdf0 con 0x7fc03814cc10 2024-03-12T17:07:23.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.233+0000 7fc040800640 1 -- 172.21.15.52:0/1442664458 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc038154990 con 0x7fc03814cc10 2024-03-12T17:07:23.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.233+0000 7fc040800640 1 -- 172.21.15.52:0/1442664458 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc038154f50 con 0x7fc03814cc10 2024-03-12T17:07:23.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.234+0000 7fc040800640 1 -- 172.21.15.52:0/1442664458 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc038066bb0 con 0x7fc03814cc10 2024-03-12T17:07:23.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.235+0000 7fc032000640 1 -- 172.21.15.52:0/1442664458 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc024015460 con 0x7fc03814cc10 2024-03-12T17:07:23.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.236+0000 7fc032000640 1 --2- 172.21.15.52:0/1442664458 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc01806ac00 0x7fc01806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:23.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.236+0000 7fc032000640 1 -- 172.21.15.52:0/1442664458 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc024089ce0 con 0x7fc03814cc10 2024-03-12T17:07:23.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.237+0000 7fc03de00640 1 --2- 172.21.15.52:0/1442664458 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc01806ac00 0x7fc01806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:23.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.238+0000 7fc03de00640 1 --2- 172.21.15.52:0/1442664458 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc01806ac00 0x7fc01806d0c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc02c036000 tx=0x7fc02c030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:23.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.242+0000 7fc032000640 1 -- 172.21.15.52:0/1442664458 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fc024054a60 con 0x7fc03814cc10 2024-03-12T17:07:23.329 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-12T17:07:23.337 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-12T17:07:23.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.443+0000 7fc040800640 1 -- 172.21.15.52:0/1442664458 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc038071520 con 0x7fc03814cc10 2024-03-12T17:07:23.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.444+0000 7fc032000640 1 -- 172.21.15.52:0/1442664458 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc0240220a0 con 0x7fc03814cc10 2024-03-12T17:07:23.438 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-03-12T17:07:23.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.447+0000 7fc040800640 1 -- 172.21.15.52:0/1442664458 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc01806ac00 msgr2=0x7fc01806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:23.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.447+0000 7fc040800640 1 --2- 172.21.15.52:0/1442664458 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc01806ac00 0x7fc01806d0c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc02c036000 tx=0x7fc02c030000 comp rx=0 tx=0).stop 2024-03-12T17:07:23.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.447+0000 7fc040800640 1 -- 172.21.15.52:0/1442664458 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc03814cc10 msgr2=0x7fc038153e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:23.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.447+0000 7fc040800640 1 --2- 172.21.15.52:0/1442664458 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc03814cc10 0x7fc038153e40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc024011600 tx=0x7fc024011ac0 comp rx=0 tx=0).stop 2024-03-12T17:07:23.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.447+0000 7fc040800640 1 -- 172.21.15.52:0/1442664458 shutdown_connections 2024-03-12T17:07:23.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.447+0000 7fc040800640 1 --2- 172.21.15.52:0/1442664458 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc01806ac00 0x7fc01806d0c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:23.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.447+0000 7fc040800640 1 --2- 172.21.15.52:0/1442664458 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc0380cc970 0x7fc0380ccdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:23.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.447+0000 7fc040800640 1 --2- 172.21.15.52:0/1442664458 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc03814cc10 0x7fc038153e40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:23.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.447+0000 7fc040800640 1 --2- 172.21.15.52:0/1442664458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc020003970 0x7fc038155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:23.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.447+0000 7fc040800640 1 -- 172.21.15.52:0/1442664458 >> 172.21.15.52:0/1442664458 conn(0x7fc03806d440 msgr2=0x7fc0380cf500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:23.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.447+0000 7fc040800640 1 -- 172.21.15.52:0/1442664458 shutdown_connections 2024-03-12T17:07:23.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.447+0000 7fc040800640 1 -- 172.21.15.52:0/1442664458 wait complete. 2024-03-12T17:07:23.458 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-12T17:07:23.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.705+0000 7f6304e00640 1 Processor -- start 2024-03-12T17:07:23.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.705+0000 7f6304e00640 1 -- start start 2024-03-12T17:07:23.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.706+0000 7f6304e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63000add30 0x7f63000ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:23.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.706+0000 7f6304e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f630006ce90 0x7f630006d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:23.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.707+0000 7f6304e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63000cf250 con 0x7f63000ce930 2024-03-12T17:07:23.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.707+0000 7f6304e00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f630006c760 con 0x7f630006ce90 2024-03-12T17:07:23.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.707+0000 7f6304e00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63000b42d0 con 0x7f63000add30 2024-03-12T17:07:23.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.707+0000 7f62fe800640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f630006ce90 0x7f630006d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:23.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.707+0000 7f62fe800640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f630006ce90 0x7f630006d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49250/0 (socket says 172.21.15.52:49250) 2024-03-12T17:07:23.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.707+0000 7f62fe800640 1 -- 172.21.15.52:0/2578609360 learned_addr learned my addr 172.21.15.52:0/2578609360 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:23.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.707+0000 7f62f5400640 1 --2- 172.21.15.52:0/2578609360 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63000add30 0x7f63000ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:23.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.707+0000 7f62fe800640 1 -- 172.21.15.52:0/2578609360 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63000add30 msgr2=0x7f63000ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:23.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.707+0000 7f62fe800640 1 --2- 172.21.15.52:0/2578609360 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63000add30 0x7f63000ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:23.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.707+0000 7f62fe800640 1 -- 172.21.15.52:0/2578609360 >> v1:172.21.15.102:6789/0 conn(0x7f63000ce930 legacy=0x7f63000cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:23.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.707+0000 7f62fe800640 1 -- 172.21.15.52:0/2578609360 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63000cdba0 con 0x7f630006ce90 2024-03-12T17:07:23.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.707+0000 7f62f5400640 1 --2- 172.21.15.52:0/2578609360 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63000add30 0x7f63000ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:23.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.709+0000 7f62fe800640 1 --2- 172.21.15.52:0/2578609360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f630006ce90 0x7f630006d320 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f62f8009e50 tx=0x7f62f802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c790a66334f81ecf server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:23.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.709+0000 7f62fd400640 1 -- 172.21.15.52:0/2578609360 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f802ca80 con 0x7f630006ce90 2024-03-12T17:07:23.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.709+0000 7f62fd400640 1 -- 172.21.15.52:0/2578609360 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f802cbe0 con 0x7f630006ce90 2024-03-12T17:07:23.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.709+0000 7f62fd400640 1 -- 172.21.15.52:0/2578609360 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f80343f0 con 0x7f630006ce90 2024-03-12T17:07:23.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.710+0000 7f6304e00640 1 -- 172.21.15.52:0/2578609360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f630006ce90 msgr2=0x7f630006d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:23.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.710+0000 7f6304e00640 1 --2- 172.21.15.52:0/2578609360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f630006ce90 0x7f630006d320 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f62f8009e50 tx=0x7f62f802bee0 comp rx=0 tx=0).stop 2024-03-12T17:07:23.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.710+0000 7f6304e00640 1 -- 172.21.15.52:0/2578609360 shutdown_connections 2024-03-12T17:07:23.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.710+0000 7f6304e00640 1 --2- 172.21.15.52:0/2578609360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f630006ce90 0x7f630006d320 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:23.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.710+0000 7f6304e00640 1 --2- 172.21.15.52:0/2578609360 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63000add30 0x7f63000ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:23.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.710+0000 7f6304e00640 1 -- 172.21.15.52:0/2578609360 >> 172.21.15.52:0/2578609360 conn(0x7f630006a0b0 msgr2=0x7f630006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:23.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.710+0000 7f6304e00640 1 -- 172.21.15.52:0/2578609360 shutdown_connections 2024-03-12T17:07:23.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.710+0000 7f6304e00640 1 -- 172.21.15.52:0/2578609360 wait complete. 2024-03-12T17:07:23.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.712+0000 7f6304e00640 1 Processor -- start 2024-03-12T17:07:23.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.712+0000 7f6304e00640 1 -- start start 2024-03-12T17:07:23.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.713+0000 7f6304e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f630006ce90 0x7f63001efe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:23.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.713+0000 7f6304e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63000add30 0x7f6300156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:23.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.713+0000 7f6304e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63000ce930 0x7f6300156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:23.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.713+0000 7f6304e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63001f0520 con 0x7f630006ce90 2024-03-12T17:07:23.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.713+0000 7f6304e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63001f0800 con 0x7f63000add30 2024-03-12T17:07:23.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.713+0000 7f6304e00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63001f0ae0 con 0x7f63000ce930 2024-03-12T17:07:23.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.713+0000 7f62fde00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f630006ce90 0x7f63001efe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:23.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.713+0000 7f62fde00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f630006ce90 0x7f63001efe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49264/0 (socket says 172.21.15.52:49264) 2024-03-12T17:07:23.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.713+0000 7f62fde00640 1 -- 172.21.15.52:0/3789641542 learned_addr learned my addr 172.21.15.52:0/3789641542 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:23.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.714+0000 7f62f5400640 1 --2- 172.21.15.52:0/3789641542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63000add30 0x7f6300156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:23.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.714+0000 7f62fde00640 1 -- 172.21.15.52:0/3789641542 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63000ce930 msgr2=0x7f6300156f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:23.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.714+0000 7f62fe800640 1 --2- 172.21.15.52:0/3789641542 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63000ce930 0x7f6300156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:23.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.714+0000 7f62fde00640 1 --2- 172.21.15.52:0/3789641542 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63000ce930 0x7f6300156f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:23.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.714+0000 7f62fde00640 1 -- 172.21.15.52:0/3789641542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63000add30 msgr2=0x7f6300156260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:23.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.714+0000 7f62fde00640 1 --2- 172.21.15.52:0/3789641542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63000add30 0x7f6300156260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:23.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.714+0000 7f62fde00640 1 -- 172.21.15.52:0/3789641542 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f8009b30 con 0x7f630006ce90 2024-03-12T17:07:23.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.714+0000 7f62f5400640 1 --2- 172.21.15.52:0/3789641542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63000add30 0x7f6300156260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:23.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.714+0000 7f62fde00640 1 --2- 172.21.15.52:0/3789641542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f630006ce90 0x7f63001efe90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f62f000b3f0 tx=0x7f62f000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:23.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.714+0000 7f62f6a00640 1 -- 172.21.15.52:0/3789641542 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f0011830 con 0x7f630006ce90 2024-03-12T17:07:23.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.714+0000 7f62f6a00640 1 -- 172.21.15.52:0/3789641542 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f0011990 con 0x7f630006ce90 2024-03-12T17:07:23.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.714+0000 7f62f6a00640 1 -- 172.21.15.52:0/3789641542 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f0011d40 con 0x7f630006ce90 2024-03-12T17:07:23.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.714+0000 7f6304e00640 1 -- 172.21.15.52:0/3789641542 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63000ab360 con 0x7f630006ce90 2024-03-12T17:07:23.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.715+0000 7f6304e00640 1 -- 172.21.15.52:0/3789641542 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63001567a0 con 0x7f630006ce90 2024-03-12T17:07:23.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.716+0000 7f6304e00640 1 -- 172.21.15.52:0/3789641542 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63000ab1c0 con 0x7f630006ce90 2024-03-12T17:07:23.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.717+0000 7f62f6a00640 1 -- 172.21.15.52:0/3789641542 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f62f001a400 con 0x7f630006ce90 2024-03-12T17:07:23.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.717+0000 7f62f6a00640 1 --2- 172.21.15.52:0/3789641542 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f62dc06ac00 0x7f62dc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:23.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.718+0000 7f62f6a00640 1 -- 172.21.15.52:0/3789641542 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f62f00894d0 con 0x7f630006ce90 2024-03-12T17:07:23.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.718+0000 7f62f5400640 1 --2- 172.21.15.52:0/3789641542 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f62dc06ac00 0x7f62dc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:23.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.719+0000 7f62f5400640 1 --2- 172.21.15.52:0/3789641542 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f62dc06ac00 0x7f62dc06d0c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f62e8006fd0 tx=0x7f62e8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:23.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.722+0000 7f62f6a00640 1 -- 172.21.15.52:0/3789641542 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f63000ab1c0 con 0x7f630006ce90 2024-03-12T17:07:23.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.922+0000 7f6304e00640 1 -- 172.21.15.52:0/3789641542 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f63000ab1c0 con 0x7f630006ce90 2024-03-12T17:07:23.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.922+0000 7f62f6a00640 1 -- 172.21.15.52:0/3789641542 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f63000ab1c0 con 0x7f630006ce90 2024-03-12T17:07:23.918 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-12T17:07:23.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.925+0000 7f6304e00640 1 -- 172.21.15.52:0/3789641542 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f62dc06ac00 msgr2=0x7f62dc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:23.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.925+0000 7f6304e00640 1 --2- 172.21.15.52:0/3789641542 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f62dc06ac00 0x7f62dc06d0c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f62e8006fd0 tx=0x7f62e8008040 comp rx=0 tx=0).stop 2024-03-12T17:07:23.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.925+0000 7f6304e00640 1 -- 172.21.15.52:0/3789641542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f630006ce90 msgr2=0x7f63001efe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:23.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.925+0000 7f6304e00640 1 --2- 172.21.15.52:0/3789641542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f630006ce90 0x7f63001efe90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f62f000b3f0 tx=0x7f62f000b8b0 comp rx=0 tx=0).stop 2024-03-12T17:07:23.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.926+0000 7f6304e00640 1 -- 172.21.15.52:0/3789641542 shutdown_connections 2024-03-12T17:07:23.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.926+0000 7f6304e00640 1 --2- 172.21.15.52:0/3789641542 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f62dc06ac00 0x7f62dc06d0c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:23.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.926+0000 7f6304e00640 1 --2- 172.21.15.52:0/3789641542 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63000ce930 0x7f6300156f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:23.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.926+0000 7f6304e00640 1 --2- 172.21.15.52:0/3789641542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63000add30 0x7f6300156260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:23.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.926+0000 7f6304e00640 1 --2- 172.21.15.52:0/3789641542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f630006ce90 0x7f63001efe90 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:23.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.926+0000 7f6304e00640 1 -- 172.21.15.52:0/3789641542 >> 172.21.15.52:0/3789641542 conn(0x7f630006a0b0 msgr2=0x7f630015adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:23.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.926+0000 7f6304e00640 1 -- 172.21.15.52:0/3789641542 shutdown_connections 2024-03-12T17:07:23.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:23.926+0000 7f6304e00640 1 -- 172.21.15.52:0/3789641542 wait complete. 2024-03-12T17:07:23.937 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-12T17:07:24.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.167+0000 7f4a2ca00640 1 Processor -- start 2024-03-12T17:07:24.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.167+0000 7f4a2ca00640 1 -- start start 2024-03-12T17:07:24.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.169+0000 7f4a2ca00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4a280b08c0 0x7f4a280b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:24.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.169+0000 7f4a2ca00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a2806aa30 con 0x7f4a28069c90 2024-03-12T17:07:24.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.169+0000 7f4a2ca00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a280b3f40 con 0x7f4a280ada70 2024-03-12T17:07:24.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.169+0000 7f4a2ca00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a2815cee0 con 0x7f4a280b08c0 2024-03-12T17:07:24.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.170+0000 7f4a1ea00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4a280b08c0 0x7f4a280b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:24.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.170+0000 7f4a27e00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f4a28069c90 0x7f4a2806a0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:57340/0 (socket says 172.21.15.52:57340) 2024-03-12T17:07:24.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.170+0000 7f4a27e00640 1 -- 172.21.15.52:0/1722254946 learned_addr learned my addr 172.21.15.52:0/1722254946 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:24.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.170+0000 7f4a26a00640 1 -- 172.21.15.52:0/1722254946 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3884757552 0 0) 0x7f4a2806aa30 con 0x7f4a28069c90 2024-03-12T17:07:24.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.170+0000 7f4a1ea00640 1 -- 172.21.15.52:0/1722254946 >> v1:172.21.15.102:6789/0 conn(0x7f4a280ada70 legacy=0x7f4a2815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:24.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.170+0000 7f4a1ea00640 1 -- 172.21.15.52:0/1722254946 >> v1:172.21.15.52:6789/0 conn(0x7f4a28069c90 legacy=0x7f4a2806a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:24.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.170+0000 7f4a1ea00640 1 -- 172.21.15.52:0/1722254946 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a280b41c0 con 0x7f4a280b08c0 2024-03-12T17:07:24.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.171+0000 7f4a1ea00640 1 --2- 172.21.15.52:0/1722254946 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a280b08c0 0x7f4a280b23c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4a100097a0 tx=0x7f4a1002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ce602cc6e242e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:24.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.172+0000 7f4a26a00640 1 -- 172.21.15.52:0/1722254946 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a10030610 con 0x7f4a280b08c0 2024-03-12T17:07:24.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.172+0000 7f4a26a00640 1 -- 172.21.15.52:0/1722254946 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a10030770 con 0x7f4a280b08c0 2024-03-12T17:07:24.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.172+0000 7f4a26a00640 1 -- 172.21.15.52:0/1722254946 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a10030b00 con 0x7f4a280b08c0 2024-03-12T17:07:24.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.172+0000 7f4a2ca00640 1 -- 172.21.15.52:0/1722254946 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a280b08c0 msgr2=0x7f4a280b23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:24.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.172+0000 7f4a2ca00640 1 --2- 172.21.15.52:0/1722254946 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a280b08c0 0x7f4a280b23c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4a100097a0 tx=0x7f4a1002b960 comp rx=0 tx=0).stop 2024-03-12T17:07:24.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.172+0000 7f4a2ca00640 1 -- 172.21.15.52:0/1722254946 shutdown_connections 2024-03-12T17:07:24.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.172+0000 7f4a2ca00640 1 --2- 172.21.15.52:0/1722254946 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a280b08c0 0x7f4a280b23c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:24.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.172+0000 7f4a2ca00640 1 -- 172.21.15.52:0/1722254946 >> 172.21.15.52:0/1722254946 conn(0x7f4a280b48e0 msgr2=0x7f4a280b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:24.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.173+0000 7f4a2ca00640 1 -- 172.21.15.52:0/1722254946 shutdown_connections 2024-03-12T17:07:24.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.173+0000 7f4a2ca00640 1 -- 172.21.15.52:0/1722254946 wait complete. 2024-03-12T17:07:24.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.174+0000 7f4a2ca00640 1 Processor -- start 2024-03-12T17:07:24.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.174+0000 7f4a2ca00640 1 -- start start 2024-03-12T17:07:24.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.175+0000 7f4a2ca00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a28069c90 0x7f4a28160240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:24.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.175+0000 7f4a2ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a280ada70 0x7f4a280ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:24.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.175+0000 7f4a2ca00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a280b08c0 0x7f4a2809a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:24.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.175+0000 7f4a2ca00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a28099f60 con 0x7f4a280b08c0 2024-03-12T17:07:24.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.175+0000 7f4a2ca00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a28099630 con 0x7f4a280ada70 2024-03-12T17:07:24.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.175+0000 7f4a2ca00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a28099910 con 0x7f4a28069c90 2024-03-12T17:07:24.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.176+0000 7f4a27e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a280b08c0 0x7f4a2809a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:24.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.176+0000 7f4a27e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a280b08c0 0x7f4a2809a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49278/0 (socket says 172.21.15.52:49278) 2024-03-12T17:07:24.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.176+0000 7f4a27e00640 1 -- 172.21.15.52:0/4000995914 learned_addr learned my addr 172.21.15.52:0/4000995914 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:24.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.176+0000 7f4a27400640 1 --2- 172.21.15.52:0/4000995914 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a28069c90 0x7f4a28160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:24.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.176+0000 7f4a1ea00640 1 --2- 172.21.15.52:0/4000995914 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a280ada70 0x7f4a280ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:24.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.176+0000 7f4a27e00640 1 -- 172.21.15.52:0/4000995914 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a28069c90 msgr2=0x7f4a28160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:24.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.176+0000 7f4a27e00640 1 --2- 172.21.15.52:0/4000995914 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a28069c90 0x7f4a28160240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:24.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.176+0000 7f4a27e00640 1 -- 172.21.15.52:0/4000995914 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a280ada70 msgr2=0x7f4a280ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:24.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.176+0000 7f4a27e00640 1 --2- 172.21.15.52:0/4000995914 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a280ada70 0x7f4a280ade50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:24.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.176+0000 7f4a27e00640 1 -- 172.21.15.52:0/4000995914 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a10009400 con 0x7f4a280b08c0 2024-03-12T17:07:24.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.177+0000 7f4a27e00640 1 --2- 172.21.15.52:0/4000995914 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a280b08c0 0x7f4a2809a3f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4a2000c7e0 tx=0x7f4a2000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:24.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.177+0000 7f4a1fe00640 1 -- 172.21.15.52:0/4000995914 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2000a640 con 0x7f4a280b08c0 2024-03-12T17:07:24.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.177+0000 7f4a1fe00640 1 -- 172.21.15.52:0/4000995914 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a2000a7c0 con 0x7f4a280b08c0 2024-03-12T17:07:24.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.177+0000 7f4a1fe00640 1 -- 172.21.15.52:0/4000995914 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2000ab70 con 0x7f4a280b08c0 2024-03-12T17:07:24.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.177+0000 7f4a2ca00640 1 -- 172.21.15.52:0/4000995914 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a28099bf0 con 0x7f4a280b08c0 2024-03-12T17:07:24.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.177+0000 7f4a2ca00640 1 -- 172.21.15.52:0/4000995914 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a28160960 con 0x7f4a280b08c0 2024-03-12T17:07:24.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.179+0000 7f4a2ca00640 1 -- 172.21.15.52:0/4000995914 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a2809a110 con 0x7f4a280b08c0 2024-03-12T17:07:24.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.180+0000 7f4a1fe00640 1 -- 172.21.15.52:0/4000995914 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4a2000acd0 con 0x7f4a280b08c0 2024-03-12T17:07:24.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.181+0000 7f4a1fe00640 1 --2- 172.21.15.52:0/4000995914 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4a0406ac00 0x7f4a0406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:24.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.181+0000 7f4a1fe00640 1 -- 172.21.15.52:0/4000995914 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4a20088b10 con 0x7f4a280b08c0 2024-03-12T17:07:24.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.182+0000 7f4a27400640 1 --2- 172.21.15.52:0/4000995914 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4a0406ac00 0x7f4a0406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:24.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.183+0000 7f4a27400640 1 --2- 172.21.15.52:0/4000995914 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4a0406ac00 0x7f4a0406d0c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4a2806d770 tx=0x7f4a18009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:24.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.185+0000 7f4a1fe00640 1 -- 172.21.15.52:0/4000995914 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f4a2809a110 con 0x7f4a280b08c0 2024-03-12T17:07:24.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.383+0000 7f4a2ca00640 1 -- 172.21.15.52:0/4000995914 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4a28160960 con 0x7f4a280b08c0 2024-03-12T17:07:24.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.384+0000 7f4a1fe00640 1 -- 172.21.15.52:0/4000995914 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4a28160960 con 0x7f4a280b08c0 2024-03-12T17:07:24.378 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-03-12T17:07:24.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.387+0000 7f4a2ca00640 1 -- 172.21.15.52:0/4000995914 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4a0406ac00 msgr2=0x7f4a0406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:24.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.387+0000 7f4a2ca00640 1 --2- 172.21.15.52:0/4000995914 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4a0406ac00 0x7f4a0406d0c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4a2806d770 tx=0x7f4a18009040 comp rx=0 tx=0).stop 2024-03-12T17:07:24.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.387+0000 7f4a2ca00640 1 -- 172.21.15.52:0/4000995914 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a280b08c0 msgr2=0x7f4a2809a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:24.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.387+0000 7f4a2ca00640 1 --2- 172.21.15.52:0/4000995914 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a280b08c0 0x7f4a2809a3f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4a2000c7e0 tx=0x7f4a2000cca0 comp rx=0 tx=0).stop 2024-03-12T17:07:24.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.387+0000 7f4a2ca00640 1 -- 172.21.15.52:0/4000995914 shutdown_connections 2024-03-12T17:07:24.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.387+0000 7f4a2ca00640 1 --2- 172.21.15.52:0/4000995914 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4a0406ac00 0x7f4a0406d0c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:24.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.387+0000 7f4a2ca00640 1 --2- 172.21.15.52:0/4000995914 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a280b08c0 0x7f4a2809a3f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:24.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.387+0000 7f4a2ca00640 1 --2- 172.21.15.52:0/4000995914 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a280ada70 0x7f4a280ade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:24.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.387+0000 7f4a2ca00640 1 --2- 172.21.15.52:0/4000995914 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a28069c90 0x7f4a28160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:24.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.387+0000 7f4a2ca00640 1 -- 172.21.15.52:0/4000995914 >> 172.21.15.52:0/4000995914 conn(0x7f4a280b48e0 msgr2=0x7f4a28157d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:24.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.387+0000 7f4a2ca00640 1 -- 172.21.15.52:0/4000995914 shutdown_connections 2024-03-12T17:07:24.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.388+0000 7f4a2ca00640 1 -- 172.21.15.52:0/4000995914 wait complete. 2024-03-12T17:07:24.399 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-12T17:07:24.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.614+0000 7fcbb6200640 1 Processor -- start 2024-03-12T17:07:24.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.614+0000 7fcbb6200640 1 -- start start 2024-03-12T17:07:24.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.615+0000 7fcbb6200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcbb00cf040 0x7fcbb0164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:24.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.615+0000 7fcbb6200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fcbb00ca7b0 0x7fcbb00b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:24.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.616+0000 7fcbb6200640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbb0165830 con 0x7fcbb0160d20 2024-03-12T17:07:24.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.616+0000 7fcbb6200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbb0164f90 con 0x7fcbb00ca7b0 2024-03-12T17:07:24.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.616+0000 7fcbb6200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb0165270 con 0x7fcbb00cf040 2024-03-12T17:07:24.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.616+0000 7fcbaf400640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fcbb0160d20 0x7fcbb00ce930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:57342/0 (socket says 172.21.15.52:57342) 2024-03-12T17:07:24.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.616+0000 7fcbaf400640 1 -- 172.21.15.52:0/1267219507 learned_addr learned my addr 172.21.15.52:0/1267219507 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:24.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.616+0000 7fcbafe00640 1 --2- 172.21.15.52:0/1267219507 >> v2:172.21.15.52:3300/0 conn(0x7fcbb00ca7b0 0x7fcbb00b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:24.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.616+0000 7fcbaea00640 1 --2- 172.21.15.52:0/1267219507 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcbb00cf040 0x7fcbb0164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:24.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.616+0000 7fcbae000640 1 -- 172.21.15.52:0/1267219507 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 701723614 0 0) 0x7fcbb0165830 con 0x7fcbb0160d20 2024-03-12T17:07:24.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.616+0000 7fcbae000640 1 -- 172.21.15.52:0/1267219507 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb94003520 con 0x7fcbb0160d20 2024-03-12T17:07:24.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.617+0000 7fcbafe00640 1 -- 172.21.15.52:0/1267219507 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcbb00cf040 msgr2=0x7fcbb0164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:24.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.617+0000 7fcbafe00640 1 --2- 172.21.15.52:0/1267219507 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcbb00cf040 0x7fcbb0164920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:24.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.617+0000 7fcbafe00640 1 -- 172.21.15.52:0/1267219507 >> v1:172.21.15.52:6789/0 conn(0x7fcbb0160d20 legacy=0x7fcbb00ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:24.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.617+0000 7fcbafe00640 1 -- 172.21.15.52:0/1267219507 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb0165550 con 0x7fcbb00ca7b0 2024-03-12T17:07:24.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.618+0000 7fcbafe00640 1 --2- 172.21.15.52:0/1267219507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbb00ca7b0 0x7fcbb00b1b80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009d90 tx=0x7fcba802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6fac0a295118ef1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:24.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.618+0000 7fcbae000640 1 -- 172.21.15.52:0/1267219507 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba802cbc0 con 0x7fcbb00ca7b0 2024-03-12T17:07:24.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.618+0000 7fcbae000640 1 -- 172.21.15.52:0/1267219507 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba802cd20 con 0x7fcbb00ca7b0 2024-03-12T17:07:24.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.618+0000 7fcbae000640 1 -- 172.21.15.52:0/1267219507 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba8034660 con 0x7fcbb00ca7b0 2024-03-12T17:07:24.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.619+0000 7fcbb6200640 1 -- 172.21.15.52:0/1267219507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbb00ca7b0 msgr2=0x7fcbb00b1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:24.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.619+0000 7fcbb6200640 1 --2- 172.21.15.52:0/1267219507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbb00ca7b0 0x7fcbb00b1b80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009d90 tx=0x7fcba802be10 comp rx=0 tx=0).stop 2024-03-12T17:07:24.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.619+0000 7fcbb6200640 1 -- 172.21.15.52:0/1267219507 shutdown_connections 2024-03-12T17:07:24.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.619+0000 7fcbb6200640 1 --2- 172.21.15.52:0/1267219507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbb00ca7b0 0x7fcbb00b1b80 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:24.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.619+0000 7fcbb6200640 1 --2- 172.21.15.52:0/1267219507 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcbb00cf040 0x7fcbb0164920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:24.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.619+0000 7fcbb6200640 1 -- 172.21.15.52:0/1267219507 >> 172.21.15.52:0/1267219507 conn(0x7fcbb006a0b0 msgr2=0x7fcbb006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:24.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.620+0000 7fcbb6200640 1 -- 172.21.15.52:0/1267219507 shutdown_connections 2024-03-12T17:07:24.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.620+0000 7fcbb6200640 1 -- 172.21.15.52:0/1267219507 wait complete. 2024-03-12T17:07:24.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.621+0000 7fcbb6200640 1 Processor -- start 2024-03-12T17:07:24.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.621+0000 7fcbb6200640 1 -- start start 2024-03-12T17:07:24.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.622+0000 7fcbb6200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbb00ca7b0 0x7fcbb0148750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:24.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.622+0000 7fcbb6200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcbb00cf040 0x7fcbb00cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:24.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.622+0000 7fcbb6200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbb0160d20 0x7fcbb014aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:24.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.622+0000 7fcbb6200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb01617e0 con 0x7fcbb0160d20 2024-03-12T17:07:24.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.622+0000 7fcbb6200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb0161ac0 con 0x7fcbb00ca7b0 2024-03-12T17:07:24.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.622+0000 7fcbb6200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb0161da0 con 0x7fcbb00cf040 2024-03-12T17:07:24.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.623+0000 7fcbafe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbb0160d20 0x7fcbb014aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:24.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.623+0000 7fcbafe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbb0160d20 0x7fcbb014aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49298/0 (socket says 172.21.15.52:49298) 2024-03-12T17:07:24.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.623+0000 7fcbafe00640 1 -- 172.21.15.52:0/2840074927 learned_addr learned my addr 172.21.15.52:0/2840074927 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:24.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.623+0000 7fcbaf400640 1 --2- 172.21.15.52:0/2840074927 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbb00ca7b0 0x7fcbb0148750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:24.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.623+0000 7fcbaea00640 1 --2- 172.21.15.52:0/2840074927 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcbb00cf040 0x7fcbb00cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:24.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.623+0000 7fcbafe00640 1 -- 172.21.15.52:0/2840074927 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcbb00cf040 msgr2=0x7fcbb00cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:24.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.623+0000 7fcbafe00640 1 --2- 172.21.15.52:0/2840074927 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcbb00cf040 0x7fcbb00cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:24.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.623+0000 7fcbafe00640 1 -- 172.21.15.52:0/2840074927 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbb00ca7b0 msgr2=0x7fcbb0148750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:24.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.623+0000 7fcbafe00640 1 --2- 172.21.15.52:0/2840074927 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbb00ca7b0 0x7fcbb0148750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:24.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.623+0000 7fcbafe00640 1 -- 172.21.15.52:0/2840074927 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba8009a70 con 0x7fcbb0160d20 2024-03-12T17:07:24.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.624+0000 7fcbafe00640 1 --2- 172.21.15.52:0/2840074927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbb0160d20 0x7fcbb014aa40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009d90 tx=0x7fcba8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:24.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.624+0000 7fcba7400640 1 -- 172.21.15.52:0/2840074927 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba8040650 con 0x7fcbb0160d20 2024-03-12T17:07:24.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.624+0000 7fcba7400640 1 -- 172.21.15.52:0/2840074927 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba802ce90 con 0x7fcbb0160d20 2024-03-12T17:07:24.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.624+0000 7fcba7400640 1 -- 172.21.15.52:0/2840074927 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba80409f0 con 0x7fcbb0160d20 2024-03-12T17:07:24.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.624+0000 7fcbb6200640 1 -- 172.21.15.52:0/2840074927 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb0162080 con 0x7fcbb0160d20 2024-03-12T17:07:24.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.624+0000 7fcbb6200640 1 -- 172.21.15.52:0/2840074927 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb00cda20 con 0x7fcbb0160d20 2024-03-12T17:07:24.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.625+0000 7fcbb6200640 1 -- 172.21.15.52:0/2840074927 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbb01625a0 con 0x7fcbb0160d20 2024-03-12T17:07:24.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.626+0000 7fcba7400640 1 -- 172.21.15.52:0/2840074927 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fcba8040b50 con 0x7fcbb0160d20 2024-03-12T17:07:24.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.627+0000 7fcba7400640 1 --2- 172.21.15.52:0/2840074927 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fcb9006ac00 0x7fcb9006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:24.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.627+0000 7fcba7400640 1 -- 172.21.15.52:0/2840074927 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fcba80ac2a0 con 0x7fcbb0160d20 2024-03-12T17:07:24.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.628+0000 7fcbaf400640 1 --2- 172.21.15.52:0/2840074927 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fcb9006ac00 0x7fcb9006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:24.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.628+0000 7fcbaf400640 1 --2- 172.21.15.52:0/2840074927 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fcb9006ac00 0x7fcb9006d0c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcba00011e0 tx=0x7fcba0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:24.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.631+0000 7fcba7400640 1 -- 172.21.15.52:0/2840074927 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fcbb01625a0 con 0x7fcbb0160d20 2024-03-12T17:07:24.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.818+0000 7fcbb6200640 1 -- 172.21.15.52:0/2840074927 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fcbb01625a0 con 0x7fcbb0160d20 2024-03-12T17:07:24.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.819+0000 7fcba7400640 1 -- 172.21.15.52:0/2840074927 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fcbb01625a0 con 0x7fcbb0160d20 2024-03-12T17:07:24.813 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-12T17:07:24.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.821+0000 7fcbb6200640 1 -- 172.21.15.52:0/2840074927 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fcb9006ac00 msgr2=0x7fcb9006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:24.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.821+0000 7fcbb6200640 1 --2- 172.21.15.52:0/2840074927 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fcb9006ac00 0x7fcb9006d0c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcba00011e0 tx=0x7fcba0009040 comp rx=0 tx=0).stop 2024-03-12T17:07:24.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.821+0000 7fcbb6200640 1 -- 172.21.15.52:0/2840074927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbb0160d20 msgr2=0x7fcbb014aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:24.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.821+0000 7fcbb6200640 1 --2- 172.21.15.52:0/2840074927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbb0160d20 0x7fcbb014aa40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009d90 tx=0x7fcba8004a40 comp rx=0 tx=0).stop 2024-03-12T17:07:24.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.822+0000 7fcbb6200640 1 -- 172.21.15.52:0/2840074927 shutdown_connections 2024-03-12T17:07:24.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.822+0000 7fcbb6200640 1 --2- 172.21.15.52:0/2840074927 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fcb9006ac00 0x7fcb9006d0c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:24.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.822+0000 7fcbb6200640 1 --2- 172.21.15.52:0/2840074927 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcbb00cf040 0x7fcbb00cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:24.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.822+0000 7fcbb6200640 1 --2- 172.21.15.52:0/2840074927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbb0160d20 0x7fcbb014aa40 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:24.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.822+0000 7fcbb6200640 1 --2- 172.21.15.52:0/2840074927 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbb00ca7b0 0x7fcbb0148750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:24.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.822+0000 7fcbb6200640 1 -- 172.21.15.52:0/2840074927 >> 172.21.15.52:0/2840074927 conn(0x7fcbb006a0b0 msgr2=0x7fcbb006d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:24.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.822+0000 7fcbb6200640 1 -- 172.21.15.52:0/2840074927 shutdown_connections 2024-03-12T17:07:24.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:24.822+0000 7fcbb6200640 1 -- 172.21.15.52:0/2840074927 wait complete. 2024-03-12T17:07:24.834 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:07:25.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.055+0000 7f467e400640 1 Processor -- start 2024-03-12T17:07:25.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.056+0000 7f467e400640 1 -- start start 2024-03-12T17:07:25.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.057+0000 7f467e400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f46780cf5c0 0x7f46780b8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:25.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.057+0000 7f467e400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f46780d0c20 0x7f46780d1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:25.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.057+0000 7f467e400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f467815ce90 con 0x7f467815cab0 2024-03-12T17:07:25.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.057+0000 7f467e400640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46780cddf0 con 0x7f46780cf5c0 2024-03-12T17:07:25.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.057+0000 7f467e400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46780cd570 con 0x7f46780d0c20 2024-03-12T17:07:25.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.058+0000 7f4677e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f46780cf5c0 0x7f46780b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:25.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.058+0000 7f4677e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f46780cf5c0 0x7f46780b8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50918/0 (socket says 172.21.15.52:50918) 2024-03-12T17:07:25.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.058+0000 7f4677e00640 1 -- 172.21.15.52:0/2625729420 learned_addr learned my addr 172.21.15.52:0/2625729420 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:25.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.058+0000 7f467da00640 1 --2- 172.21.15.52:0/2625729420 >> v2:172.21.15.102:3300/0 conn(0x7f46780d0c20 0x7f46780d1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:25.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.058+0000 7f4677e00640 1 -- 172.21.15.52:0/2625729420 >> v2:172.21.15.102:3300/0 conn(0x7f46780d0c20 msgr2=0x7f46780d1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:25.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.058+0000 7f4677e00640 1 --2- 172.21.15.52:0/2625729420 >> v2:172.21.15.102:3300/0 conn(0x7f46780d0c20 0x7f46780d1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:25.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.058+0000 7f4677e00640 1 -- 172.21.15.52:0/2625729420 >> v1:172.21.15.102:6789/0 conn(0x7f467815cab0 legacy=0x7f46780ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:25.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.058+0000 7f4677e00640 1 -- 172.21.15.52:0/2625729420 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4678161370 con 0x7f46780cf5c0 2024-03-12T17:07:25.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.059+0000 7f4677e00640 1 --2- 172.21.15.52:0/2625729420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46780cf5c0 0x7f46780b8050 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f46780c9700 tx=0x7f466402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe3d05e237e08dc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:25.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.060+0000 7f4677400640 1 -- 172.21.15.52:0/2625729420 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4664031870 con 0x7f46780cf5c0 2024-03-12T17:07:25.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.060+0000 7f4677400640 1 -- 172.21.15.52:0/2625729420 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46640319d0 con 0x7f46780cf5c0 2024-03-12T17:07:25.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.060+0000 7f4677400640 1 -- 172.21.15.52:0/2625729420 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4664031d60 con 0x7f46780cf5c0 2024-03-12T17:07:25.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.060+0000 7f467e400640 1 -- 172.21.15.52:0/2625729420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46780cf5c0 msgr2=0x7f46780b8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:25.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.060+0000 7f467e400640 1 --2- 172.21.15.52:0/2625729420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46780cf5c0 0x7f46780b8050 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f46780c9700 tx=0x7f466402b910 comp rx=0 tx=0).stop 2024-03-12T17:07:25.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.061+0000 7f467e400640 1 -- 172.21.15.52:0/2625729420 shutdown_connections 2024-03-12T17:07:25.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.061+0000 7f467e400640 1 --2- 172.21.15.52:0/2625729420 >> v2:172.21.15.102:3300/0 conn(0x7f46780d0c20 0x7f46780d1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:25.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.061+0000 7f467e400640 1 --2- 172.21.15.52:0/2625729420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46780cf5c0 0x7f46780b8050 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:25.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.061+0000 7f467e400640 1 -- 172.21.15.52:0/2625729420 >> 172.21.15.52:0/2625729420 conn(0x7f467806ae90 msgr2=0x7f467806b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:25.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.061+0000 7f467e400640 1 -- 172.21.15.52:0/2625729420 shutdown_connections 2024-03-12T17:07:25.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.061+0000 7f467e400640 1 -- 172.21.15.52:0/2625729420 wait complete. 2024-03-12T17:07:25.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.063+0000 7f467e400640 1 Processor -- start 2024-03-12T17:07:25.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.063+0000 7f467e400640 1 -- start start 2024-03-12T17:07:25.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.064+0000 7f467e400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46780cf5c0 0x7f467815eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:25.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.064+0000 7f467e400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46780d0c20 0x7f46780cab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:25.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.065+0000 7f467e400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f467815cab0 0x7f46780cb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:25.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.065+0000 7f467e400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46780cc3b0 con 0x7f46780cf5c0 2024-03-12T17:07:25.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.065+0000 7f467e400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46780c9b20 con 0x7f46780d0c20 2024-03-12T17:07:25.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.065+0000 7f467e400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46780cb730 con 0x7f467815cab0 2024-03-12T17:07:25.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.065+0000 7f467d000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46780cf5c0 0x7f467815eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:25.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.065+0000 7f467d000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46780cf5c0 0x7f467815eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50922/0 (socket says 172.21.15.52:50922) 2024-03-12T17:07:25.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.065+0000 7f467d000640 1 -- 172.21.15.52:0/802421087 learned_addr learned my addr 172.21.15.52:0/802421087 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:25.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.065+0000 7f4677e00640 1 --2- 172.21.15.52:0/802421087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46780d0c20 0x7f46780cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:25.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.065+0000 7f467da00640 1 --2- 172.21.15.52:0/802421087 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f467815cab0 0x7f46780cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:25.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.065+0000 7f467d000640 1 -- 172.21.15.52:0/802421087 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f467815cab0 msgr2=0x7f46780cb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:25.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.065+0000 7f467d000640 1 --2- 172.21.15.52:0/802421087 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f467815cab0 0x7f46780cb060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:25.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.065+0000 7f467d000640 1 -- 172.21.15.52:0/802421087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46780d0c20 msgr2=0x7f46780cab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:25.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.065+0000 7f467d000640 1 --2- 172.21.15.52:0/802421087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46780d0c20 0x7f46780cab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:25.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.065+0000 7f467d000640 1 -- 172.21.15.52:0/802421087 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46640094d0 con 0x7f46780cf5c0 2024-03-12T17:07:25.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.065+0000 7f4677e00640 1 --2- 172.21.15.52:0/802421087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46780d0c20 0x7f46780cab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:25.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.066+0000 7f467d000640 1 --2- 172.21.15.52:0/802421087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46780cf5c0 0x7f467815eca0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f466800b3f0 tx=0x7f466800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:25.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.066+0000 7f4674c00640 1 -- 172.21.15.52:0/802421087 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4668011830 con 0x7f46780cf5c0 2024-03-12T17:07:25.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.066+0000 7f4674c00640 1 -- 172.21.15.52:0/802421087 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4668011990 con 0x7f46780cf5c0 2024-03-12T17:07:25.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.066+0000 7f4674c00640 1 -- 172.21.15.52:0/802421087 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4668011d40 con 0x7f46780cf5c0 2024-03-12T17:07:25.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.066+0000 7f467e400640 1 -- 172.21.15.52:0/802421087 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46780cba10 con 0x7f46780cf5c0 2024-03-12T17:07:25.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.066+0000 7f467e400640 1 -- 172.21.15.52:0/802421087 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46780cbfd0 con 0x7f46780cf5c0 2024-03-12T17:07:25.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.067+0000 7f467e400640 1 -- 172.21.15.52:0/802421087 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f467804e030 con 0x7f46780cf5c0 2024-03-12T17:07:25.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.068+0000 7f4674c00640 1 -- 172.21.15.52:0/802421087 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f466801a400 con 0x7f46780cf5c0 2024-03-12T17:07:25.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.070+0000 7f4674c00640 1 --2- 172.21.15.52:0/802421087 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f465806ac00 0x7f465806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:25.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.070+0000 7f4674c00640 1 -- 172.21.15.52:0/802421087 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4668089520 con 0x7f46780cf5c0 2024-03-12T17:07:25.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.070+0000 7f4677e00640 1 --2- 172.21.15.52:0/802421087 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f465806ac00 0x7f465806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:25.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.071+0000 7f4677e00640 1 --2- 172.21.15.52:0/802421087 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f465806ac00 0x7f465806d0c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4664036000 tx=0x7f466403c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:25.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.074+0000 7f4674c00640 1 -- 172.21.15.52:0/802421087 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f46680543d0 con 0x7f46780cf5c0 2024-03-12T17:07:25.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.266+0000 7f467e400640 1 -- 172.21.15.52:0/802421087 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46780d02d0 con 0x7f46780cf5c0 2024-03-12T17:07:25.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.267+0000 7f4674c00640 1 -- 172.21.15.52:0/802421087 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18207 (secure 0 0 0) 0x7f46680579f0 con 0x7f46780cf5c0 2024-03-12T17:07:25.262 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:07:25.262 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":27,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:07:12.969614+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T17:06:19.218543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T17:06:23.525474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T17:06:24.911574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-12T17:06:28.017172+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.733031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.775178+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.591522+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.820838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.829252+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.582481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.18","mappings":[{"from":3,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:07:25.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.270+0000 7f467e400640 1 -- 172.21.15.52:0/802421087 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f465806ac00 msgr2=0x7f465806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:25.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.270+0000 7f467e400640 1 --2- 172.21.15.52:0/802421087 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f465806ac00 0x7f465806d0c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4664036000 tx=0x7f466403c040 comp rx=0 tx=0).stop 2024-03-12T17:07:25.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.270+0000 7f467e400640 1 -- 172.21.15.52:0/802421087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46780cf5c0 msgr2=0x7f467815eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:25.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.270+0000 7f467e400640 1 --2- 172.21.15.52:0/802421087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46780cf5c0 0x7f467815eca0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f466800b3f0 tx=0x7f466800b8b0 comp rx=0 tx=0).stop 2024-03-12T17:07:25.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.270+0000 7f467e400640 1 -- 172.21.15.52:0/802421087 shutdown_connections 2024-03-12T17:07:25.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.270+0000 7f467e400640 1 --2- 172.21.15.52:0/802421087 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f465806ac00 0x7f465806d0c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:25.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.270+0000 7f467e400640 1 --2- 172.21.15.52:0/802421087 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f467815cab0 0x7f46780cb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:25.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.270+0000 7f467e400640 1 --2- 172.21.15.52:0/802421087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46780d0c20 0x7f46780cab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:25.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.270+0000 7f467e400640 1 --2- 172.21.15.52:0/802421087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46780cf5c0 0x7f467815eca0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:25.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.270+0000 7f467e400640 1 -- 172.21.15.52:0/802421087 >> 172.21.15.52:0/802421087 conn(0x7f467806ae90 msgr2=0x7f46780b7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:25.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.270+0000 7f467e400640 1 -- 172.21.15.52:0/802421087 shutdown_connections 2024-03-12T17:07:25.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.270+0000 7f467e400640 1 -- 172.21.15.52:0/802421087 wait complete. 2024-03-12T17:07:25.282 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T17:06:19.218543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-12T17:06:23.525474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-12T17:06:24.911574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-12T17:06:28.017172+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-12T17:07:25.283 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T17:07:25.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.505+0000 7fe8a2e00640 1 Processor -- start 2024-03-12T17:07:25.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.505+0000 7fe8a2e00640 1 -- start start 2024-03-12T17:07:25.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.507+0000 7fe8a2e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe89c0cacf0 0x7fe89c0cb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:25.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.507+0000 7fe8a2e00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe89c15e320 con 0x7fe89c0b22b0 2024-03-12T17:07:25.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.507+0000 7fe8a2e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe89c0cda20 con 0x7fe89c15a850 2024-03-12T17:07:25.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.507+0000 7fe8a2e00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c0cb6c0 con 0x7fe89c0cacf0 2024-03-12T17:07:25.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.507+0000 7fe8a1000640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fe89c0b22b0 0x7fe89c15eae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:52300/0 (socket says 172.21.15.52:52300) 2024-03-12T17:07:25.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.507+0000 7fe8a1000640 1 -- 172.21.15.52:0/4013962604 learned_addr learned my addr 172.21.15.52:0/4013962604 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:25.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.507+0000 7fe8a2400640 1 --2- 172.21.15.52:0/4013962604 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe89c0cacf0 0x7fe89c0cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:25.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.508+0000 7fe89be00640 1 -- 172.21.15.52:0/4013962604 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 87895097 0 0) 0x7fe89c15e320 con 0x7fe89c0b22b0 2024-03-12T17:07:25.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.508+0000 7fe89be00640 1 -- 172.21.15.52:0/4013962604 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe884003520 con 0x7fe89c0b22b0 2024-03-12T17:07:25.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.508+0000 7fe89be00640 1 -- 172.21.15.52:0/4013962604 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 231681982 0 0) 0x7fe884003520 con 0x7fe89c0b22b0 2024-03-12T17:07:25.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.508+0000 7fe89be00640 1 -- 172.21.15.52:0/4013962604 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe89c0cacf0 msgr2=0x7fe89c0cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:25.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.508+0000 7fe89be00640 1 --2- 172.21.15.52:0/4013962604 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe89c0cacf0 0x7fe89c0cb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:25.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.508+0000 7fe89be00640 1 -- 172.21.15.52:0/4013962604 >> v1:172.21.15.102:6789/0 conn(0x7fe89c15a850 legacy=0x7fe89c0cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:25.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.508+0000 7fe89be00640 1 -- 172.21.15.52:0/4013962604 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe89c0cdd00 con 0x7fe89c0b22b0 2024-03-12T17:07:25.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.508+0000 7fe89be00640 1 -- 172.21.15.52:0/4013962604 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7fe888001670 con 0x7fe89c0b22b0 2024-03-12T17:07:25.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.508+0000 7fe89be00640 1 -- 172.21.15.52:0/4013962604 >> v1:172.21.15.52:6789/0 conn(0x7fe89c0b22b0 legacy=0x7fe89c15eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:25.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.508+0000 7fe89be00640 1 --2- 172.21.15.52:0/4013962604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8840039a0 0x7fe884003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:25.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.508+0000 7fe89be00640 1 --2- 172.21.15.52:0/4013962604 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe89c0cacf0 0x7fe8840047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:25.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.508+0000 7fe89be00640 1 --2- 172.21.15.52:0/4013962604 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe884004d00 0x7fe884005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:25.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.508+0000 7fe89be00640 1 -- 172.21.15.52:0/4013962604 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c0cb6c0 con 0x7fe8840039a0 2024-03-12T17:07:25.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.508+0000 7fe89be00640 1 -- 172.21.15.52:0/4013962604 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88400ba60 con 0x7fe884004d00 2024-03-12T17:07:25.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.508+0000 7fe89be00640 1 -- 172.21.15.52:0/4013962604 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe88400bc20 con 0x7fe89c0cacf0 2024-03-12T17:07:25.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.509+0000 7fe8a1a00640 1 --2- 172.21.15.52:0/4013962604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8840039a0 0x7fe884003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:25.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.509+0000 7fe8a2400640 1 --2- 172.21.15.52:0/4013962604 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe884004d00 0x7fe884005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:25.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.509+0000 7fe8a1000640 1 --2- 172.21.15.52:0/4013962604 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe89c0cacf0 0x7fe8840047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:25.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.509+0000 7fe8a1a00640 1 -- 172.21.15.52:0/4013962604 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe89c0cacf0 msgr2=0x7fe8840047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:25.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.509+0000 7fe8a1a00640 1 --2- 172.21.15.52:0/4013962604 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe89c0cacf0 0x7fe8840047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:25.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.509+0000 7fe8a1a00640 1 -- 172.21.15.52:0/4013962604 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe884004d00 msgr2=0x7fe884005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:25.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.509+0000 7fe8a1a00640 1 --2- 172.21.15.52:0/4013962604 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe884004d00 0x7fe884005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:25.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.509+0000 7fe8a1a00640 1 -- 172.21.15.52:0/4013962604 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe884003520 con 0x7fe8840039a0 2024-03-12T17:07:25.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.509+0000 7fe8a1000640 1 --2- 172.21.15.52:0/4013962604 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe89c0cacf0 0x7fe8840047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:07:25.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.511+0000 7fe8a1a00640 1 --2- 172.21.15.52:0/4013962604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8840039a0 0x7fe884003dc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe890001720 tx=0x7fe8840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa18ed0d0cbb14b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:25.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.511+0000 7fe89be00640 1 -- 172.21.15.52:0/4013962604 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8900052d0 con 0x7fe8840039a0 2024-03-12T17:07:25.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.511+0000 7fe89be00640 1 -- 172.21.15.52:0/4013962604 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89002e050 con 0x7fe8840039a0 2024-03-12T17:07:25.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.511+0000 7fe89be00640 1 -- 172.21.15.52:0/4013962604 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8900035f0 con 0x7fe8840039a0 2024-03-12T17:07:25.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.511+0000 7fe8a2e00640 1 -- 172.21.15.52:0/4013962604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8840039a0 msgr2=0x7fe884003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:25.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.511+0000 7fe8a2e00640 1 --2- 172.21.15.52:0/4013962604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8840039a0 0x7fe884003dc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe890001720 tx=0x7fe8840057f0 comp rx=0 tx=0).stop 2024-03-12T17:07:25.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.511+0000 7fe8a2400640 1 -- 172.21.15.52:0/4013962604 reap_dead start 2024-03-12T17:07:25.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.512+0000 7fe8a2e00640 1 -- 172.21.15.52:0/4013962604 shutdown_connections 2024-03-12T17:07:25.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.512+0000 7fe8a2e00640 1 -- 172.21.15.52:0/4013962604 >> 172.21.15.52:0/4013962604 conn(0x7fe89c0b01b0 msgr2=0x7fe89c0b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:25.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.512+0000 7fe8a2e00640 1 -- 172.21.15.52:0/4013962604 shutdown_connections 2024-03-12T17:07:25.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.512+0000 7fe8a2e00640 1 -- 172.21.15.52:0/4013962604 wait complete. 2024-03-12T17:07:25.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.514+0000 7fe8a2e00640 1 Processor -- start 2024-03-12T17:07:25.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.514+0000 7fe8a2e00640 1 -- start start 2024-03-12T17:07:25.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.515+0000 7fe8a2e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8840039a0 0x7fe89c0abc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:25.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.515+0000 7fe8a2e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe89c099900 0x7fe89c0b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:25.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.515+0000 7fe8a2e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe89c099d60 0x7fe89c149e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:25.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.515+0000 7fe8a2e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c14aab0 con 0x7fe8840039a0 2024-03-12T17:07:25.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.515+0000 7fe8a2e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c14b8a0 con 0x7fe89c099900 2024-03-12T17:07:25.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.515+0000 7fe8a2e00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c14ac90 con 0x7fe89c099d60 2024-03-12T17:07:25.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.515+0000 7fe8a1a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8840039a0 0x7fe89c0abc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:25.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.515+0000 7fe8a1a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8840039a0 0x7fe89c0abc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50940/0 (socket says 172.21.15.52:50940) 2024-03-12T17:07:25.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.515+0000 7fe8a1a00640 1 -- 172.21.15.52:0/1351609011 learned_addr learned my addr 172.21.15.52:0/1351609011 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:25.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.515+0000 7fe8a1a00640 1 -- 172.21.15.52:0/1351609011 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe89c099d60 msgr2=0x7fe89c149e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:25.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.515+0000 7fe8a1a00640 1 --2- 172.21.15.52:0/1351609011 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe89c099d60 0x7fe89c149e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:25.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.515+0000 7fe8a1a00640 1 -- 172.21.15.52:0/1351609011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe89c099900 msgr2=0x7fe89c0b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:25.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.515+0000 7fe8a1a00640 1 --2- 172.21.15.52:0/1351609011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe89c099900 0x7fe89c0b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:25.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.515+0000 7fe8a1a00640 1 -- 172.21.15.52:0/1351609011 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe89c14af70 con 0x7fe8840039a0 2024-03-12T17:07:25.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.516+0000 7fe8a1a00640 1 --2- 172.21.15.52:0/1351609011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8840039a0 0x7fe89c0abc50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe890008fd0 tx=0x7fe890001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:25.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.516+0000 7fe899600640 1 -- 172.21.15.52:0/1351609011 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe890007a30 con 0x7fe8840039a0 2024-03-12T17:07:25.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.516+0000 7fe899600640 1 -- 172.21.15.52:0/1351609011 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe890007640 con 0x7fe8840039a0 2024-03-12T17:07:25.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.516+0000 7fe899600640 1 -- 172.21.15.52:0/1351609011 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe890006be0 con 0x7fe8840039a0 2024-03-12T17:07:25.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.516+0000 7fe8a2e00640 1 -- 172.21.15.52:0/1351609011 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe89c14b250 con 0x7fe8840039a0 2024-03-12T17:07:25.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.516+0000 7fe8a2e00640 1 -- 172.21.15.52:0/1351609011 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe89c158d20 con 0x7fe8840039a0 2024-03-12T17:07:25.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.518+0000 7fe8a2e00640 1 -- 172.21.15.52:0/1351609011 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe89c04e030 con 0x7fe8840039a0 2024-03-12T17:07:25.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.518+0000 7fe899600640 1 -- 172.21.15.52:0/1351609011 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe890006d40 con 0x7fe8840039a0 2024-03-12T17:07:25.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.519+0000 7fe899600640 1 --2- 172.21.15.52:0/1351609011 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe87c06ac00 0x7fe87c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:25.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.519+0000 7fe899600640 1 -- 172.21.15.52:0/1351609011 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe8900ab900 con 0x7fe8840039a0 2024-03-12T17:07:25.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.520+0000 7fe8a1000640 1 --2- 172.21.15.52:0/1351609011 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe87c06ac00 0x7fe87c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:25.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.521+0000 7fe8a1000640 1 --2- 172.21.15.52:0/1351609011 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe87c06ac00 0x7fe87c06d0c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe888000ff0 tx=0x7fe8880011c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:25.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.524+0000 7fe899600640 1 -- 172.21.15.52:0/1351609011 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fe890076710 con 0x7fe8840039a0 2024-03-12T17:07:25.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.723+0000 7fe8a2e00640 1 -- 172.21.15.52:0/1351609011 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe89c0ccaa0 con 0x7fe8840039a0 2024-03-12T17:07:25.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.725+0000 7fe899600640 1 -- 172.21.15.52:0/1351609011 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe89004b0a0 con 0x7fe8840039a0 2024-03-12T17:07:25.719 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-03-12T17:07:25.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.727+0000 7fe8a2e00640 1 -- 172.21.15.52:0/1351609011 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe87c06ac00 msgr2=0x7fe87c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:25.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.727+0000 7fe8a2e00640 1 --2- 172.21.15.52:0/1351609011 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe87c06ac00 0x7fe87c06d0c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe888000ff0 tx=0x7fe8880011c0 comp rx=0 tx=0).stop 2024-03-12T17:07:25.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.727+0000 7fe8a2e00640 1 -- 172.21.15.52:0/1351609011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8840039a0 msgr2=0x7fe89c0abc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:25.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.727+0000 7fe8a2e00640 1 --2- 172.21.15.52:0/1351609011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8840039a0 0x7fe89c0abc50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe890008fd0 tx=0x7fe890001120 comp rx=0 tx=0).stop 2024-03-12T17:07:25.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.727+0000 7fe8a2e00640 1 -- 172.21.15.52:0/1351609011 shutdown_connections 2024-03-12T17:07:25.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.727+0000 7fe8a2e00640 1 --2- 172.21.15.52:0/1351609011 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe87c06ac00 0x7fe87c06d0c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:25.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.727+0000 7fe8a2e00640 1 --2- 172.21.15.52:0/1351609011 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe89c099d60 0x7fe89c149e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:25.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.727+0000 7fe8a2e00640 1 --2- 172.21.15.52:0/1351609011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe89c099900 0x7fe89c0b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:25.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.727+0000 7fe8a2e00640 1 --2- 172.21.15.52:0/1351609011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8840039a0 0x7fe89c0abc50 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:25.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.727+0000 7fe8a2e00640 1 -- 172.21.15.52:0/1351609011 >> 172.21.15.52:0/1351609011 conn(0x7fe89c0b01b0 msgr2=0x7fe89c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:25.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.727+0000 7fe8a2e00640 1 -- 172.21.15.52:0/1351609011 shutdown_connections 2024-03-12T17:07:25.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.727+0000 7fe8a2e00640 1 -- 172.21.15.52:0/1351609011 wait complete. 2024-03-12T17:07:25.738 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-12T17:07:25.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.956+0000 7f07aca00640 1 Processor -- start 2024-03-12T17:07:25.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.956+0000 7f07aca00640 1 -- start start 2024-03-12T17:07:25.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.958+0000 7f07aca00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f07a80add30 0x7f07a80ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:25.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.958+0000 7f07aca00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f07a806ce90 0x7f07a806d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:25.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.958+0000 7f07aca00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07a80cf250 con 0x7f07a80ce930 2024-03-12T17:07:25.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.958+0000 7f07aca00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a806c760 con 0x7f07a806ce90 2024-03-12T17:07:25.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.958+0000 7f07aca00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a80b42d0 con 0x7f07a80add30 2024-03-12T17:07:25.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.958+0000 7f07a6800640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f07a806ce90 0x7f07a806d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:25.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.958+0000 7f07a6800640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f07a806ce90 0x7f07a806d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50952/0 (socket says 172.21.15.52:50952) 2024-03-12T17:07:25.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.958+0000 7f07a6800640 1 -- 172.21.15.52:0/3542598394 learned_addr learned my addr 172.21.15.52:0/3542598394 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:25.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.959+0000 7f07a5400640 1 --2- 172.21.15.52:0/3542598394 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f07a80add30 0x7f07a80ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:25.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.959+0000 7f07a6800640 1 -- 172.21.15.52:0/3542598394 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f07a80add30 msgr2=0x7f07a80ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:25.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.959+0000 7f07a6800640 1 --2- 172.21.15.52:0/3542598394 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f07a80add30 0x7f07a80ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:25.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.959+0000 7f07a6800640 1 -- 172.21.15.52:0/3542598394 >> v1:172.21.15.102:6789/0 conn(0x7f07a80ce930 legacy=0x7f07a80cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:25.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.959+0000 7f07a6800640 1 -- 172.21.15.52:0/3542598394 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a80cdba0 con 0x7f07a806ce90 2024-03-12T17:07:25.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.960+0000 7f07a6800640 1 --2- 172.21.15.52:0/3542598394 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07a806ce90 0x7f07a806d320 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f07a0009e50 tx=0x7f07a002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9edd21c9146b15b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:25.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.960+0000 7f07a4a00640 1 -- 172.21.15.52:0/3542598394 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a002ca70 con 0x7f07a806ce90 2024-03-12T17:07:25.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.960+0000 7f07a4a00640 1 -- 172.21.15.52:0/3542598394 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a002cbd0 con 0x7f07a806ce90 2024-03-12T17:07:25.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.960+0000 7f07a4a00640 1 -- 172.21.15.52:0/3542598394 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a00343f0 con 0x7f07a806ce90 2024-03-12T17:07:25.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.961+0000 7f07aca00640 1 -- 172.21.15.52:0/3542598394 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07a806ce90 msgr2=0x7f07a806d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:25.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.961+0000 7f07aca00640 1 --2- 172.21.15.52:0/3542598394 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07a806ce90 0x7f07a806d320 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f07a0009e50 tx=0x7f07a002bed0 comp rx=0 tx=0).stop 2024-03-12T17:07:25.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.961+0000 7f07aca00640 1 -- 172.21.15.52:0/3542598394 shutdown_connections 2024-03-12T17:07:25.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.961+0000 7f07aca00640 1 --2- 172.21.15.52:0/3542598394 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07a806ce90 0x7f07a806d320 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:25.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.961+0000 7f07aca00640 1 --2- 172.21.15.52:0/3542598394 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f07a80add30 0x7f07a80ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:25.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.961+0000 7f07aca00640 1 -- 172.21.15.52:0/3542598394 >> 172.21.15.52:0/3542598394 conn(0x7f07a806a0b0 msgr2=0x7f07a806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:25.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.961+0000 7f07aca00640 1 -- 172.21.15.52:0/3542598394 shutdown_connections 2024-03-12T17:07:25.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.961+0000 7f07aca00640 1 -- 172.21.15.52:0/3542598394 wait complete. 2024-03-12T17:07:25.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.963+0000 7f07aca00640 1 Processor -- start 2024-03-12T17:07:25.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.963+0000 7f07aca00640 1 -- start start 2024-03-12T17:07:25.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.964+0000 7f07aca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07a806ce90 0x7f07a81eff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:25.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.964+0000 7f07aca00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f07a80add30 0x7f07a8150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:25.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.964+0000 7f07aca00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07a80ce930 0x7f07a8156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:25.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.964+0000 7f07aca00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a81f0490 con 0x7f07a80ce930 2024-03-12T17:07:25.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.964+0000 7f07aca00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a8159050 con 0x7f07a806ce90 2024-03-12T17:07:25.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.964+0000 7f07aca00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a81598f0 con 0x7f07a80add30 2024-03-12T17:07:25.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.965+0000 7f07a6800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07a80ce930 0x7f07a8156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:25.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.965+0000 7f07a6800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07a80ce930 0x7f07a8156f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50966/0 (socket says 172.21.15.52:50966) 2024-03-12T17:07:25.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.965+0000 7f07a6800640 1 -- 172.21.15.52:0/1849272662 learned_addr learned my addr 172.21.15.52:0/1849272662 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:25.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.965+0000 7f07a5e00640 1 --2- 172.21.15.52:0/1849272662 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07a806ce90 0x7f07a81eff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:25.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.965+0000 7f07a5400640 1 --2- 172.21.15.52:0/1849272662 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f07a80add30 0x7f07a8150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:25.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.965+0000 7f07a6800640 1 -- 172.21.15.52:0/1849272662 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f07a80add30 msgr2=0x7f07a8150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:25.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.965+0000 7f07a6800640 1 --2- 172.21.15.52:0/1849272662 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f07a80add30 0x7f07a8150fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:25.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.965+0000 7f07a6800640 1 -- 172.21.15.52:0/1849272662 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07a806ce90 msgr2=0x7f07a81eff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:25.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.965+0000 7f07a6800640 1 --2- 172.21.15.52:0/1849272662 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07a806ce90 0x7f07a81eff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:25.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.965+0000 7f07a6800640 1 -- 172.21.15.52:0/1849272662 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a0009b30 con 0x7f07a80ce930 2024-03-12T17:07:25.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.966+0000 7f07a6800640 1 --2- 172.21.15.52:0/1849272662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07a80ce930 0x7f07a8156f90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f07a0009e50 tx=0x7f07a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:25.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.966+0000 7f079e000640 1 -- 172.21.15.52:0/1849272662 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a0009510 con 0x7f07a80ce930 2024-03-12T17:07:25.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.966+0000 7f079e000640 1 -- 172.21.15.52:0/1849272662 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a003e450 con 0x7f07a80ce930 2024-03-12T17:07:25.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.966+0000 7f079e000640 1 -- 172.21.15.52:0/1849272662 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a003e7e0 con 0x7f07a80ce930 2024-03-12T17:07:25.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.966+0000 7f07aca00640 1 -- 172.21.15.52:0/1849272662 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a8155f10 con 0x7f07a80ce930 2024-03-12T17:07:25.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.966+0000 7f07aca00640 1 -- 172.21.15.52:0/1849272662 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a8159580 con 0x7f07a80ce930 2024-03-12T17:07:25.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.967+0000 7f07aca00640 1 -- 172.21.15.52:0/1849272662 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07a8159740 con 0x7f07a80ce930 2024-03-12T17:07:25.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.968+0000 7f079e000640 1 -- 172.21.15.52:0/1849272662 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f07a003e940 con 0x7f07a80ce930 2024-03-12T17:07:25.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.969+0000 7f079e000640 1 --2- 172.21.15.52:0/1849272662 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f078806ac50 0x7f078806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:25.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.969+0000 7f079e000640 1 -- 172.21.15.52:0/1849272662 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f07a00ac4c0 con 0x7f07a80ce930 2024-03-12T17:07:25.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.970+0000 7f07a5e00640 1 --2- 172.21.15.52:0/1849272662 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f078806ac50 0x7f078806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:25.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.971+0000 7f07a5e00640 1 --2- 172.21.15.52:0/1849272662 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f078806ac50 0x7f078806d110 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0798006fd0 tx=0x7f0798008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:25.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:25.974+0000 7f079e000640 1 -- 172.21.15.52:0/1849272662 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f07a00772c0 con 0x7f07a80ce930 2024-03-12T17:07:26.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.186+0000 7f07aca00640 1 -- 172.21.15.52:0/1849272662 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f07a8159a60 con 0x7f07a80ce930 2024-03-12T17:07:26.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.187+0000 7f079e000640 1 -- 172.21.15.52:0/1849272662 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f07a007a8e0 con 0x7f07a80ce930 2024-03-12T17:07:26.181 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-12T17:07:26.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.190+0000 7f07aca00640 1 -- 172.21.15.52:0/1849272662 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f078806ac50 msgr2=0x7f078806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:26.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.190+0000 7f07aca00640 1 --2- 172.21.15.52:0/1849272662 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f078806ac50 0x7f078806d110 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0798006fd0 tx=0x7f0798008040 comp rx=0 tx=0).stop 2024-03-12T17:07:26.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.190+0000 7f07aca00640 1 -- 172.21.15.52:0/1849272662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07a80ce930 msgr2=0x7f07a8156f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:26.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.190+0000 7f07aca00640 1 --2- 172.21.15.52:0/1849272662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07a80ce930 0x7f07a8156f90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f07a0009e50 tx=0x7f07a0004a40 comp rx=0 tx=0).stop 2024-03-12T17:07:26.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.190+0000 7f07aca00640 1 -- 172.21.15.52:0/1849272662 shutdown_connections 2024-03-12T17:07:26.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.190+0000 7f07aca00640 1 --2- 172.21.15.52:0/1849272662 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f078806ac50 0x7f078806d110 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:26.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.190+0000 7f07aca00640 1 --2- 172.21.15.52:0/1849272662 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f07a80add30 0x7f07a8150fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:26.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.190+0000 7f07aca00640 1 --2- 172.21.15.52:0/1849272662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07a80ce930 0x7f07a8156f90 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:26.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.190+0000 7f07aca00640 1 --2- 172.21.15.52:0/1849272662 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07a806ce90 0x7f07a81eff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:26.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.190+0000 7f07aca00640 1 -- 172.21.15.52:0/1849272662 >> 172.21.15.52:0/1849272662 conn(0x7f07a806a0b0 msgr2=0x7f07a815cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:26.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.190+0000 7f07aca00640 1 -- 172.21.15.52:0/1849272662 shutdown_connections 2024-03-12T17:07:26.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.190+0000 7f07aca00640 1 -- 172.21.15.52:0/1849272662 wait complete. 2024-03-12T17:07:26.202 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-12T17:07:26.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.429+0000 7fa0b3200640 1 Processor -- start 2024-03-12T17:07:26.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.430+0000 7fa0b3200640 1 -- start start 2024-03-12T17:07:26.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.431+0000 7fa0b3200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa0b415a850 0x7fa0b40cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:26.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.431+0000 7fa0b3200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa0b40b22b0 0x7fa0b415eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:26.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.431+0000 7fa0b3200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0b415e320 con 0x7fa0b415f020 2024-03-12T17:07:26.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.431+0000 7fa0b3200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0b415ac30 con 0x7fa0b40b22b0 2024-03-12T17:07:26.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.431+0000 7fa0b3200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b40cda20 con 0x7fa0b415a850 2024-03-12T17:07:26.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.432+0000 7fa0b1400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa0b40b22b0 0x7fa0b415eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:26.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.432+0000 7fa0b1400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa0b40b22b0 0x7fa0b415eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:38938/0 (socket says 172.21.15.52:38938) 2024-03-12T17:07:26.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.432+0000 7fa0b1400640 1 -- 172.21.15.52:0/3194872433 learned_addr learned my addr 172.21.15.52:0/3194872433 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:26.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.432+0000 7fa0b1e00640 1 --2- 172.21.15.52:0/3194872433 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa0b415a850 0x7fa0b40cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:26.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.433+0000 7fa0b1400640 1 -- 172.21.15.52:0/3194872433 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa0b415a850 msgr2=0x7fa0b40cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:26.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.433+0000 7fa0b1400640 1 --2- 172.21.15.52:0/3194872433 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa0b415a850 0x7fa0b40cd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:26.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.433+0000 7fa0b1400640 1 -- 172.21.15.52:0/3194872433 >> v1:172.21.15.102:6789/0 conn(0x7fa0b415f020 legacy=0x7fa0b40cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:26.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.433+0000 7fa0b1400640 1 -- 172.21.15.52:0/3194872433 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b40cdd00 con 0x7fa0b40b22b0 2024-03-12T17:07:26.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.434+0000 7fa0b1400640 1 --2- 172.21.15.52:0/3194872433 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0b40b22b0 0x7fa0b415eae0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4009e50 tx=0x7fa0a402bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=530f142c39aff629 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:26.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.435+0000 7fa0b0a00640 1 -- 172.21.15.52:0/3194872433 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a402cb00 con 0x7fa0b40b22b0 2024-03-12T17:07:26.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.435+0000 7fa0b0a00640 1 -- 172.21.15.52:0/3194872433 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a402cc60 con 0x7fa0b40b22b0 2024-03-12T17:07:26.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.435+0000 7fa0b0a00640 1 -- 172.21.15.52:0/3194872433 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a40343f0 con 0x7fa0b40b22b0 2024-03-12T17:07:26.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.435+0000 7fa0b3200640 1 -- 172.21.15.52:0/3194872433 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0b40b22b0 msgr2=0x7fa0b415eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:26.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.436+0000 7fa0b3200640 1 --2- 172.21.15.52:0/3194872433 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0b40b22b0 0x7fa0b415eae0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4009e50 tx=0x7fa0a402bf60 comp rx=0 tx=0).stop 2024-03-12T17:07:26.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.436+0000 7fa0b3200640 1 -- 172.21.15.52:0/3194872433 shutdown_connections 2024-03-12T17:07:26.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.436+0000 7fa0b3200640 1 --2- 172.21.15.52:0/3194872433 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0b40b22b0 0x7fa0b415eae0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:26.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.436+0000 7fa0b3200640 1 --2- 172.21.15.52:0/3194872433 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa0b415a850 0x7fa0b40cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:26.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.436+0000 7fa0b3200640 1 -- 172.21.15.52:0/3194872433 >> 172.21.15.52:0/3194872433 conn(0x7fa0b40b01b0 msgr2=0x7fa0b40b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:26.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.436+0000 7fa0b3200640 1 -- 172.21.15.52:0/3194872433 shutdown_connections 2024-03-12T17:07:26.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.436+0000 7fa0b3200640 1 -- 172.21.15.52:0/3194872433 wait complete. 2024-03-12T17:07:26.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.438+0000 7fa0b3200640 1 Processor -- start 2024-03-12T17:07:26.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.438+0000 7fa0b3200640 1 -- start start 2024-03-12T17:07:26.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.440+0000 7fa0b3200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa0b40b22b0 0x7fa0b414b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:26.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.440+0000 7fa0b3200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0b415a850 0x7fa0b415c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:26.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.440+0000 7fa0b3200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0b415f020 0x7fa0b415ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:26.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.440+0000 7fa0b3200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b415b6e0 con 0x7fa0b415a850 2024-03-12T17:07:26.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.440+0000 7fa0b3200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b415b9c0 con 0x7fa0b415f020 2024-03-12T17:07:26.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.440+0000 7fa0b3200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b415bca0 con 0x7fa0b40b22b0 2024-03-12T17:07:26.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.440+0000 7fa0b1400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0b415a850 0x7fa0b415c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:26.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.440+0000 7fa0b1400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0b415a850 0x7fa0b415c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50968/0 (socket says 172.21.15.52:50968) 2024-03-12T17:07:26.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.440+0000 7fa0b1400640 1 -- 172.21.15.52:0/3811600673 learned_addr learned my addr 172.21.15.52:0/3811600673 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:26.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.440+0000 7fa0b1e00640 1 --2- 172.21.15.52:0/3811600673 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa0b40b22b0 0x7fa0b414b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:26.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.440+0000 7fa0b2800640 1 --2- 172.21.15.52:0/3811600673 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0b415f020 0x7fa0b415ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:26.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.441+0000 7fa0b1400640 1 -- 172.21.15.52:0/3811600673 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa0b40b22b0 msgr2=0x7fa0b414b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:26.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.441+0000 7fa0b1400640 1 --2- 172.21.15.52:0/3811600673 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa0b40b22b0 0x7fa0b414b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:26.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.441+0000 7fa0b1400640 1 -- 172.21.15.52:0/3811600673 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0b415f020 msgr2=0x7fa0b415ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:26.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.441+0000 7fa0b1400640 1 --2- 172.21.15.52:0/3811600673 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0b415f020 0x7fa0b415ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:26.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.441+0000 7fa0b1400640 1 -- 172.21.15.52:0/3811600673 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a4009b30 con 0x7fa0b415a850 2024-03-12T17:07:26.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.441+0000 7fa0b2800640 1 --2- 172.21.15.52:0/3811600673 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0b415f020 0x7fa0b415ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:26.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.441+0000 7fa0b1400640 1 --2- 172.21.15.52:0/3811600673 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0b415a850 0x7fa0b415c200 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4000f80 tx=0x7fa0a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:26.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.441+0000 7fa0aa000640 1 -- 172.21.15.52:0/3811600673 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a4009510 con 0x7fa0b415a850 2024-03-12T17:07:26.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.441+0000 7fa0aa000640 1 -- 172.21.15.52:0/3811600673 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a403e450 con 0x7fa0b415a850 2024-03-12T17:07:26.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.441+0000 7fa0aa000640 1 -- 172.21.15.52:0/3811600673 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a403e7e0 con 0x7fa0b415a850 2024-03-12T17:07:26.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.441+0000 7fa0b3200640 1 -- 172.21.15.52:0/3811600673 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b4152c90 con 0x7fa0b415a850 2024-03-12T17:07:26.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.442+0000 7fa0b3200640 1 -- 172.21.15.52:0/3811600673 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b414bc80 con 0x7fa0b415a850 2024-03-12T17:07:26.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.443+0000 7fa0b3200640 1 -- 172.21.15.52:0/3811600673 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b40735a0 con 0x7fa0b415a850 2024-03-12T17:07:26.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.444+0000 7fa0aa000640 1 -- 172.21.15.52:0/3811600673 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa0a403e940 con 0x7fa0b415a850 2024-03-12T17:07:26.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.445+0000 7fa0aa000640 1 --2- 172.21.15.52:0/3811600673 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa09006ac00 0x7fa09006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:26.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.445+0000 7fa0aa000640 1 -- 172.21.15.52:0/3811600673 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa0a40ac470 con 0x7fa0b415a850 2024-03-12T17:07:26.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.445+0000 7fa0b1e00640 1 --2- 172.21.15.52:0/3811600673 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa09006ac00 0x7fa09006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:26.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.446+0000 7fa0b1e00640 1 --2- 172.21.15.52:0/3811600673 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa09006ac00 0x7fa09006d0c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa09c009ad0 tx=0x7fa09c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:26.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.449+0000 7fa0aa000640 1 -- 172.21.15.52:0/3811600673 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fa0a4077320 con 0x7fa0b415a850 2024-03-12T17:07:26.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.652+0000 7fa0b3200640 1 -- 172.21.15.52:0/3811600673 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa0b404e030 con 0x7fa0b415a850 2024-03-12T17:07:26.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.653+0000 7fa0aa000640 1 -- 172.21.15.52:0/3811600673 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa0a407a940 con 0x7fa0b415a850 2024-03-12T17:07:26.647 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-03-12T17:07:26.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.656+0000 7fa0b3200640 1 -- 172.21.15.52:0/3811600673 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa09006ac00 msgr2=0x7fa09006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:26.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.656+0000 7fa0b3200640 1 --2- 172.21.15.52:0/3811600673 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa09006ac00 0x7fa09006d0c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa09c009ad0 tx=0x7fa09c008040 comp rx=0 tx=0).stop 2024-03-12T17:07:26.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.656+0000 7fa0b3200640 1 -- 172.21.15.52:0/3811600673 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0b415a850 msgr2=0x7fa0b415c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:26.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.656+0000 7fa0b3200640 1 --2- 172.21.15.52:0/3811600673 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0b415a850 0x7fa0b415c200 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4000f80 tx=0x7fa0a4004a40 comp rx=0 tx=0).stop 2024-03-12T17:07:26.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.656+0000 7fa0b3200640 1 -- 172.21.15.52:0/3811600673 shutdown_connections 2024-03-12T17:07:26.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.656+0000 7fa0b3200640 1 --2- 172.21.15.52:0/3811600673 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa09006ac00 0x7fa09006d0c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:26.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.656+0000 7fa0b3200640 1 --2- 172.21.15.52:0/3811600673 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0b415f020 0x7fa0b415ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:26.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.656+0000 7fa0b3200640 1 --2- 172.21.15.52:0/3811600673 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0b415a850 0x7fa0b415c200 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:26.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.656+0000 7fa0b3200640 1 --2- 172.21.15.52:0/3811600673 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa0b40b22b0 0x7fa0b414b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:26.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.656+0000 7fa0b3200640 1 -- 172.21.15.52:0/3811600673 >> 172.21.15.52:0/3811600673 conn(0x7fa0b40b01b0 msgr2=0x7fa0b40b0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:26.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.656+0000 7fa0b3200640 1 -- 172.21.15.52:0/3811600673 shutdown_connections 2024-03-12T17:07:26.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.656+0000 7fa0b3200640 1 -- 172.21.15.52:0/3811600673 wait complete. 2024-03-12T17:07:26.669 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-12T17:07:26.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.893+0000 7f5197000640 1 Processor -- start 2024-03-12T17:07:26.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.894+0000 7f5197000640 1 -- start start 2024-03-12T17:07:26.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.895+0000 7f5197000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f51900d22d0 0x7f51900cf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:26.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.895+0000 7f5197000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f51900cf600 0x7f51900c6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:26.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.895+0000 7f5197000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51900c6290 con 0x7f5190160170 2024-03-12T17:07:26.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.895+0000 7f5197000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51900cfa90 con 0x7f51900cf600 2024-03-12T17:07:26.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.895+0000 7f5197000640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51900cdcd0 con 0x7f51900d22d0 2024-03-12T17:07:26.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.896+0000 7f5196600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f51900cf600 0x7f51900c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:26.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.896+0000 7f5195200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f51900d22d0 0x7f51900cf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:26.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.896+0000 7f5196600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f51900cf600 0x7f51900c6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:38954/0 (socket says 172.21.15.52:38954) 2024-03-12T17:07:26.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.896+0000 7f5196600640 1 -- 172.21.15.52:0/4187391091 learned_addr learned my addr 172.21.15.52:0/4187391091 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:26.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.896+0000 7f5196600640 1 -- 172.21.15.52:0/4187391091 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f51900d22d0 msgr2=0x7f51900cf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:26.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.896+0000 7f5196600640 1 --2- 172.21.15.52:0/4187391091 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f51900d22d0 0x7f51900cf0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:26.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.896+0000 7f5196600640 1 -- 172.21.15.52:0/4187391091 >> v1:172.21.15.102:6789/0 conn(0x7f5190160170 legacy=0x7f51900d2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:26.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.897+0000 7f5196600640 1 -- 172.21.15.52:0/4187391091 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51900d34d0 con 0x7f51900cf600 2024-03-12T17:07:26.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.898+0000 7f5196600640 1 --2- 172.21.15.52:0/4187391091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51900cf600 0x7f51900c6720 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5188009e50 tx=0x7f518802bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b05656a52759020 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:26.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.898+0000 7f518fe00640 1 -- 172.21.15.52:0/4187391091 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518802ca70 con 0x7f51900cf600 2024-03-12T17:07:26.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.898+0000 7f518fe00640 1 -- 172.21.15.52:0/4187391091 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518802cbd0 con 0x7f51900cf600 2024-03-12T17:07:26.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.898+0000 7f518fe00640 1 -- 172.21.15.52:0/4187391091 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51880343f0 con 0x7f51900cf600 2024-03-12T17:07:26.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.899+0000 7f5197000640 1 -- 172.21.15.52:0/4187391091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51900cf600 msgr2=0x7f51900c6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:26.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.899+0000 7f5197000640 1 --2- 172.21.15.52:0/4187391091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51900cf600 0x7f51900c6720 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5188009e50 tx=0x7f518802bed0 comp rx=0 tx=0).stop 2024-03-12T17:07:26.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.899+0000 7f5197000640 1 -- 172.21.15.52:0/4187391091 shutdown_connections 2024-03-12T17:07:26.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.899+0000 7f5197000640 1 --2- 172.21.15.52:0/4187391091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51900cf600 0x7f51900c6720 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:26.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.899+0000 7f5197000640 1 --2- 172.21.15.52:0/4187391091 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f51900d22d0 0x7f51900cf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:26.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.899+0000 7f5197000640 1 -- 172.21.15.52:0/4187391091 >> 172.21.15.52:0/4187391091 conn(0x7f5190089fd0 msgr2=0x7f51900d26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:26.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.899+0000 7f5197000640 1 -- 172.21.15.52:0/4187391091 shutdown_connections 2024-03-12T17:07:26.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.899+0000 7f5197000640 1 -- 172.21.15.52:0/4187391091 wait complete. 2024-03-12T17:07:26.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.901+0000 7f5197000640 1 Processor -- start 2024-03-12T17:07:26.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.901+0000 7f5197000640 1 -- start start 2024-03-12T17:07:26.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.902+0000 7f5197000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51900cf600 0x7f519006b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:26.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.902+0000 7f5197000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51900d22d0 0x7f519009f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:26.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.902+0000 7f5197000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5190160170 0x7f519015b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:26.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.902+0000 7f5197000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519009fc90 con 0x7f51900d22d0 2024-03-12T17:07:26.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.902+0000 7f5197000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51900b5340 con 0x7f51900cf600 2024-03-12T17:07:26.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.902+0000 7f5197000640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519015be80 con 0x7f5190160170 2024-03-12T17:07:26.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.902+0000 7f5195200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51900d22d0 0x7f519009f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:26.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.902+0000 7f5195200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51900d22d0 0x7f519009f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50984/0 (socket says 172.21.15.52:50984) 2024-03-12T17:07:26.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.902+0000 7f5195200640 1 -- 172.21.15.52:0/2974974261 learned_addr learned my addr 172.21.15.52:0/2974974261 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:26.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.903+0000 7f5196600640 1 --2- 172.21.15.52:0/2974974261 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5190160170 0x7f519015b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:26.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.903+0000 7f5195c00640 1 --2- 172.21.15.52:0/2974974261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51900cf600 0x7f519006b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:26.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.903+0000 7f5195200640 1 -- 172.21.15.52:0/2974974261 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5190160170 msgr2=0x7f519015b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:26.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.903+0000 7f5195200640 1 --2- 172.21.15.52:0/2974974261 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5190160170 0x7f519015b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:26.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.903+0000 7f5195200640 1 -- 172.21.15.52:0/2974974261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51900cf600 msgr2=0x7f519006b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:26.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.903+0000 7f5195200640 1 --2- 172.21.15.52:0/2974974261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51900cf600 0x7f519006b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:26.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.903+0000 7f5195200640 1 -- 172.21.15.52:0/2974974261 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5188009b30 con 0x7f51900d22d0 2024-03-12T17:07:26.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.903+0000 7f5195200640 1 --2- 172.21.15.52:0/2974974261 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51900d22d0 0x7f519009f750 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f517c00e9b0 tx=0x7f517c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:26.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.903+0000 7f518d600640 1 -- 172.21.15.52:0/2974974261 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c009670 con 0x7f51900d22d0 2024-03-12T17:07:26.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.903+0000 7f518d600640 1 -- 172.21.15.52:0/2974974261 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517c00c9a0 con 0x7f51900d22d0 2024-03-12T17:07:26.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.903+0000 7f518d600640 1 -- 172.21.15.52:0/2974974261 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c00cd50 con 0x7f51900d22d0 2024-03-12T17:07:26.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.903+0000 7f5197000640 1 -- 172.21.15.52:0/2974974261 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51900b55e0 con 0x7f51900d22d0 2024-03-12T17:07:26.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.904+0000 7f5197000640 1 -- 172.21.15.52:0/2974974261 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51900b5ba0 con 0x7f51900d22d0 2024-03-12T17:07:26.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.905+0000 7f5197000640 1 -- 172.21.15.52:0/2974974261 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51900b5d60 con 0x7f51900d22d0 2024-03-12T17:07:26.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.906+0000 7f518d600640 1 -- 172.21.15.52:0/2974974261 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f517c013470 con 0x7f51900d22d0 2024-03-12T17:07:26.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.907+0000 7f518d600640 1 --2- 172.21.15.52:0/2974974261 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f517006ac00 0x7f517006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:26.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.907+0000 7f518d600640 1 -- 172.21.15.52:0/2974974261 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f517c0899c0 con 0x7f51900d22d0 2024-03-12T17:07:26.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.907+0000 7f5195c00640 1 --2- 172.21.15.52:0/2974974261 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f517006ac00 0x7f517006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:26.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.908+0000 7f5195c00640 1 --2- 172.21.15.52:0/2974974261 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f517006ac00 0x7f517006d0c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5184001640 tx=0x7f5184009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:26.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:26.911+0000 7f518d600640 1 -- 172.21.15.52:0/2974974261 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f517c0547d0 con 0x7f51900d22d0 2024-03-12T17:07:27.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.113+0000 7f5197000640 1 -- 172.21.15.52:0/2974974261 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f519004e030 con 0x7f51900d22d0 2024-03-12T17:07:27.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.114+0000 7f518d600640 1 -- 172.21.15.52:0/2974974261 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f517c057df0 con 0x7f51900d22d0 2024-03-12T17:07:27.108 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-12T17:07:27.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.117+0000 7f5197000640 1 -- 172.21.15.52:0/2974974261 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f517006ac00 msgr2=0x7f517006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:27.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.117+0000 7f5197000640 1 --2- 172.21.15.52:0/2974974261 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f517006ac00 0x7f517006d0c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5184001640 tx=0x7f5184009040 comp rx=0 tx=0).stop 2024-03-12T17:07:27.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.117+0000 7f5197000640 1 -- 172.21.15.52:0/2974974261 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51900d22d0 msgr2=0x7f519009f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:27.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.117+0000 7f5197000640 1 --2- 172.21.15.52:0/2974974261 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51900d22d0 0x7f519009f750 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f517c00e9b0 tx=0x7f517c00ee70 comp rx=0 tx=0).stop 2024-03-12T17:07:27.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.117+0000 7f5197000640 1 -- 172.21.15.52:0/2974974261 shutdown_connections 2024-03-12T17:07:27.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.117+0000 7f5197000640 1 --2- 172.21.15.52:0/2974974261 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f517006ac00 0x7f517006d0c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:27.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.117+0000 7f5197000640 1 --2- 172.21.15.52:0/2974974261 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5190160170 0x7f519015b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:27.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.117+0000 7f5197000640 1 --2- 172.21.15.52:0/2974974261 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51900d22d0 0x7f519009f750 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:27.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.117+0000 7f5197000640 1 --2- 172.21.15.52:0/2974974261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51900cf600 0x7f519006b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:27.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.117+0000 7f5197000640 1 -- 172.21.15.52:0/2974974261 >> 172.21.15.52:0/2974974261 conn(0x7f5190089fd0 msgr2=0x7f519015ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:27.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.117+0000 7f5197000640 1 -- 172.21.15.52:0/2974974261 shutdown_connections 2024-03-12T17:07:27.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.117+0000 7f5197000640 1 -- 172.21.15.52:0/2974974261 wait complete. 2024-03-12T17:07:27.128 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:07:27.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.343+0000 7f178e400640 1 Processor -- start 2024-03-12T17:07:27.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.344+0000 7f178e400640 1 -- start start 2024-03-12T17:07:27.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.345+0000 7f178e400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f17880b3310 0x7f17880cae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:27.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.345+0000 7f178e400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f17880cb370 0x7f17880b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:27.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.345+0000 7f178e400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17880cc8c0 con 0x7f1788156e30 2024-03-12T17:07:27.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.345+0000 7f178e400640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17880cb7c0 con 0x7f17880b3310 2024-03-12T17:07:27.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.345+0000 7f178e400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178815ce60 con 0x7f17880cb370 2024-03-12T17:07:27.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.346+0000 7f1787e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f17880b3310 0x7f17880cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:27.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.346+0000 7f1787e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f17880b3310 0x7f17880cae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50986/0 (socket says 172.21.15.52:50986) 2024-03-12T17:07:27.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.346+0000 7f1787e00640 1 -- 172.21.15.52:0/334612005 learned_addr learned my addr 172.21.15.52:0/334612005 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:27.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.346+0000 7f178da00640 1 --2- 172.21.15.52:0/334612005 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f17880cb370 0x7f17880b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:27.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.346+0000 7f1787e00640 1 -- 172.21.15.52:0/334612005 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f17880cb370 msgr2=0x7f17880b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:27.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.346+0000 7f1787e00640 1 --2- 172.21.15.52:0/334612005 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f17880cb370 0x7f17880b1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:27.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.346+0000 7f1787e00640 1 -- 172.21.15.52:0/334612005 >> v1:172.21.15.102:6789/0 conn(0x7f1788156e30 legacy=0x7f178806d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:27.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.346+0000 7f1787e00640 1 -- 172.21.15.52:0/334612005 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f178815d120 con 0x7f17880b3310 2024-03-12T17:07:27.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.347+0000 7f1787e00640 1 --2- 172.21.15.52:0/334612005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17880b3310 0x7f17880cae30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1774009870 tx=0x7f177402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317c9c6a92106c58 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:27.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.348+0000 7f1787400640 1 -- 172.21.15.52:0/334612005 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1774030840 con 0x7f17880b3310 2024-03-12T17:07:27.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.348+0000 7f1787400640 1 -- 172.21.15.52:0/334612005 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17740309a0 con 0x7f17880b3310 2024-03-12T17:07:27.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.348+0000 7f1787400640 1 -- 172.21.15.52:0/334612005 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1774030d30 con 0x7f17880b3310 2024-03-12T17:07:27.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.348+0000 7f178e400640 1 -- 172.21.15.52:0/334612005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17880b3310 msgr2=0x7f17880cae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:27.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.348+0000 7f178e400640 1 --2- 172.21.15.52:0/334612005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17880b3310 0x7f17880cae30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1774009870 tx=0x7f177402b980 comp rx=0 tx=0).stop 2024-03-12T17:07:27.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.348+0000 7f178e400640 1 -- 172.21.15.52:0/334612005 shutdown_connections 2024-03-12T17:07:27.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.348+0000 7f178e400640 1 --2- 172.21.15.52:0/334612005 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f17880cb370 0x7f17880b1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:27.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.348+0000 7f178e400640 1 --2- 172.21.15.52:0/334612005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17880b3310 0x7f17880cae30 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:27.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.348+0000 7f178e400640 1 -- 172.21.15.52:0/334612005 >> 172.21.15.52:0/334612005 conn(0x7f1788065df0 msgr2=0x7f1788069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:27.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.349+0000 7f178e400640 1 -- 172.21.15.52:0/334612005 shutdown_connections 2024-03-12T17:07:27.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.349+0000 7f178e400640 1 -- 172.21.15.52:0/334612005 wait complete. 2024-03-12T17:07:27.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.350+0000 7f178e400640 1 Processor -- start 2024-03-12T17:07:27.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.350+0000 7f178e400640 1 -- start start 2024-03-12T17:07:27.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.351+0000 7f178e400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f17880b3310 0x7f17880b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:27.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.351+0000 7f178e400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17880cb370 0x7f1788142cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:27.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.351+0000 7f178e400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1788156e30 0x7f178814e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:27.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.351+0000 7f178e400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178814dfa0 con 0x7f1788156e30 2024-03-12T17:07:27.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.351+0000 7f178e400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1788143230 con 0x7f17880cb370 2024-03-12T17:07:27.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.351+0000 7f178e400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178814d320 con 0x7f17880b3310 2024-03-12T17:07:27.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.352+0000 7f178da00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1788156e30 0x7f178814e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:27.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.352+0000 7f178da00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1788156e30 0x7f178814e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50990/0 (socket says 172.21.15.52:50990) 2024-03-12T17:07:27.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.352+0000 7f178da00640 1 -- 172.21.15.52:0/627704848 learned_addr learned my addr 172.21.15.52:0/627704848 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:27.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.352+0000 7f1787e00640 1 --2- 172.21.15.52:0/627704848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17880cb370 0x7f1788142cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:27.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.352+0000 7f178da00640 1 -- 172.21.15.52:0/627704848 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f17880b3310 msgr2=0x7f17880b0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:27.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.352+0000 7f178d000640 1 --2- 172.21.15.52:0/627704848 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f17880b3310 0x7f17880b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:27.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.352+0000 7f178da00640 1 --2- 172.21.15.52:0/627704848 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f17880b3310 0x7f17880b0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:27.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.352+0000 7f178da00640 1 -- 172.21.15.52:0/627704848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17880cb370 msgr2=0x7f1788142cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:27.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.352+0000 7f178da00640 1 --2- 172.21.15.52:0/627704848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17880cb370 0x7f1788142cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:27.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.352+0000 7f178da00640 1 -- 172.21.15.52:0/627704848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17740094d0 con 0x7f1788156e30 2024-03-12T17:07:27.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.352+0000 7f1787e00640 1 --2- 172.21.15.52:0/627704848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17880cb370 0x7f1788142cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:27.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.352+0000 7f178da00640 1 --2- 172.21.15.52:0/627704848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1788156e30 0x7f178814e3c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1780007e40 tx=0x7f178000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:27.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.353+0000 7f1784c00640 1 -- 172.21.15.52:0/627704848 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1780014620 con 0x7f1788156e30 2024-03-12T17:07:27.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.353+0000 7f1784c00640 1 -- 172.21.15.52:0/627704848 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f178000ee90 con 0x7f1788156e30 2024-03-12T17:07:27.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.353+0000 7f1784c00640 1 -- 172.21.15.52:0/627704848 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17800149e0 con 0x7f1788156e30 2024-03-12T17:07:27.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.353+0000 7f178e400640 1 -- 172.21.15.52:0/627704848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f178814d600 con 0x7f1788156e30 2024-03-12T17:07:27.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.353+0000 7f178e400640 1 -- 172.21.15.52:0/627704848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f178814dbc0 con 0x7f1788156e30 2024-03-12T17:07:27.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.354+0000 7f178e400640 1 -- 172.21.15.52:0/627704848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f178814e0e0 con 0x7f1788156e30 2024-03-12T17:07:27.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.355+0000 7f1784c00640 1 -- 172.21.15.52:0/627704848 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1780014b40 con 0x7f1788156e30 2024-03-12T17:07:27.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.356+0000 7f1784c00640 1 --2- 172.21.15.52:0/627704848 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f176806ac00 0x7f176806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:27.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.356+0000 7f1784c00640 1 -- 172.21.15.52:0/627704848 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f178008a190 con 0x7f1788156e30 2024-03-12T17:07:27.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.356+0000 7f178d000640 1 --2- 172.21.15.52:0/627704848 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f176806ac00 0x7f176806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:27.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.358+0000 7f178d000640 1 --2- 172.21.15.52:0/627704848 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f176806ac00 0x7f176806d0c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f177c000f60 tx=0x7f177c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:27.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.361+0000 7f1784c00640 1 -- 172.21.15.52:0/627704848 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f178814e0e0 con 0x7f1788156e30 2024-03-12T17:07:27.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.559+0000 7f178e400640 1 -- 172.21.15.52:0/627704848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f178814e0e0 con 0x7f1788156e30 2024-03-12T17:07:27.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.561+0000 7f1784c00640 1 -- 172.21.15.52:0/627704848 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18207 (secure 0 0 0) 0x7f178814e0e0 con 0x7f1788156e30 2024-03-12T17:07:27.555 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:07:27.555 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":27,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:07:12.969614+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T17:06:19.218543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T17:06:23.525474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T17:06:24.911574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-12T17:06:28.017172+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.733031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.775178+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.591522+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.820838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.829252+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.582481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.18","mappings":[{"from":3,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:07:27.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.564+0000 7f178e400640 1 -- 172.21.15.52:0/627704848 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f176806ac00 msgr2=0x7f176806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:27.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.564+0000 7f178e400640 1 --2- 172.21.15.52:0/627704848 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f176806ac00 0x7f176806d0c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f177c000f60 tx=0x7f177c009040 comp rx=0 tx=0).stop 2024-03-12T17:07:27.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.564+0000 7f178e400640 1 -- 172.21.15.52:0/627704848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1788156e30 msgr2=0x7f178814e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:27.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.564+0000 7f178e400640 1 --2- 172.21.15.52:0/627704848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1788156e30 0x7f178814e3c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1780007e40 tx=0x7f178000e4b0 comp rx=0 tx=0).stop 2024-03-12T17:07:27.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.564+0000 7f178e400640 1 -- 172.21.15.52:0/627704848 shutdown_connections 2024-03-12T17:07:27.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.564+0000 7f178e400640 1 --2- 172.21.15.52:0/627704848 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f176806ac00 0x7f176806d0c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:27.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.564+0000 7f178e400640 1 --2- 172.21.15.52:0/627704848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1788156e30 0x7f178814e3c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:27.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.564+0000 7f178e400640 1 --2- 172.21.15.52:0/627704848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17880cb370 0x7f1788142cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:27.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.564+0000 7f178e400640 1 --2- 172.21.15.52:0/627704848 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f17880b3310 0x7f17880b0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:27.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.564+0000 7f178e400640 1 -- 172.21.15.52:0/627704848 >> 172.21.15.52:0/627704848 conn(0x7f1788065df0 msgr2=0x7f17880abe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:27.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.564+0000 7f178e400640 1 -- 172.21.15.52:0/627704848 shutdown_connections 2024-03-12T17:07:27.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.564+0000 7f178e400640 1 -- 172.21.15.52:0/627704848 wait complete. 2024-03-12T17:07:27.577 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T17:06:19.218543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-12T17:06:23.525474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-12T17:06:24.911574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-12T17:06:28.017172+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-12T17:07:27.578 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T17:07:27.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.796+0000 7f0011800640 1 Processor -- start 2024-03-12T17:07:27.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.797+0000 7f0011800640 1 -- start start 2024-03-12T17:07:27.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.798+0000 7f0011800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f000c15c820 0x7f000c0cd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:27.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.798+0000 7f0011800640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f000c160e90 0x7f000c0cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:27.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.798+0000 7f0011800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f000c0cf5f0 0x7f000c0d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:27.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.798+0000 7f0011800640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f000c0cdd70 con 0x7f000c160e90 2024-03-12T17:07:27.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.798+0000 7f0011800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f000c0cfa60 con 0x7f000c15c820 2024-03-12T17:07:27.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.798+0000 7f0011800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000c0ccd90 con 0x7f000c0cf5f0 2024-03-12T17:07:27.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.798+0000 7f000b400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f000c160e90 0x7f000c0cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:27.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.798+0000 7f000b400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f000c160e90 0x7f000c0cf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51004/0 (socket says 172.21.15.52:51004) 2024-03-12T17:07:27.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.798+0000 7f000b400640 1 -- 172.21.15.52:0/2110518123 learned_addr learned my addr 172.21.15.52:0/2110518123 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:27.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.799+0000 7f000be00640 1 --2- 172.21.15.52:0/2110518123 >> v2:172.21.15.102:3300/0 conn(0x7f000c15c820 0x7f000c0cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:27.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.799+0000 7f0010e00640 1 --2- 172.21.15.52:0/2110518123 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f000c0cf5f0 0x7f000c0d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:27.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.799+0000 7f000b400640 1 -- 172.21.15.52:0/2110518123 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f000c0cf5f0 msgr2=0x7f000c0d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:27.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.799+0000 7f000b400640 1 --2- 172.21.15.52:0/2110518123 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f000c0cf5f0 0x7f000c0d0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:27.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.799+0000 7f000b400640 1 -- 172.21.15.52:0/2110518123 >> v2:172.21.15.102:3300/0 conn(0x7f000c15c820 msgr2=0x7f000c0cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:27.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.799+0000 7f000b400640 1 --2- 172.21.15.52:0/2110518123 >> v2:172.21.15.102:3300/0 conn(0x7f000c15c820 0x7f000c0cd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:27.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.799+0000 7f000b400640 1 -- 172.21.15.52:0/2110518123 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000c0b7fc0 con 0x7f000c160e90 2024-03-12T17:07:27.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.800+0000 7f000b400640 1 --2- 172.21.15.52:0/2110518123 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f000c160e90 0x7f000c0cf0b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f000c08ab20 tx=0x7efff402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=950eea71b84fe3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:27.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.800+0000 7f000aa00640 1 -- 172.21.15.52:0/2110518123 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff402cb00 con 0x7f000c160e90 2024-03-12T17:07:27.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.800+0000 7f000aa00640 1 -- 172.21.15.52:0/2110518123 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff402cc60 con 0x7f000c160e90 2024-03-12T17:07:27.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.800+0000 7f000aa00640 1 -- 172.21.15.52:0/2110518123 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff40343f0 con 0x7f000c160e90 2024-03-12T17:07:27.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.801+0000 7f0011800640 1 -- 172.21.15.52:0/2110518123 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f000c160e90 msgr2=0x7f000c0cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:27.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.801+0000 7f0011800640 1 --2- 172.21.15.52:0/2110518123 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f000c160e90 0x7f000c0cf0b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f000c08ab20 tx=0x7efff402bf60 comp rx=0 tx=0).stop 2024-03-12T17:07:27.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.801+0000 7f0011800640 1 -- 172.21.15.52:0/2110518123 shutdown_connections 2024-03-12T17:07:27.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.801+0000 7f0011800640 1 --2- 172.21.15.52:0/2110518123 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f000c0cf5f0 0x7f000c0d0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:27.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.801+0000 7f0011800640 1 --2- 172.21.15.52:0/2110518123 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f000c160e90 0x7f000c0cf0b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:27.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.801+0000 7f0011800640 1 --2- 172.21.15.52:0/2110518123 >> v2:172.21.15.102:3300/0 conn(0x7f000c15c820 0x7f000c0cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:27.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.801+0000 7f0011800640 1 -- 172.21.15.52:0/2110518123 >> 172.21.15.52:0/2110518123 conn(0x7f000c06b050 msgr2=0x7f000c06b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:27.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.801+0000 7f0011800640 1 -- 172.21.15.52:0/2110518123 shutdown_connections 2024-03-12T17:07:27.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.801+0000 7f0011800640 1 -- 172.21.15.52:0/2110518123 wait complete. 2024-03-12T17:07:27.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.803+0000 7f0011800640 1 Processor -- start 2024-03-12T17:07:27.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.803+0000 7f0011800640 1 -- start start 2024-03-12T17:07:27.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.804+0000 7f0011800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f000c0cf5f0 0x7f000c1529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:27.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.804+0000 7f0011800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f000c15c820 0x7f000c0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:27.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.804+0000 7f0011800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f000c160e90 0x7f000c0b4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:27.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.804+0000 7f0011800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000c0b49b0 con 0x7f000c160e90 2024-03-12T17:07:27.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.804+0000 7f0011800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000c0b4bb0 con 0x7f000c15c820 2024-03-12T17:07:27.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.804+0000 7f0011800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000c0b4e90 con 0x7f000c0cf5f0 2024-03-12T17:07:27.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.805+0000 7f0010e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f000c160e90 0x7f000c0b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:27.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.805+0000 7f0010e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f000c160e90 0x7f000c0b4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51006/0 (socket says 172.21.15.52:51006) 2024-03-12T17:07:27.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.805+0000 7f0010e00640 1 -- 172.21.15.52:0/1705617063 learned_addr learned my addr 172.21.15.52:0/1705617063 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:27.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.805+0000 7f000b400640 1 --2- 172.21.15.52:0/1705617063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f000c15c820 0x7f000c0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:27.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.805+0000 7f000be00640 1 --2- 172.21.15.52:0/1705617063 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f000c0cf5f0 0x7f000c1529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:27.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.805+0000 7f0010e00640 1 -- 172.21.15.52:0/1705617063 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f000c0cf5f0 msgr2=0x7f000c1529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:27.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.805+0000 7f0010e00640 1 --2- 172.21.15.52:0/1705617063 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f000c0cf5f0 0x7f000c1529a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:27.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.805+0000 7f0010e00640 1 -- 172.21.15.52:0/1705617063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f000c15c820 msgr2=0x7f000c0b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:27.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.805+0000 7f0010e00640 1 --2- 172.21.15.52:0/1705617063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f000c15c820 0x7f000c0b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:27.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.805+0000 7f0010e00640 1 -- 172.21.15.52:0/1705617063 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff4009b30 con 0x7f000c160e90 2024-03-12T17:07:27.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.805+0000 7f000b400640 1 --2- 172.21.15.52:0/1705617063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f000c15c820 0x7f000c0b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:07:27.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.806+0000 7f0010e00640 1 --2- 172.21.15.52:0/1705617063 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f000c160e90 0x7f000c0b4470 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f000400d800 tx=0x7f000400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:27.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.806+0000 7f0003e00640 1 -- 172.21.15.52:0/1705617063 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000400b6c0 con 0x7f000c160e90 2024-03-12T17:07:27.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.806+0000 7f0003e00640 1 -- 172.21.15.52:0/1705617063 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000400b840 con 0x7f000c160e90 2024-03-12T17:07:27.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.806+0000 7f0003e00640 1 -- 172.21.15.52:0/1705617063 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000400bbf0 con 0x7f000c160e90 2024-03-12T17:07:27.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.806+0000 7f0011800640 1 -- 172.21.15.52:0/1705617063 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f000c15cc00 con 0x7f000c160e90 2024-03-12T17:07:27.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.806+0000 7f0011800640 1 -- 172.21.15.52:0/1705617063 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f000c15d1c0 con 0x7f000c160e90 2024-03-12T17:07:27.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.807+0000 7f0011800640 1 -- 172.21.15.52:0/1705617063 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f000c04e030 con 0x7f000c160e90 2024-03-12T17:07:27.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.808+0000 7f0003e00640 1 -- 172.21.15.52:0/1705617063 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f000400bd50 con 0x7f000c160e90 2024-03-12T17:07:27.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.809+0000 7f0003e00640 1 --2- 172.21.15.52:0/1705617063 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7effe806ac80 0x7effe806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:27.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.809+0000 7f0003e00640 1 -- 172.21.15.52:0/1705617063 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0004089b60 con 0x7f000c160e90 2024-03-12T17:07:27.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.810+0000 7f000be00640 1 --2- 172.21.15.52:0/1705617063 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7effe806ac80 0x7effe806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:27.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.811+0000 7f000be00640 1 --2- 172.21.15.52:0/1705617063 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7effe806ac80 0x7effe806d140 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7efffc006fd0 tx=0x7efffc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:27.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:27.813+0000 7f0003e00640 1 -- 172.21.15.52:0/1705617063 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f0004054a60 con 0x7f000c160e90 2024-03-12T17:07:28.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.011+0000 7f0011800640 1 -- 172.21.15.52:0/1705617063 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f000c0d15a0 con 0x7f000c160e90 2024-03-12T17:07:28.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.012+0000 7f0003e00640 1 -- 172.21.15.52:0/1705617063 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0004058080 con 0x7f000c160e90 2024-03-12T17:07:28.006 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-03-12T17:07:28.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.015+0000 7f0011800640 1 -- 172.21.15.52:0/1705617063 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7effe806ac80 msgr2=0x7effe806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:28.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.015+0000 7f0011800640 1 --2- 172.21.15.52:0/1705617063 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7effe806ac80 0x7effe806d140 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7efffc006fd0 tx=0x7efffc008040 comp rx=0 tx=0).stop 2024-03-12T17:07:28.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.015+0000 7f0011800640 1 -- 172.21.15.52:0/1705617063 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f000c160e90 msgr2=0x7f000c0b4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:28.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.015+0000 7f0011800640 1 --2- 172.21.15.52:0/1705617063 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f000c160e90 0x7f000c0b4470 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f000400d800 tx=0x7f000400dcc0 comp rx=0 tx=0).stop 2024-03-12T17:07:28.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.015+0000 7f0011800640 1 -- 172.21.15.52:0/1705617063 shutdown_connections 2024-03-12T17:07:28.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.015+0000 7f0011800640 1 --2- 172.21.15.52:0/1705617063 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7effe806ac80 0x7effe806d140 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:28.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.015+0000 7f0011800640 1 --2- 172.21.15.52:0/1705617063 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f000c160e90 0x7f000c0b4470 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:28.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.015+0000 7f0011800640 1 --2- 172.21.15.52:0/1705617063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f000c15c820 0x7f000c0b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:28.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.015+0000 7f0011800640 1 --2- 172.21.15.52:0/1705617063 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f000c0cf5f0 0x7f000c1529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:28.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.015+0000 7f0011800640 1 -- 172.21.15.52:0/1705617063 >> 172.21.15.52:0/1705617063 conn(0x7f000c06b050 msgr2=0x7f000c06b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:28.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.015+0000 7f0011800640 1 -- 172.21.15.52:0/1705617063 shutdown_connections 2024-03-12T17:07:28.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.015+0000 7f0011800640 1 -- 172.21.15.52:0/1705617063 wait complete. 2024-03-12T17:07:28.028 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-12T17:07:28.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.252+0000 7f1cb7a00640 1 Processor -- start 2024-03-12T17:07:28.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.253+0000 7f1cb7a00640 1 -- start start 2024-03-12T17:07:28.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.254+0000 7f1cb7a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1cb00d0d40 0x7f1cb00b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:28.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.254+0000 7f1cb7a00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cb00d20c0 con 0x7f1cb00cd740 2024-03-12T17:07:28.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.254+0000 7f1cb7a00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cb00b8940 con 0x7f1cb00b84b0 2024-03-12T17:07:28.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.254+0000 7f1cb7a00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb00d14b0 con 0x7f1cb00d0d40 2024-03-12T17:07:28.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.254+0000 7f1cb5000640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f1cb00cd740 0x7f1cb00cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:52316/0 (socket says 172.21.15.52:52316) 2024-03-12T17:07:28.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.255+0000 7f1cb5000640 1 -- 172.21.15.52:0/2329907221 learned_addr learned my addr 172.21.15.52:0/2329907221 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:28.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.255+0000 7f1cb5a00640 1 --2- 172.21.15.52:0/2329907221 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1cb00d0d40 0x7f1cb00b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:28.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.255+0000 7f1caf400640 1 -- 172.21.15.52:0/2329907221 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2737670982 0 0) 0x7f1cb00d20c0 con 0x7f1cb00cd740 2024-03-12T17:07:28.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.255+0000 7f1caf400640 1 -- 172.21.15.52:0/2329907221 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c94003520 con 0x7f1cb00cd740 2024-03-12T17:07:28.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.255+0000 7f1caf400640 1 -- 172.21.15.52:0/2329907221 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 760836198 0 0) 0x7f1c94003520 con 0x7f1cb00cd740 2024-03-12T17:07:28.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.255+0000 7f1caf400640 1 -- 172.21.15.52:0/2329907221 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1cb00d0d40 msgr2=0x7f1cb00b61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:28.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.255+0000 7f1caf400640 1 --2- 172.21.15.52:0/2329907221 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1cb00d0d40 0x7f1cb00b61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:28.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.256+0000 7f1caf400640 1 -- 172.21.15.52:0/2329907221 >> v1:172.21.15.102:6789/0 conn(0x7f1cb00b84b0 legacy=0x7f1cb00b5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:28.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.256+0000 7f1caf400640 1 -- 172.21.15.52:0/2329907221 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb00d1790 con 0x7f1cb00cd740 2024-03-12T17:07:28.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.256+0000 7f1caf400640 1 -- 172.21.15.52:0/2329907221 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f1ca0001670 con 0x7f1cb00cd740 2024-03-12T17:07:28.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.256+0000 7f1caf400640 1 -- 172.21.15.52:0/2329907221 >> v1:172.21.15.52:6789/0 conn(0x7f1cb00cd740 legacy=0x7f1cb00cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:28.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.256+0000 7f1caf400640 1 --2- 172.21.15.52:0/2329907221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c940039a0 0x7f1c94003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:28.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.256+0000 7f1caf400640 1 --2- 172.21.15.52:0/2329907221 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1cb00d0d40 0x7f1c940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:28.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.256+0000 7f1caf400640 1 --2- 172.21.15.52:0/2329907221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c94004d00 0x7f1c94005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:28.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.256+0000 7f1caf400640 1 -- 172.21.15.52:0/2329907221 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb00d14b0 con 0x7f1c940039a0 2024-03-12T17:07:28.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.256+0000 7f1caf400640 1 -- 172.21.15.52:0/2329907221 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9400b9b0 con 0x7f1c94004d00 2024-03-12T17:07:28.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.256+0000 7f1caf400640 1 -- 172.21.15.52:0/2329907221 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9400bb70 con 0x7f1cb00d0d40 2024-03-12T17:07:28.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.256+0000 7f1cb5000640 1 --2- 172.21.15.52:0/2329907221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c940039a0 0x7f1c94003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:28.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.256+0000 7f1cb5000640 1 -- 172.21.15.52:0/2329907221 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1cb00d0d40 msgr2=0x7f1c940047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:28.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.256+0000 7f1cb5000640 1 --2- 172.21.15.52:0/2329907221 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1cb00d0d40 0x7f1c940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:28.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.256+0000 7f1cb5000640 1 -- 172.21.15.52:0/2329907221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c94004d00 msgr2=0x7f1c94005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:28.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.256+0000 7f1cb5000640 1 --2- 172.21.15.52:0/2329907221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c94004d00 0x7f1c94005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:28.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.256+0000 7f1cb5000640 1 -- 172.21.15.52:0/2329907221 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c94003520 con 0x7f1c940039a0 2024-03-12T17:07:28.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.257+0000 7f1cb5000640 1 --2- 172.21.15.52:0/2329907221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c940039a0 0x7f1c94003d80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000c210 tx=0x7f1c940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8405c8851f4a16d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:28.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.258+0000 7f1caf400640 1 -- 172.21.15.52:0/2329907221 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0005ad0 con 0x7f1c940039a0 2024-03-12T17:07:28.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.258+0000 7f1caf400640 1 -- 172.21.15.52:0/2329907221 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca0005c50 con 0x7f1c940039a0 2024-03-12T17:07:28.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.258+0000 7f1caf400640 1 -- 172.21.15.52:0/2329907221 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca002c830 con 0x7f1c940039a0 2024-03-12T17:07:28.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.258+0000 7f1cb7a00640 1 -- 172.21.15.52:0/2329907221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c940039a0 msgr2=0x7f1c94003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:28.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.258+0000 7f1cb7a00640 1 --2- 172.21.15.52:0/2329907221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c940039a0 0x7f1c94003d80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000c210 tx=0x7f1c940057f0 comp rx=0 tx=0).stop 2024-03-12T17:07:28.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.258+0000 7f1cb5a00640 1 -- 172.21.15.52:0/2329907221 reap_dead start 2024-03-12T17:07:28.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.258+0000 7f1cb7a00640 1 -- 172.21.15.52:0/2329907221 shutdown_connections 2024-03-12T17:07:28.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.258+0000 7f1cb7a00640 1 -- 172.21.15.52:0/2329907221 >> 172.21.15.52:0/2329907221 conn(0x7f1cb00cf5d0 msgr2=0x7f1cb00cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:28.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.259+0000 7f1cb7a00640 1 -- 172.21.15.52:0/2329907221 shutdown_connections 2024-03-12T17:07:28.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.259+0000 7f1cb7a00640 1 -- 172.21.15.52:0/2329907221 wait complete. 2024-03-12T17:07:28.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.260+0000 7f1cb7a00640 1 Processor -- start 2024-03-12T17:07:28.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.260+0000 7f1cb7a00640 1 -- start start 2024-03-12T17:07:28.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.261+0000 7f1cb7a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c940039a0 0x7f1cb0152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:28.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.261+0000 7f1cb7a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1cb00cc040 0x7f1cb0156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:28.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.261+0000 7f1cb7a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1cb00b5a60 0x7f1cb00b5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:28.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.261+0000 7f1cb7a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb00b4fe0 con 0x7f1cb00b5a60 2024-03-12T17:07:28.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.261+0000 7f1cb7a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb0153430 con 0x7f1cb00cc040 2024-03-12T17:07:28.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.261+0000 7f1cb7a00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb00b6420 con 0x7f1c940039a0 2024-03-12T17:07:28.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.262+0000 7f1cb5a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1cb00b5a60 0x7f1cb00b5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:28.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.262+0000 7f1cb5a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1cb00b5a60 0x7f1cb00b5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51026/0 (socket says 172.21.15.52:51026) 2024-03-12T17:07:28.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.262+0000 7f1cb5a00640 1 -- 172.21.15.52:0/303704702 learned_addr learned my addr 172.21.15.52:0/303704702 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:28.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.262+0000 7f1cb5000640 1 --2- 172.21.15.52:0/303704702 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c940039a0 0x7f1cb0152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:28.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.262+0000 7f1cafe00640 1 --2- 172.21.15.52:0/303704702 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1cb00cc040 0x7f1cb0156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:28.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.262+0000 7f1cb5a00640 1 -- 172.21.15.52:0/303704702 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c940039a0 msgr2=0x7f1cb0152ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:28.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.262+0000 7f1cb5a00640 1 --2- 172.21.15.52:0/303704702 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c940039a0 0x7f1cb0152ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:28.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.262+0000 7f1cb5a00640 1 -- 172.21.15.52:0/303704702 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1cb00cc040 msgr2=0x7f1cb0156330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:28.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.262+0000 7f1cb5a00640 1 --2- 172.21.15.52:0/303704702 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1cb00cc040 0x7f1cb0156330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:28.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.262+0000 7f1cb5a00640 1 -- 172.21.15.52:0/303704702 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb0156870 con 0x7f1cb00b5a60 2024-03-12T17:07:28.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.263+0000 7f1cb5a00640 1 --2- 172.21.15.52:0/303704702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1cb00b5a60 0x7f1cb00b5ee0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00d1120 tx=0x7f1ca800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:28.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.263+0000 7f1cacc00640 1 -- 172.21.15.52:0/303704702 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8010040 con 0x7f1cb00b5a60 2024-03-12T17:07:28.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.263+0000 7f1cacc00640 1 -- 172.21.15.52:0/303704702 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca800bce0 con 0x7f1cb00b5a60 2024-03-12T17:07:28.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.263+0000 7f1cb7a00640 1 -- 172.21.15.52:0/303704702 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cb00b4460 con 0x7f1cb00b5a60 2024-03-12T17:07:28.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.263+0000 7f1cacc00640 1 -- 172.21.15.52:0/303704702 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8014640 con 0x7f1cb00b5a60 2024-03-12T17:07:28.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.263+0000 7f1cb7a00640 1 -- 172.21.15.52:0/303704702 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb00b4a20 con 0x7f1cb00b5a60 2024-03-12T17:07:28.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.264+0000 7f1cb7a00640 1 -- 172.21.15.52:0/303704702 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cb00c9310 con 0x7f1cb00b5a60 2024-03-12T17:07:28.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.265+0000 7f1cacc00640 1 -- 172.21.15.52:0/303704702 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1ca80147a0 con 0x7f1cb00b5a60 2024-03-12T17:07:28.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.266+0000 7f1cacc00640 1 --2- 172.21.15.52:0/303704702 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1c9006ac30 0x7f1c9006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:28.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.266+0000 7f1cacc00640 1 -- 172.21.15.52:0/303704702 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1ca8089a20 con 0x7f1cb00b5a60 2024-03-12T17:07:28.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.267+0000 7f1cb5000640 1 --2- 172.21.15.52:0/303704702 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1c9006ac30 0x7f1c9006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:28.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.268+0000 7f1cb5000640 1 --2- 172.21.15.52:0/303704702 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1c9006ac30 0x7f1c9006d0f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0034000 tx=0x7f1ca0009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:28.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.271+0000 7f1cacc00640 1 -- 172.21.15.52:0/303704702 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f1ca80547a0 con 0x7f1cb00b5a60 2024-03-12T17:07:28.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.472+0000 7f1cb7a00640 1 -- 172.21.15.52:0/303704702 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1cb0073390 con 0x7f1cb00b5a60 2024-03-12T17:07:28.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.474+0000 7f1cacc00640 1 -- 172.21.15.52:0/303704702 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1ca8057dc0 con 0x7f1cb00b5a60 2024-03-12T17:07:28.468 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-12T17:07:28.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.476+0000 7f1cb7a00640 1 -- 172.21.15.52:0/303704702 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1c9006ac30 msgr2=0x7f1c9006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:28.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.476+0000 7f1cb7a00640 1 --2- 172.21.15.52:0/303704702 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1c9006ac30 0x7f1c9006d0f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0034000 tx=0x7f1ca0009030 comp rx=0 tx=0).stop 2024-03-12T17:07:28.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.476+0000 7f1cb7a00640 1 -- 172.21.15.52:0/303704702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1cb00b5a60 msgr2=0x7f1cb00b5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:28.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.476+0000 7f1cb7a00640 1 --2- 172.21.15.52:0/303704702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1cb00b5a60 0x7f1cb00b5ee0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00d1120 tx=0x7f1ca800b480 comp rx=0 tx=0).stop 2024-03-12T17:07:28.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.477+0000 7f1cb7a00640 1 -- 172.21.15.52:0/303704702 shutdown_connections 2024-03-12T17:07:28.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.477+0000 7f1cb7a00640 1 --2- 172.21.15.52:0/303704702 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1c9006ac30 0x7f1c9006d0f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:28.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.477+0000 7f1cb7a00640 1 --2- 172.21.15.52:0/303704702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1cb00b5a60 0x7f1cb00b5ee0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:28.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.477+0000 7f1cb7a00640 1 --2- 172.21.15.52:0/303704702 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1cb00cc040 0x7f1cb0156330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:28.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.477+0000 7f1cb7a00640 1 --2- 172.21.15.52:0/303704702 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c940039a0 0x7f1cb0152ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:28.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.477+0000 7f1cb7a00640 1 -- 172.21.15.52:0/303704702 >> 172.21.15.52:0/303704702 conn(0x7f1cb00cf5d0 msgr2=0x7f1cb015f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:28.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.477+0000 7f1cb7a00640 1 -- 172.21.15.52:0/303704702 shutdown_connections 2024-03-12T17:07:28.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.477+0000 7f1cb7a00640 1 -- 172.21.15.52:0/303704702 wait complete. 2024-03-12T17:07:28.489 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-12T17:07:28.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.719+0000 7f4a5ca00640 1 Processor -- start 2024-03-12T17:07:28.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.720+0000 7f4a5ca00640 1 -- start start 2024-03-12T17:07:28.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.721+0000 7f4a5ca00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4a5815a850 0x7f4a580cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:28.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.721+0000 7f4a5ca00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4a580b22b0 0x7f4a5815eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:28.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.721+0000 7f4a5ca00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a5815e320 con 0x7f4a5815f020 2024-03-12T17:07:28.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.721+0000 7f4a5ca00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a5815ac30 con 0x7f4a580b22b0 2024-03-12T17:07:28.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.721+0000 7f4a5ca00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a580cda20 con 0x7f4a5815a850 2024-03-12T17:07:28.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.722+0000 7f4a55e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4a580b22b0 0x7f4a5815eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:28.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.722+0000 7f4a55e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4a580b22b0 0x7f4a5815eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51032/0 (socket says 172.21.15.52:51032) 2024-03-12T17:07:28.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.722+0000 7f4a55e00640 1 -- 172.21.15.52:0/4163798491 learned_addr learned my addr 172.21.15.52:0/4163798491 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:28.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.722+0000 7f4a56800640 1 --2- 172.21.15.52:0/4163798491 >> v2:172.21.15.102:3300/0 conn(0x7f4a5815a850 0x7f4a580cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:28.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.722+0000 7f4a55e00640 1 -- 172.21.15.52:0/4163798491 >> v2:172.21.15.102:3300/0 conn(0x7f4a5815a850 msgr2=0x7f4a580cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:28.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.722+0000 7f4a55e00640 1 --2- 172.21.15.52:0/4163798491 >> v2:172.21.15.102:3300/0 conn(0x7f4a5815a850 0x7f4a580cd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:28.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.722+0000 7f4a55e00640 1 -- 172.21.15.52:0/4163798491 >> v1:172.21.15.52:6789/0 conn(0x7f4a5815f020 legacy=0x7f4a580cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:28.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.722+0000 7f4a55400640 1 -- 172.21.15.52:0/4163798491 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1413699458 0 0) 0x7f4a5815e320 con 0x7f4a5815f020 2024-03-12T17:07:28.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.722+0000 7f4a55e00640 1 -- 172.21.15.52:0/4163798491 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a580cdd00 con 0x7f4a580b22b0 2024-03-12T17:07:28.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.723+0000 7f4a55e00640 1 --2- 172.21.15.52:0/4163798491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a580b22b0 0x7f4a5815eae0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4a5806adb0 tx=0x7f4a4002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e36279899eac399f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:28.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.724+0000 7f4a55400640 1 -- 172.21.15.52:0/4163798491 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a40030860 con 0x7f4a580b22b0 2024-03-12T17:07:28.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.724+0000 7f4a55400640 1 -- 172.21.15.52:0/4163798491 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a400309c0 con 0x7f4a580b22b0 2024-03-12T17:07:28.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.724+0000 7f4a55400640 1 -- 172.21.15.52:0/4163798491 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a40030d50 con 0x7f4a580b22b0 2024-03-12T17:07:28.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.724+0000 7f4a5ca00640 1 -- 172.21.15.52:0/4163798491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a580b22b0 msgr2=0x7f4a5815eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:28.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.724+0000 7f4a5ca00640 1 --2- 172.21.15.52:0/4163798491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a580b22b0 0x7f4a5815eae0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4a5806adb0 tx=0x7f4a4002b9a0 comp rx=0 tx=0).stop 2024-03-12T17:07:28.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.724+0000 7f4a5ca00640 1 -- 172.21.15.52:0/4163798491 shutdown_connections 2024-03-12T17:07:28.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.724+0000 7f4a5ca00640 1 --2- 172.21.15.52:0/4163798491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a580b22b0 0x7f4a5815eae0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:28.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.724+0000 7f4a5ca00640 1 --2- 172.21.15.52:0/4163798491 >> v2:172.21.15.102:3300/0 conn(0x7f4a5815a850 0x7f4a580cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:28.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.724+0000 7f4a5ca00640 1 -- 172.21.15.52:0/4163798491 >> 172.21.15.52:0/4163798491 conn(0x7f4a580b01b0 msgr2=0x7f4a580b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:28.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.724+0000 7f4a5ca00640 1 -- 172.21.15.52:0/4163798491 shutdown_connections 2024-03-12T17:07:28.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.725+0000 7f4a5ca00640 1 -- 172.21.15.52:0/4163798491 wait complete. 2024-03-12T17:07:28.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.726+0000 7f4a5ca00640 1 Processor -- start 2024-03-12T17:07:28.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.726+0000 7f4a5ca00640 1 -- start start 2024-03-12T17:07:28.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.727+0000 7f4a5ca00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a580b22b0 0x7f4a580ac5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:28.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.727+0000 7f4a5ca00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a5815a850 0x7f4a580ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:28.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.727+0000 7f4a5ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a5815f020 0x7f4a58156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:28.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.727+0000 7f4a5ca00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a580caa00 con 0x7f4a5815a850 2024-03-12T17:07:28.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.727+0000 7f4a5ca00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a580c7650 con 0x7f4a5815f020 2024-03-12T17:07:28.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.727+0000 7f4a5ca00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a580c7bd0 con 0x7f4a580b22b0 2024-03-12T17:07:28.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.728+0000 7f4a55e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a5815a850 0x7f4a580ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:28.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.728+0000 7f4a55e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a5815a850 0x7f4a580ca4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51042/0 (socket says 172.21.15.52:51042) 2024-03-12T17:07:28.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.728+0000 7f4a55e00640 1 -- 172.21.15.52:0/28278778 learned_addr learned my addr 172.21.15.52:0/28278778 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:28.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.728+0000 7f4a56800640 1 --2- 172.21.15.52:0/28278778 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a580b22b0 0x7f4a580ac5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:28.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.728+0000 7f4a57200640 1 --2- 172.21.15.52:0/28278778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a5815f020 0x7f4a58156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:28.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.728+0000 7f4a55e00640 1 -- 172.21.15.52:0/28278778 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a580b22b0 msgr2=0x7f4a580ac5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:28.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.728+0000 7f4a55e00640 1 --2- 172.21.15.52:0/28278778 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a580b22b0 0x7f4a580ac5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:28.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.728+0000 7f4a55e00640 1 -- 172.21.15.52:0/28278778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a5815f020 msgr2=0x7f4a58156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:28.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.728+0000 7f4a55e00640 1 --2- 172.21.15.52:0/28278778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a5815f020 0x7f4a58156ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:28.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.728+0000 7f4a55e00640 1 -- 172.21.15.52:0/28278778 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a400094d0 con 0x7f4a5815a850 2024-03-12T17:07:28.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.728+0000 7f4a56800640 1 --2- 172.21.15.52:0/28278778 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a580b22b0 0x7f4a580ac5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:28.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.728+0000 7f4a57200640 1 --2- 172.21.15.52:0/28278778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a5815f020 0x7f4a58156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:28.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.729+0000 7f4a55e00640 1 --2- 172.21.15.52:0/28278778 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a5815a850 0x7f4a580ca4c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4a40000c00 tx=0x7f4a40004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:28.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.729+0000 7f4a4ea00640 1 -- 172.21.15.52:0/28278778 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a40039d60 con 0x7f4a5815a850 2024-03-12T17:07:28.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.729+0000 7f4a4ea00640 1 -- 172.21.15.52:0/28278778 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a40043400 con 0x7f4a5815a850 2024-03-12T17:07:28.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.729+0000 7f4a4ea00640 1 -- 172.21.15.52:0/28278778 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a400437b0 con 0x7f4a5815a850 2024-03-12T17:07:28.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.729+0000 7f4a5ca00640 1 -- 172.21.15.52:0/28278778 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a580accd0 con 0x7f4a5815a850 2024-03-12T17:07:28.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.729+0000 7f4a5ca00640 1 -- 172.21.15.52:0/28278778 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a580c7860 con 0x7f4a5815a850 2024-03-12T17:07:28.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.730+0000 7f4a5ca00640 1 -- 172.21.15.52:0/28278778 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a580c79f0 con 0x7f4a5815a850 2024-03-12T17:07:28.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.731+0000 7f4a4ea00640 1 -- 172.21.15.52:0/28278778 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4a40043910 con 0x7f4a5815a850 2024-03-12T17:07:28.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.732+0000 7f4a4ea00640 1 --2- 172.21.15.52:0/28278778 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4a3406ac00 0x7f4a3406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:28.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.732+0000 7f4a4ea00640 1 -- 172.21.15.52:0/28278778 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4a4003e070 con 0x7f4a5815a850 2024-03-12T17:07:28.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.733+0000 7f4a56800640 1 --2- 172.21.15.52:0/28278778 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4a3406ac00 0x7f4a3406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:28.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.734+0000 7f4a56800640 1 --2- 172.21.15.52:0/28278778 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4a3406ac00 0x7f4a3406d0c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4a5815e280 tx=0x7f4a4800b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:28.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.737+0000 7f4a4ea00640 1 -- 172.21.15.52:0/28278778 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f4a40077750 con 0x7f4a5815a850 2024-03-12T17:07:28.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.936+0000 7f4a5ca00640 1 -- 172.21.15.52:0/28278778 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4a580c7d40 con 0x7f4a5815a850 2024-03-12T17:07:28.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.937+0000 7f4a4ea00640 1 -- 172.21.15.52:0/28278778 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4a4007ad70 con 0x7f4a5815a850 2024-03-12T17:07:28.931 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-03-12T17:07:28.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.940+0000 7f4a5ca00640 1 -- 172.21.15.52:0/28278778 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4a3406ac00 msgr2=0x7f4a3406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:28.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.940+0000 7f4a5ca00640 1 --2- 172.21.15.52:0/28278778 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4a3406ac00 0x7f4a3406d0c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4a5815e280 tx=0x7f4a4800b040 comp rx=0 tx=0).stop 2024-03-12T17:07:28.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.940+0000 7f4a5ca00640 1 -- 172.21.15.52:0/28278778 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a5815a850 msgr2=0x7f4a580ca4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:28.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.940+0000 7f4a5ca00640 1 --2- 172.21.15.52:0/28278778 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a5815a850 0x7f4a580ca4c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4a40000c00 tx=0x7f4a40004730 comp rx=0 tx=0).stop 2024-03-12T17:07:28.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.940+0000 7f4a5ca00640 1 -- 172.21.15.52:0/28278778 shutdown_connections 2024-03-12T17:07:28.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.940+0000 7f4a5ca00640 1 --2- 172.21.15.52:0/28278778 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4a3406ac00 0x7f4a3406d0c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:28.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.940+0000 7f4a5ca00640 1 --2- 172.21.15.52:0/28278778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a5815f020 0x7f4a58156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:28.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.940+0000 7f4a5ca00640 1 --2- 172.21.15.52:0/28278778 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a5815a850 0x7f4a580ca4c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:28.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.940+0000 7f4a5ca00640 1 --2- 172.21.15.52:0/28278778 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a580b22b0 0x7f4a580ac5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:28.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.940+0000 7f4a5ca00640 1 -- 172.21.15.52:0/28278778 >> 172.21.15.52:0/28278778 conn(0x7f4a580b01b0 msgr2=0x7f4a580b1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:28.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.940+0000 7f4a5ca00640 1 -- 172.21.15.52:0/28278778 shutdown_connections 2024-03-12T17:07:28.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:28.940+0000 7f4a5ca00640 1 -- 172.21.15.52:0/28278778 wait complete. 2024-03-12T17:07:28.954 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-12T17:07:29.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.190+0000 7fde82400640 1 Processor -- start 2024-03-12T17:07:29.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.190+0000 7fde82400640 1 -- start start 2024-03-12T17:07:29.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.191+0000 7fde82400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fde7c0cf5c0 0x7fde7c0b8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:29.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.191+0000 7fde82400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fde7c0d0c20 0x7fde7c0d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:29.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.191+0000 7fde82400640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde7c15ce90 con 0x7fde7c15cab0 2024-03-12T17:07:29.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.191+0000 7fde82400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde7c0cddf0 con 0x7fde7c0d0c20 2024-03-12T17:07:29.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.191+0000 7fde82400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde7c0cd570 con 0x7fde7c0cf5c0 2024-03-12T17:07:29.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.192+0000 7fde81000640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fde7c15cab0 0x7fde7c0ceeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:52336/0 (socket says 172.21.15.52:52336) 2024-03-12T17:07:29.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.192+0000 7fde81000640 1 -- 172.21.15.52:0/3893422184 learned_addr learned my addr 172.21.15.52:0/3893422184 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:29.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.192+0000 7fde81a00640 1 --2- 172.21.15.52:0/3893422184 >> v2:172.21.15.102:3300/0 conn(0x7fde7c0d0c20 0x7fde7c0d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:29.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.192+0000 7fde73e00640 1 --2- 172.21.15.52:0/3893422184 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fde7c0cf5c0 0x7fde7c0b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:29.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.192+0000 7fde7be00640 1 -- 172.21.15.52:0/3893422184 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2249000848 0 0) 0x7fde7c15ce90 con 0x7fde7c15cab0 2024-03-12T17:07:29.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.192+0000 7fde7be00640 1 -- 172.21.15.52:0/3893422184 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde60003520 con 0x7fde7c15cab0 2024-03-12T17:07:29.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.192+0000 7fde7be00640 1 -- 172.21.15.52:0/3893422184 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2497194831 0 0) 0x7fde60003520 con 0x7fde7c15cab0 2024-03-12T17:07:29.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.193+0000 7fde7be00640 1 -- 172.21.15.52:0/3893422184 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fde7c0cf5c0 msgr2=0x7fde7c0b8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:29.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.193+0000 7fde7be00640 1 --2- 172.21.15.52:0/3893422184 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fde7c0cf5c0 0x7fde7c0b8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:29.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.193+0000 7fde7be00640 1 -- 172.21.15.52:0/3893422184 >> v2:172.21.15.102:3300/0 conn(0x7fde7c0d0c20 msgr2=0x7fde7c0d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:29.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.193+0000 7fde7be00640 1 --2- 172.21.15.52:0/3893422184 >> v2:172.21.15.102:3300/0 conn(0x7fde7c0d0c20 0x7fde7c0d10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:29.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.193+0000 7fde7be00640 1 -- 172.21.15.52:0/3893422184 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde7c161370 con 0x7fde7c15cab0 2024-03-12T17:07:29.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.193+0000 7fde7be00640 1 -- 172.21.15.52:0/3893422184 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7fde6c001670 con 0x7fde7c15cab0 2024-03-12T17:07:29.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.193+0000 7fde7be00640 1 -- 172.21.15.52:0/3893422184 >> v1:172.21.15.52:6789/0 conn(0x7fde7c15cab0 legacy=0x7fde7c0ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:29.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.193+0000 7fde7be00640 1 --2- 172.21.15.52:0/3893422184 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fde7c0cf5c0 0x7fde60003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:29.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.193+0000 7fde7be00640 1 --2- 172.21.15.52:0/3893422184 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde60003ef0 0x7fde600047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:29.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.193+0000 7fde7be00640 1 --2- 172.21.15.52:0/3893422184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde60004d00 0x7fde60005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:29.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.193+0000 7fde7be00640 1 -- 172.21.15.52:0/3893422184 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7c0cddf0 con 0x7fde60003ef0 2024-03-12T17:07:29.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.193+0000 7fde7be00640 1 -- 172.21.15.52:0/3893422184 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7c0cd570 con 0x7fde60004d00 2024-03-12T17:07:29.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.193+0000 7fde7be00640 1 -- 172.21.15.52:0/3893422184 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde6000b980 con 0x7fde7c0cf5c0 2024-03-12T17:07:29.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.193+0000 7fde73e00640 1 --2- 172.21.15.52:0/3893422184 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde60003ef0 0x7fde600047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:29.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.193+0000 7fde81000640 1 --2- 172.21.15.52:0/3893422184 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fde7c0cf5c0 0x7fde60003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:29.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.193+0000 7fde81a00640 1 --2- 172.21.15.52:0/3893422184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde60004d00 0x7fde60005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:29.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.194+0000 7fde73e00640 1 -- 172.21.15.52:0/3893422184 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fde7c0cf5c0 msgr2=0x7fde60003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:29.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.194+0000 7fde73e00640 1 --2- 172.21.15.52:0/3893422184 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fde7c0cf5c0 0x7fde60003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:29.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.194+0000 7fde73e00640 1 -- 172.21.15.52:0/3893422184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde60004d00 msgr2=0x7fde60005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:29.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.194+0000 7fde73e00640 1 --2- 172.21.15.52:0/3893422184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde60004d00 0x7fde60005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:29.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.194+0000 7fde73e00640 1 -- 172.21.15.52:0/3893422184 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde60003520 con 0x7fde60003ef0 2024-03-12T17:07:29.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.195+0000 7fde73e00640 1 --2- 172.21.15.52:0/3893422184 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde60003ef0 0x7fde600047c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fde64009ac0 tx=0x7fde600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63aabd06ae16a5db server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:29.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.195+0000 7fde7be00640 1 -- 172.21.15.52:0/3893422184 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6402c6b0 con 0x7fde60003ef0 2024-03-12T17:07:29.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.195+0000 7fde7be00640 1 -- 172.21.15.52:0/3893422184 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde6402c810 con 0x7fde60003ef0 2024-03-12T17:07:29.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.195+0000 7fde7be00640 1 -- 172.21.15.52:0/3893422184 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6402cba0 con 0x7fde60003ef0 2024-03-12T17:07:29.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.195+0000 7fde82400640 1 -- 172.21.15.52:0/3893422184 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde60003ef0 msgr2=0x7fde600047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:29.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.195+0000 7fde82400640 1 --2- 172.21.15.52:0/3893422184 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde60003ef0 0x7fde600047c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fde64009ac0 tx=0x7fde600057f0 comp rx=0 tx=0).stop 2024-03-12T17:07:29.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.196+0000 7fde81a00640 1 -- 172.21.15.52:0/3893422184 reap_dead start 2024-03-12T17:07:29.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.196+0000 7fde82400640 1 -- 172.21.15.52:0/3893422184 shutdown_connections 2024-03-12T17:07:29.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.196+0000 7fde82400640 1 -- 172.21.15.52:0/3893422184 >> 172.21.15.52:0/3893422184 conn(0x7fde7c06ae90 msgr2=0x7fde7c06b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:29.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.196+0000 7fde82400640 1 -- 172.21.15.52:0/3893422184 shutdown_connections 2024-03-12T17:07:29.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.196+0000 7fde82400640 1 -- 172.21.15.52:0/3893422184 wait complete. 2024-03-12T17:07:29.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.198+0000 7fde82400640 1 Processor -- start 2024-03-12T17:07:29.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.198+0000 7fde82400640 1 -- start start 2024-03-12T17:07:29.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.199+0000 7fde82400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde60003ef0 0x7fde7c1f6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:29.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.199+0000 7fde82400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde7c15f4f0 0x7fde7c0b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:29.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.199+0000 7fde82400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fde7c0c8540 0x7fde7c0c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:29.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.199+0000 7fde82400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7c1f66c0 con 0x7fde60003ef0 2024-03-12T17:07:29.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.199+0000 7fde82400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7c15e450 con 0x7fde7c15f4f0 2024-03-12T17:07:29.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.199+0000 7fde82400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde7c15e730 con 0x7fde7c0c8540 2024-03-12T17:07:29.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.199+0000 7fde81000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde60003ef0 0x7fde7c1f6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:29.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.199+0000 7fde81000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde60003ef0 0x7fde7c1f6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51068/0 (socket says 172.21.15.52:51068) 2024-03-12T17:07:29.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.199+0000 7fde81000640 1 -- 172.21.15.52:0/3683677659 learned_addr learned my addr 172.21.15.52:0/3683677659 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:29.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.199+0000 7fde81a00640 1 --2- 172.21.15.52:0/3683677659 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fde7c0c8540 0x7fde7c0c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:29.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.199+0000 7fde73e00640 1 --2- 172.21.15.52:0/3683677659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde7c15f4f0 0x7fde7c0b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:29.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.199+0000 7fde81000640 1 -- 172.21.15.52:0/3683677659 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fde7c0c8540 msgr2=0x7fde7c0c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:29.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.199+0000 7fde81000640 1 --2- 172.21.15.52:0/3683677659 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fde7c0c8540 0x7fde7c0c89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:29.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.199+0000 7fde81000640 1 -- 172.21.15.52:0/3683677659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde7c15f4f0 msgr2=0x7fde7c0b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:29.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.199+0000 7fde81000640 1 --2- 172.21.15.52:0/3683677659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde7c15f4f0 0x7fde7c0b3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:29.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.199+0000 7fde81000640 1 -- 172.21.15.52:0/3683677659 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde7c15ea10 con 0x7fde60003ef0 2024-03-12T17:07:29.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.199+0000 7fde73e00640 1 --2- 172.21.15.52:0/3683677659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde7c15f4f0 0x7fde7c0b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:29.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.200+0000 7fde81000640 1 --2- 172.21.15.52:0/3683677659 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde60003ef0 0x7fde7c1f6180 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fde6c00e820 tx=0x7fde6c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:29.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.200+0000 7fde79600640 1 -- 172.21.15.52:0/3683677659 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6c00c870 con 0x7fde60003ef0 2024-03-12T17:07:29.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.200+0000 7fde79600640 1 -- 172.21.15.52:0/3683677659 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde6c00c9f0 con 0x7fde60003ef0 2024-03-12T17:07:29.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.200+0000 7fde79600640 1 -- 172.21.15.52:0/3683677659 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6c00cda0 con 0x7fde60003ef0 2024-03-12T17:07:29.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.200+0000 7fde82400640 1 -- 172.21.15.52:0/3683677659 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde7c15ecf0 con 0x7fde60003ef0 2024-03-12T17:07:29.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.200+0000 7fde82400640 1 -- 172.21.15.52:0/3683677659 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde7c0c8f00 con 0x7fde60003ef0 2024-03-12T17:07:29.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.202+0000 7fde82400640 1 -- 172.21.15.52:0/3683677659 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde7c04e030 con 0x7fde60003ef0 2024-03-12T17:07:29.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.202+0000 7fde79600640 1 -- 172.21.15.52:0/3683677659 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fde6c01d460 con 0x7fde60003ef0 2024-03-12T17:07:29.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.203+0000 7fde79600640 1 --2- 172.21.15.52:0/3683677659 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fde5806ac00 0x7fde5806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:29.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.204+0000 7fde79600640 1 -- 172.21.15.52:0/3683677659 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fde6c089770 con 0x7fde60003ef0 2024-03-12T17:07:29.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.204+0000 7fde73e00640 1 --2- 172.21.15.52:0/3683677659 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fde5806ac00 0x7fde5806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:29.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.205+0000 7fde73e00640 1 --2- 172.21.15.52:0/3683677659 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fde5806ac00 0x7fde5806d0c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fde64036000 tx=0x7fde64030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:29.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.208+0000 7fde79600640 1 -- 172.21.15.52:0/3683677659 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fde6c054580 con 0x7fde60003ef0 2024-03-12T17:07:29.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.404+0000 7fde82400640 1 -- 172.21.15.52:0/3683677659 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fde7c0d02d0 con 0x7fde60003ef0 2024-03-12T17:07:29.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.405+0000 7fde79600640 1 -- 172.21.15.52:0/3683677659 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fde6c057ba0 con 0x7fde60003ef0 2024-03-12T17:07:29.399 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-12T17:07:29.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.408+0000 7fde82400640 1 -- 172.21.15.52:0/3683677659 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fde5806ac00 msgr2=0x7fde5806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:29.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.408+0000 7fde82400640 1 --2- 172.21.15.52:0/3683677659 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fde5806ac00 0x7fde5806d0c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fde64036000 tx=0x7fde64030000 comp rx=0 tx=0).stop 2024-03-12T17:07:29.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.408+0000 7fde82400640 1 -- 172.21.15.52:0/3683677659 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde60003ef0 msgr2=0x7fde7c1f6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:29.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.408+0000 7fde82400640 1 --2- 172.21.15.52:0/3683677659 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde60003ef0 0x7fde7c1f6180 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fde6c00e820 tx=0x7fde6c00ece0 comp rx=0 tx=0).stop 2024-03-12T17:07:29.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.408+0000 7fde82400640 1 -- 172.21.15.52:0/3683677659 shutdown_connections 2024-03-12T17:07:29.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.408+0000 7fde82400640 1 --2- 172.21.15.52:0/3683677659 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fde5806ac00 0x7fde5806d0c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:29.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.408+0000 7fde82400640 1 --2- 172.21.15.52:0/3683677659 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fde7c0c8540 0x7fde7c0c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:29.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.408+0000 7fde82400640 1 --2- 172.21.15.52:0/3683677659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde7c15f4f0 0x7fde7c0b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:29.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.408+0000 7fde82400640 1 --2- 172.21.15.52:0/3683677659 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde60003ef0 0x7fde7c1f6180 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:29.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.408+0000 7fde82400640 1 -- 172.21.15.52:0/3683677659 >> 172.21.15.52:0/3683677659 conn(0x7fde7c06ae90 msgr2=0x7fde7c0b76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:29.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.408+0000 7fde82400640 1 -- 172.21.15.52:0/3683677659 shutdown_connections 2024-03-12T17:07:29.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.408+0000 7fde82400640 1 -- 172.21.15.52:0/3683677659 wait complete. 2024-03-12T17:07:29.423 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:07:29.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.641+0000 7f726fc00640 1 Processor -- start 2024-03-12T17:07:29.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.641+0000 7f726fc00640 1 -- start start 2024-03-12T17:07:29.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.642+0000 7f726fc00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f726815b560 0x7f726815b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:29.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.642+0000 7f726fc00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f72680b39b0 0x7f72680b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:29.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.642+0000 7f726fc00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f726806c6f0 con 0x7f726815e500 2024-03-12T17:07:29.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.642+0000 7f726fc00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72680b4370 con 0x7f72680b39b0 2024-03-12T17:07:29.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.642+0000 7f726fc00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f726806c9d0 con 0x7f726815b560 2024-03-12T17:07:29.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.643+0000 7f726dc00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f726815e500 0x7f726815e990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:52352/0 (socket says 172.21.15.52:52352) 2024-03-12T17:07:29.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.643+0000 7f726dc00640 1 -- 172.21.15.52:0/458826703 learned_addr learned my addr 172.21.15.52:0/458826703 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:29.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.643+0000 7f7267e00640 1 --2- 172.21.15.52:0/458826703 >> v2:172.21.15.102:3300/0 conn(0x7f72680b39b0 0x7f72680b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:29.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.643+0000 7f726d200640 1 --2- 172.21.15.52:0/458826703 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f726815b560 0x7f726815b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:29.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.643+0000 7f7267400640 1 -- 172.21.15.52:0/458826703 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2341466512 0 0) 0x7f726806c6f0 con 0x7f726815e500 2024-03-12T17:07:29.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.643+0000 7f7267400640 1 -- 172.21.15.52:0/458826703 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7250003520 con 0x7f726815e500 2024-03-12T17:07:29.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.644+0000 7f7267400640 1 -- 172.21.15.52:0/458826703 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3493596039 0 0) 0x7f7250003520 con 0x7f726815e500 2024-03-12T17:07:29.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.644+0000 7f7267400640 1 -- 172.21.15.52:0/458826703 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f726815b560 msgr2=0x7f726815b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:29.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.644+0000 7f7267400640 1 --2- 172.21.15.52:0/458826703 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f726815b560 0x7f726815b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:29.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.644+0000 7f7267400640 1 -- 172.21.15.52:0/458826703 >> v2:172.21.15.102:3300/0 conn(0x7f72680b39b0 msgr2=0x7f72680b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:29.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.644+0000 7f7267400640 1 --2- 172.21.15.52:0/458826703 >> v2:172.21.15.102:3300/0 conn(0x7f72680b39b0 0x7f72680b3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:29.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.644+0000 7f7267400640 1 -- 172.21.15.52:0/458826703 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f726806ccb0 con 0x7f726815e500 2024-03-12T17:07:29.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.644+0000 7f7267400640 1 -- 172.21.15.52:0/458826703 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f72600019f0 con 0x7f726815e500 2024-03-12T17:07:29.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.644+0000 7f7267400640 1 -- 172.21.15.52:0/458826703 >> v1:172.21.15.52:6789/0 conn(0x7f726815e500 legacy=0x7f726815e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:29.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.644+0000 7f7267400640 1 --2- 172.21.15.52:0/458826703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7250003970 0x7f7250003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:29.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.644+0000 7f7267400640 1 --2- 172.21.15.52:0/458826703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72500047c0 0x7f7250004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:29.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.644+0000 7f7267400640 1 --2- 172.21.15.52:0/458826703 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f726815b560 0x7f7250005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:29.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.644+0000 7f7267400640 1 -- 172.21.15.52:0/458826703 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72680b4370 con 0x7f7250003970 2024-03-12T17:07:29.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.644+0000 7f7267400640 1 -- 172.21.15.52:0/458826703 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726806c9d0 con 0x7f72500047c0 2024-03-12T17:07:29.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.644+0000 7f7267400640 1 -- 172.21.15.52:0/458826703 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f725000b980 con 0x7f726815b560 2024-03-12T17:07:29.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.644+0000 7f726d200640 1 --2- 172.21.15.52:0/458826703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7250003970 0x7f7250003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:29.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.645+0000 7f7267e00640 1 --2- 172.21.15.52:0/458826703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72500047c0 0x7f7250004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:29.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.645+0000 7f726d200640 1 -- 172.21.15.52:0/458826703 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f726815b560 msgr2=0x7f7250005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:29.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.645+0000 7f726dc00640 1 --2- 172.21.15.52:0/458826703 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f726815b560 0x7f7250005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:29.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.645+0000 7f726d200640 1 --2- 172.21.15.52:0/458826703 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f726815b560 0x7f7250005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:29.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.645+0000 7f726d200640 1 -- 172.21.15.52:0/458826703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72500047c0 msgr2=0x7f7250004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:29.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.645+0000 7f726d200640 1 --2- 172.21.15.52:0/458826703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72500047c0 0x7f7250004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:29.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.645+0000 7f726d200640 1 -- 172.21.15.52:0/458826703 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7250003520 con 0x7f7250003970 2024-03-12T17:07:29.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.645+0000 7f7267e00640 1 --2- 172.21.15.52:0/458826703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72500047c0 0x7f7250004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:29.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.645+0000 7f726dc00640 1 --2- 172.21.15.52:0/458826703 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f726815b560 0x7f7250005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:29.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.646+0000 7f726d200640 1 --2- 172.21.15.52:0/458826703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7250003970 0x7f7250003d90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f725c009ac0 tx=0x7f72500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f536238695d7e60a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:29.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.646+0000 7f7267400640 1 -- 172.21.15.52:0/458826703 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c02c6b0 con 0x7f7250003970 2024-03-12T17:07:29.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.646+0000 7f7267400640 1 -- 172.21.15.52:0/458826703 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725c02c810 con 0x7f7250003970 2024-03-12T17:07:29.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.646+0000 7f7267400640 1 -- 172.21.15.52:0/458826703 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c02cba0 con 0x7f7250003970 2024-03-12T17:07:29.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.647+0000 7f726fc00640 1 -- 172.21.15.52:0/458826703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7250003970 msgr2=0x7f7250003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:29.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.647+0000 7f726fc00640 1 --2- 172.21.15.52:0/458826703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7250003970 0x7f7250003d90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f725c009ac0 tx=0x7f72500057f0 comp rx=0 tx=0).stop 2024-03-12T17:07:29.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.647+0000 7f726dc00640 1 -- 172.21.15.52:0/458826703 reap_dead start 2024-03-12T17:07:29.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.647+0000 7f726fc00640 1 -- 172.21.15.52:0/458826703 shutdown_connections 2024-03-12T17:07:29.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.647+0000 7f726fc00640 1 -- 172.21.15.52:0/458826703 >> 172.21.15.52:0/458826703 conn(0x7f726806a3d0 msgr2=0x7f72680b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:29.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.647+0000 7f726fc00640 1 -- 172.21.15.52:0/458826703 shutdown_connections 2024-03-12T17:07:29.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.647+0000 7f726fc00640 1 -- 172.21.15.52:0/458826703 wait complete. 2024-03-12T17:07:29.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.649+0000 7f726fc00640 1 Processor -- start 2024-03-12T17:07:29.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.649+0000 7f726fc00640 1 -- start start 2024-03-12T17:07:29.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.650+0000 7f726fc00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7250003970 0x7f726813c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:29.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.650+0000 7f726fc00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f726813ea80 0x7f726815d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:29.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.650+0000 7f726fc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f726815dd80 0x7f72680cc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:29.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.650+0000 7f726fc00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72681f2850 con 0x7f7250003970 2024-03-12T17:07:29.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.650+0000 7f726fc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726815e200 con 0x7f726815dd80 2024-03-12T17:07:29.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.650+0000 7f726fc00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f726813ddb0 con 0x7f726813ea80 2024-03-12T17:07:29.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.650+0000 7f726d200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7250003970 0x7f726813c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:29.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.650+0000 7f726d200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7250003970 0x7f726813c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51086/0 (socket says 172.21.15.52:51086) 2024-03-12T17:07:29.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.650+0000 7f726d200640 1 -- 172.21.15.52:0/2091591547 learned_addr learned my addr 172.21.15.52:0/2091591547 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:29.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.650+0000 7f726d200640 1 -- 172.21.15.52:0/2091591547 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f726813ea80 msgr2=0x7f726815d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:29.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.650+0000 7f7267e00640 1 --2- 172.21.15.52:0/2091591547 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f726813ea80 0x7f726815d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:29.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.651+0000 7f726d200640 1 --2- 172.21.15.52:0/2091591547 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f726813ea80 0x7f726815d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:29.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.651+0000 7f726d200640 1 -- 172.21.15.52:0/2091591547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f726815dd80 msgr2=0x7f72680cc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:29.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.651+0000 7f726d200640 1 --2- 172.21.15.52:0/2091591547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f726815dd80 0x7f72680cc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:29.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.651+0000 7f726d200640 1 -- 172.21.15.52:0/2091591547 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f726813e5f0 con 0x7f7250003970 2024-03-12T17:07:29.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.651+0000 7f726d200640 1 --2- 172.21.15.52:0/2091591547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7250003970 0x7f726813c540 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f725c009ac0 tx=0x7f725c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:29.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.651+0000 7f7264c00640 1 -- 172.21.15.52:0/2091591547 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c003810 con 0x7f7250003970 2024-03-12T17:07:29.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.651+0000 7f7264c00640 1 -- 172.21.15.52:0/2091591547 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725c003970 con 0x7f7250003970 2024-03-12T17:07:29.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.651+0000 7f7264c00640 1 -- 172.21.15.52:0/2091591547 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c003d00 con 0x7f7250003970 2024-03-12T17:07:29.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.651+0000 7f726fc00640 1 -- 172.21.15.52:0/2091591547 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72680cd190 con 0x7f7250003970 2024-03-12T17:07:29.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.651+0000 7f726fc00640 1 -- 172.21.15.52:0/2091591547 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72680cceb0 con 0x7f7250003970 2024-03-12T17:07:29.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.652+0000 7f726fc00640 1 -- 172.21.15.52:0/2091591547 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f726806ac20 con 0x7f7250003970 2024-03-12T17:07:29.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.653+0000 7f7264c00640 1 -- 172.21.15.52:0/2091591547 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f725c040450 con 0x7f7250003970 2024-03-12T17:07:29.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.654+0000 7f7264c00640 1 --2- 172.21.15.52:0/2091591547 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f724c06ac00 0x7f724c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:29.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.655+0000 7f7264c00640 1 -- 172.21.15.52:0/2091591547 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f725c0ab970 con 0x7f7250003970 2024-03-12T17:07:29.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.655+0000 7f7267e00640 1 --2- 172.21.15.52:0/2091591547 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f724c06ac00 0x7f724c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:29.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.656+0000 7f7267e00640 1 --2- 172.21.15.52:0/2091591547 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f724c06ac00 0x7f724c06d0c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f72681f5e10 tx=0x7f7258008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:29.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.659+0000 7f7264c00640 1 -- 172.21.15.52:0/2091591547 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f725c076820 con 0x7f7250003970 2024-03-12T17:07:29.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.858+0000 7f726fc00640 1 -- 172.21.15.52:0/2091591547 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7268073390 con 0x7f7250003970 2024-03-12T17:07:29.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.861+0000 7f7264c00640 1 -- 172.21.15.52:0/2091591547 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18207 (secure 0 0 0) 0x7f725c0044e0 con 0x7f7250003970 2024-03-12T17:07:29.855 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:07:29.856 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":27,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:07:12.969614+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T17:06:19.218543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T17:06:23.525474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T17:06:24.911574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-12T17:06:28.017172+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.733031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.775178+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.591522+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.820838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.829252+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.582481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.18","mappings":[{"from":3,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:07:29.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.864+0000 7f726fc00640 1 -- 172.21.15.52:0/2091591547 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f724c06ac00 msgr2=0x7f724c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:29.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.864+0000 7f726fc00640 1 --2- 172.21.15.52:0/2091591547 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f724c06ac00 0x7f724c06d0c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f72681f5e10 tx=0x7f7258008040 comp rx=0 tx=0).stop 2024-03-12T17:07:29.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.864+0000 7f726fc00640 1 -- 172.21.15.52:0/2091591547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7250003970 msgr2=0x7f726813c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:29.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.864+0000 7f726fc00640 1 --2- 172.21.15.52:0/2091591547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7250003970 0x7f726813c540 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f725c009ac0 tx=0x7f725c004730 comp rx=0 tx=0).stop 2024-03-12T17:07:29.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.864+0000 7f726fc00640 1 -- 172.21.15.52:0/2091591547 shutdown_connections 2024-03-12T17:07:29.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.864+0000 7f726fc00640 1 --2- 172.21.15.52:0/2091591547 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f724c06ac00 0x7f724c06d0c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:29.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.864+0000 7f726fc00640 1 --2- 172.21.15.52:0/2091591547 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f726813ea80 0x7f726815d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:29.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.864+0000 7f726fc00640 1 --2- 172.21.15.52:0/2091591547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f726815dd80 0x7f72680cc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:29.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.864+0000 7f726fc00640 1 --2- 172.21.15.52:0/2091591547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7250003970 0x7f726813c540 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:29.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.864+0000 7f726fc00640 1 -- 172.21.15.52:0/2091591547 >> 172.21.15.52:0/2091591547 conn(0x7f726806a3d0 msgr2=0x7f72680b1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:29.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.864+0000 7f726fc00640 1 -- 172.21.15.52:0/2091591547 shutdown_connections 2024-03-12T17:07:29.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:29.864+0000 7f726fc00640 1 -- 172.21.15.52:0/2091591547 wait complete. 2024-03-12T17:07:29.875 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T17:06:19.218543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-12T17:06:23.525474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-12T17:06:24.911574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-12T17:06:28.017172+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-12T17:07:29.876 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T17:07:30.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.100+0000 7f4ec8a00640 1 Processor -- start 2024-03-12T17:07:30.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.100+0000 7f4ec8a00640 1 -- start start 2024-03-12T17:07:30.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.102+0000 7f4ec8a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4ec41456a0 0x7f4ec40ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:30.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.102+0000 7f4ec8a00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4ec4070cd0 0x7f4ec40cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:30.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.102+0000 7f4ec8a00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec406d390 con 0x7f4ec4145c20 2024-03-12T17:07:30.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.102+0000 7f4ec8a00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ec40ad890 con 0x7f4ec4070cd0 2024-03-12T17:07:30.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.102+0000 7f4ec8a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ec4145a80 con 0x7f4ec41456a0 2024-03-12T17:07:30.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.102+0000 7f4ec2800640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f4ec4145c20 0x7f4ec40ab8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:52362/0 (socket says 172.21.15.52:52362) 2024-03-12T17:07:30.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.102+0000 7f4ec2800640 1 -- 172.21.15.52:0/928875784 learned_addr learned my addr 172.21.15.52:0/928875784 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:30.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.102+0000 7f4ec1400640 1 --2- 172.21.15.52:0/928875784 >> v2:172.21.15.52:3300/0 conn(0x7f4ec4070cd0 0x7f4ec40cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:30.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.102+0000 7f4ec1e00640 1 --2- 172.21.15.52:0/928875784 >> v2:172.21.15.102:3300/0 conn(0x7f4ec41456a0 0x7f4ec40ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:30.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.103+0000 7f4ec1400640 1 -- 172.21.15.52:0/928875784 >> v2:172.21.15.102:3300/0 conn(0x7f4ec41456a0 msgr2=0x7f4ec40ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:30.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.103+0000 7f4ec1400640 1 --2- 172.21.15.52:0/928875784 >> v2:172.21.15.102:3300/0 conn(0x7f4ec41456a0 0x7f4ec40ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:30.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.103+0000 7f4ec1400640 1 -- 172.21.15.52:0/928875784 >> v1:172.21.15.52:6789/0 conn(0x7f4ec4145c20 legacy=0x7f4ec40ab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:30.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.103+0000 7f4ec1400640 1 -- 172.21.15.52:0/928875784 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec406d670 con 0x7f4ec4070cd0 2024-03-12T17:07:30.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.104+0000 7f4ec1400640 1 --2- 172.21.15.52:0/928875784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ec4070cd0 0x7f4ec40cee70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4ec40cf4f0 tx=0x7f4eb002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25806f2610013cc0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:30.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.104+0000 7f4ec0a00640 1 -- 172.21.15.52:0/928875784 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0031830 con 0x7f4ec4070cd0 2024-03-12T17:07:30.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.104+0000 7f4ec0a00640 1 -- 172.21.15.52:0/928875784 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb0031990 con 0x7f4ec4070cd0 2024-03-12T17:07:30.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.104+0000 7f4ec0a00640 1 -- 172.21.15.52:0/928875784 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0031d20 con 0x7f4ec4070cd0 2024-03-12T17:07:30.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.105+0000 7f4ec8a00640 1 -- 172.21.15.52:0/928875784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ec4070cd0 msgr2=0x7f4ec40cee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:30.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.105+0000 7f4ec8a00640 1 --2- 172.21.15.52:0/928875784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ec4070cd0 0x7f4ec40cee70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4ec40cf4f0 tx=0x7f4eb002b8d0 comp rx=0 tx=0).stop 2024-03-12T17:07:30.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.105+0000 7f4ec8a00640 1 -- 172.21.15.52:0/928875784 shutdown_connections 2024-03-12T17:07:30.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.105+0000 7f4ec8a00640 1 --2- 172.21.15.52:0/928875784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ec4070cd0 0x7f4ec40cee70 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:30.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.105+0000 7f4ec8a00640 1 --2- 172.21.15.52:0/928875784 >> v2:172.21.15.102:3300/0 conn(0x7f4ec41456a0 0x7f4ec40ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:30.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.105+0000 7f4ec8a00640 1 -- 172.21.15.52:0/928875784 >> 172.21.15.52:0/928875784 conn(0x7f4ec406a0b0 msgr2=0x7f4ec406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:30.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.105+0000 7f4ec8a00640 1 -- 172.21.15.52:0/928875784 shutdown_connections 2024-03-12T17:07:30.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.105+0000 7f4ec8a00640 1 -- 172.21.15.52:0/928875784 wait complete. 2024-03-12T17:07:30.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.107+0000 7f4ec8a00640 1 Processor -- start 2024-03-12T17:07:30.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.107+0000 7f4ec8a00640 1 -- start start 2024-03-12T17:07:30.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.109+0000 7f4ec8a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ec4070cd0 0x7f4ec413d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:30.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.109+0000 7f4ec8a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ec41456a0 0x7f4ec415e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:30.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.109+0000 7f4ec8a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4ec4145c20 0x7f4ec41f2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:30.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.109+0000 7f4ec8a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec41f5f60 con 0x7f4ec41456a0 2024-03-12T17:07:30.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.109+0000 7f4ec8a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec41f6240 con 0x7f4ec4070cd0 2024-03-12T17:07:30.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.109+0000 7f4ec8a00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec41f6520 con 0x7f4ec4145c20 2024-03-12T17:07:30.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.109+0000 7f4ec1400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ec41456a0 0x7f4ec415e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:30.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.109+0000 7f4ec1400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ec41456a0 0x7f4ec415e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51096/0 (socket says 172.21.15.52:51096) 2024-03-12T17:07:30.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.109+0000 7f4ec1400640 1 -- 172.21.15.52:0/2680137311 learned_addr learned my addr 172.21.15.52:0/2680137311 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:30.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.109+0000 7f4ec1e00640 1 --2- 172.21.15.52:0/2680137311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ec4070cd0 0x7f4ec413d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:30.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.109+0000 7f4ec2800640 1 --2- 172.21.15.52:0/2680137311 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4ec4145c20 0x7f4ec41f2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:30.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.110+0000 7f4ec1400640 1 -- 172.21.15.52:0/2680137311 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4ec4145c20 msgr2=0x7f4ec41f2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:30.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.110+0000 7f4ec1400640 1 --2- 172.21.15.52:0/2680137311 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4ec4145c20 0x7f4ec41f2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:30.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.110+0000 7f4ec1400640 1 -- 172.21.15.52:0/2680137311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ec4070cd0 msgr2=0x7f4ec413d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:30.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.110+0000 7f4ec1400640 1 --2- 172.21.15.52:0/2680137311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ec4070cd0 0x7f4ec413d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:30.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.110+0000 7f4ec1400640 1 -- 172.21.15.52:0/2680137311 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb0009400 con 0x7f4ec41456a0 2024-03-12T17:07:30.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.110+0000 7f4ec1e00640 1 --2- 172.21.15.52:0/2680137311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ec4070cd0 0x7f4ec413d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:07:30.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.110+0000 7f4ec1400640 1 --2- 172.21.15.52:0/2680137311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ec41456a0 0x7f4ec415e5b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4eb002bee0 tx=0x7f4eb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:30.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.110+0000 7f4eba000640 1 -- 172.21.15.52:0/2680137311 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb003a6f0 con 0x7f4ec41456a0 2024-03-12T17:07:30.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.111+0000 7f4eba000640 1 -- 172.21.15.52:0/2680137311 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb003a870 con 0x7f4ec41456a0 2024-03-12T17:07:30.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.111+0000 7f4eba000640 1 -- 172.21.15.52:0/2680137311 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb003ac20 con 0x7f4ec41456a0 2024-03-12T17:07:30.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.111+0000 7f4ec8a00640 1 -- 172.21.15.52:0/2680137311 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec415c820 con 0x7f4ec41456a0 2024-03-12T17:07:30.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.111+0000 7f4ec8a00640 1 -- 172.21.15.52:0/2680137311 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec413d780 con 0x7f4ec41456a0 2024-03-12T17:07:30.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.112+0000 7f4ec8a00640 1 -- 172.21.15.52:0/2680137311 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ec4073390 con 0x7f4ec41456a0 2024-03-12T17:07:30.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.113+0000 7f4eba000640 1 -- 172.21.15.52:0/2680137311 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4eb0030380 con 0x7f4ec41456a0 2024-03-12T17:07:30.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.114+0000 7f4eba000640 1 --2- 172.21.15.52:0/2680137311 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4ea406ac00 0x7f4ea406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:30.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.114+0000 7f4eba000640 1 -- 172.21.15.52:0/2680137311 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4eb00ab120 con 0x7f4ec41456a0 2024-03-12T17:07:30.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.114+0000 7f4ec1e00640 1 --2- 172.21.15.52:0/2680137311 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4ea406ac00 0x7f4ea406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:30.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.116+0000 7f4ec1e00640 1 --2- 172.21.15.52:0/2680137311 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4ea406ac00 0x7f4ea406d0c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4006fd0 tx=0x7f4eb4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:30.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.119+0000 7f4eba000640 1 -- 172.21.15.52:0/2680137311 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f4eb0075f30 con 0x7f4ec41456a0 2024-03-12T17:07:30.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.318+0000 7f4ec8a00640 1 -- 172.21.15.52:0/2680137311 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4ec406cbc0 con 0x7f4ec41456a0 2024-03-12T17:07:30.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.319+0000 7f4eba000640 1 -- 172.21.15.52:0/2680137311 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4eb00350a0 con 0x7f4ec41456a0 2024-03-12T17:07:30.313 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-03-12T17:07:30.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.322+0000 7f4ec8a00640 1 -- 172.21.15.52:0/2680137311 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4ea406ac00 msgr2=0x7f4ea406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:30.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.322+0000 7f4ec8a00640 1 --2- 172.21.15.52:0/2680137311 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4ea406ac00 0x7f4ea406d0c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4006fd0 tx=0x7f4eb4008040 comp rx=0 tx=0).stop 2024-03-12T17:07:30.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.322+0000 7f4ec8a00640 1 -- 172.21.15.52:0/2680137311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ec41456a0 msgr2=0x7f4ec415e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:30.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.322+0000 7f4ec8a00640 1 --2- 172.21.15.52:0/2680137311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ec41456a0 0x7f4ec415e5b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4eb002bee0 tx=0x7f4eb00046c0 comp rx=0 tx=0).stop 2024-03-12T17:07:30.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.322+0000 7f4ec8a00640 1 -- 172.21.15.52:0/2680137311 shutdown_connections 2024-03-12T17:07:30.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.322+0000 7f4ec8a00640 1 --2- 172.21.15.52:0/2680137311 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4ea406ac00 0x7f4ea406d0c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:30.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.322+0000 7f4ec8a00640 1 --2- 172.21.15.52:0/2680137311 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4ec4145c20 0x7f4ec41f2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:30.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.322+0000 7f4ec8a00640 1 --2- 172.21.15.52:0/2680137311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ec41456a0 0x7f4ec415e5b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:30.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.322+0000 7f4ec8a00640 1 --2- 172.21.15.52:0/2680137311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ec4070cd0 0x7f4ec413d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:30.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.322+0000 7f4ec8a00640 1 -- 172.21.15.52:0/2680137311 >> 172.21.15.52:0/2680137311 conn(0x7f4ec406a0b0 msgr2=0x7f4ec4140190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:30.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.322+0000 7f4ec8a00640 1 -- 172.21.15.52:0/2680137311 shutdown_connections 2024-03-12T17:07:30.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.322+0000 7f4ec8a00640 1 -- 172.21.15.52:0/2680137311 wait complete. 2024-03-12T17:07:30.336 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-12T17:07:30.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.568+0000 7f8d88c00640 1 Processor -- start 2024-03-12T17:07:30.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.568+0000 7f8d88c00640 1 -- start start 2024-03-12T17:07:30.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.570+0000 7f8d88c00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8d840d22d0 0x7f8d840cf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:30.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.570+0000 7f8d88c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8d840cf600 0x7f8d840c6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:30.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.570+0000 7f8d88c00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d840c6290 con 0x7f8d84160170 2024-03-12T17:07:30.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.570+0000 7f8d88c00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d840cfa90 con 0x7f8d840cf600 2024-03-12T17:07:30.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.570+0000 7f8d88c00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d840cdcd0 con 0x7f8d840d22d0 2024-03-12T17:07:30.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.571+0000 7f8d83e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8d840cf600 0x7f8d840c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:30.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.571+0000 7f8d7aa00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8d840d22d0 0x7f8d840cf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:30.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.571+0000 7f8d83e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8d840cf600 0x7f8d840c6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:39116/0 (socket says 172.21.15.52:39116) 2024-03-12T17:07:30.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.571+0000 7f8d83e00640 1 -- 172.21.15.52:0/946509510 learned_addr learned my addr 172.21.15.52:0/946509510 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:30.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.571+0000 7f8d83e00640 1 -- 172.21.15.52:0/946509510 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8d840d22d0 msgr2=0x7f8d840cf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:30.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.571+0000 7f8d83e00640 1 --2- 172.21.15.52:0/946509510 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8d840d22d0 0x7f8d840cf0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:30.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.571+0000 7f8d83e00640 1 -- 172.21.15.52:0/946509510 >> v1:172.21.15.102:6789/0 conn(0x7f8d84160170 legacy=0x7f8d840d2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:30.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.571+0000 7f8d83e00640 1 -- 172.21.15.52:0/946509510 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d840d34d0 con 0x7f8d840cf600 2024-03-12T17:07:30.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.573+0000 7f8d83e00640 1 --2- 172.21.15.52:0/946509510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d840cf600 0x7f8d840c6720 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c009870 tx=0x7f8d7c02b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be4d296e7e8455d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:30.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.573+0000 7f8d82a00640 1 -- 172.21.15.52:0/946509510 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c031640 con 0x7f8d840cf600 2024-03-12T17:07:30.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.573+0000 7f8d82a00640 1 -- 172.21.15.52:0/946509510 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d7c0317a0 con 0x7f8d840cf600 2024-03-12T17:07:30.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.573+0000 7f8d82a00640 1 -- 172.21.15.52:0/946509510 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c031b30 con 0x7f8d840cf600 2024-03-12T17:07:30.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.574+0000 7f8d88c00640 1 -- 172.21.15.52:0/946509510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d840cf600 msgr2=0x7f8d840c6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:30.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.574+0000 7f8d88c00640 1 --2- 172.21.15.52:0/946509510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d840cf600 0x7f8d840c6720 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c009870 tx=0x7f8d7c02b8f0 comp rx=0 tx=0).stop 2024-03-12T17:07:30.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.574+0000 7f8d88c00640 1 -- 172.21.15.52:0/946509510 shutdown_connections 2024-03-12T17:07:30.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.574+0000 7f8d88c00640 1 --2- 172.21.15.52:0/946509510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d840cf600 0x7f8d840c6720 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:30.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.574+0000 7f8d88c00640 1 --2- 172.21.15.52:0/946509510 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8d840d22d0 0x7f8d840cf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:30.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.574+0000 7f8d88c00640 1 -- 172.21.15.52:0/946509510 >> 172.21.15.52:0/946509510 conn(0x7f8d84089fd0 msgr2=0x7f8d840d26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:30.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.574+0000 7f8d88c00640 1 -- 172.21.15.52:0/946509510 shutdown_connections 2024-03-12T17:07:30.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.574+0000 7f8d88c00640 1 -- 172.21.15.52:0/946509510 wait complete. 2024-03-12T17:07:30.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.576+0000 7f8d88c00640 1 Processor -- start 2024-03-12T17:07:30.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.576+0000 7f8d88c00640 1 -- start start 2024-03-12T17:07:30.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.577+0000 7f8d88c00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8d840cf600 0x7f8d840b79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:30.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.577+0000 7f8d88c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d840d22d0 0x7f8d84159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:30.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.577+0000 7f8d88c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d84160170 0x7f8d8406af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:30.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.577+0000 7f8d88c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d840cc560 con 0x7f8d84160170 2024-03-12T17:07:30.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.577+0000 7f8d88c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d840b7540 con 0x7f8d840d22d0 2024-03-12T17:07:30.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.577+0000 7f8d88c00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d840b6930 con 0x7f8d840cf600 2024-03-12T17:07:30.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.577+0000 7f8d83e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d84160170 0x7f8d8406af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:30.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.578+0000 7f8d83e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d84160170 0x7f8d8406af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51104/0 (socket says 172.21.15.52:51104) 2024-03-12T17:07:30.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.578+0000 7f8d83e00640 1 -- 172.21.15.52:0/1617060311 learned_addr learned my addr 172.21.15.52:0/1617060311 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:30.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.578+0000 7f8d7aa00640 1 --2- 172.21.15.52:0/1617060311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d840d22d0 0x7f8d84159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:30.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.578+0000 7f8d83400640 1 --2- 172.21.15.52:0/1617060311 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8d840cf600 0x7f8d840b79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:30.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.578+0000 7f8d83e00640 1 -- 172.21.15.52:0/1617060311 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8d840cf600 msgr2=0x7f8d840b79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:30.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.578+0000 7f8d83e00640 1 --2- 172.21.15.52:0/1617060311 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8d840cf600 0x7f8d840b79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:30.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.578+0000 7f8d83e00640 1 -- 172.21.15.52:0/1617060311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d840d22d0 msgr2=0x7f8d84159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:30.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.578+0000 7f8d83e00640 1 --2- 172.21.15.52:0/1617060311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d840d22d0 0x7f8d84159c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:30.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.578+0000 7f8d83e00640 1 -- 172.21.15.52:0/1617060311 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d7c0094d0 con 0x7f8d84160170 2024-03-12T17:07:30.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.578+0000 7f8d83400640 1 --2- 172.21.15.52:0/1617060311 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8d840cf600 0x7f8d840b79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:30.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.578+0000 7f8d7aa00640 1 --2- 172.21.15.52:0/1617060311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d840d22d0 0x7f8d84159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:07:30.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.578+0000 7f8d83e00640 1 --2- 172.21.15.52:0/1617060311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d84160170 0x7f8d8406af30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c009870 tx=0x7f8d7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:30.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.579+0000 7f8d7be00640 1 -- 172.21.15.52:0/1617060311 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c03a930 con 0x7f8d84160170 2024-03-12T17:07:30.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.579+0000 7f8d7be00640 1 -- 172.21.15.52:0/1617060311 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d7c03aab0 con 0x7f8d84160170 2024-03-12T17:07:30.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.579+0000 7f8d88c00640 1 -- 172.21.15.52:0/1617060311 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d840b6c10 con 0x7f8d84160170 2024-03-12T17:07:30.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.579+0000 7f8d7be00640 1 -- 172.21.15.52:0/1617060311 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c03ae60 con 0x7f8d84160170 2024-03-12T17:07:30.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.579+0000 7f8d88c00640 1 -- 172.21.15.52:0/1617060311 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d840b71d0 con 0x7f8d84160170 2024-03-12T17:07:30.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.580+0000 7f8d88c00640 1 -- 172.21.15.52:0/1617060311 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d840ce460 con 0x7f8d84160170 2024-03-12T17:07:30.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.581+0000 7f8d7be00640 1 -- 172.21.15.52:0/1617060311 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8d7c02f050 con 0x7f8d84160170 2024-03-12T17:07:30.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.582+0000 7f8d7be00640 1 --2- 172.21.15.52:0/1617060311 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f8d6006ac00 0x7f8d6006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:30.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.582+0000 7f8d7be00640 1 -- 172.21.15.52:0/1617060311 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8d7c0ab5f0 con 0x7f8d84160170 2024-03-12T17:07:30.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.583+0000 7f8d83400640 1 --2- 172.21.15.52:0/1617060311 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f8d6006ac00 0x7f8d6006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:30.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.584+0000 7f8d83400640 1 --2- 172.21.15.52:0/1617060311 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f8d6006ac00 0x7f8d6006d0c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8d74001cd0 tx=0x7f8d74009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:30.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.586+0000 7f8d7be00640 1 -- 172.21.15.52:0/1617060311 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f8d7c076370 con 0x7f8d84160170 2024-03-12T17:07:30.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.782+0000 7f8d88c00640 1 -- 172.21.15.52:0/1617060311 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8d840b76b0 con 0x7f8d84160170 2024-03-12T17:07:30.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.783+0000 7f8d7be00640 1 -- 172.21.15.52:0/1617060311 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8d7c079990 con 0x7f8d84160170 2024-03-12T17:07:30.777 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-12T17:07:30.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.785+0000 7f8d88c00640 1 -- 172.21.15.52:0/1617060311 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f8d6006ac00 msgr2=0x7f8d6006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:30.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.785+0000 7f8d88c00640 1 --2- 172.21.15.52:0/1617060311 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f8d6006ac00 0x7f8d6006d0c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8d74001cd0 tx=0x7f8d74009040 comp rx=0 tx=0).stop 2024-03-12T17:07:30.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.785+0000 7f8d88c00640 1 -- 172.21.15.52:0/1617060311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d84160170 msgr2=0x7f8d8406af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:30.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.785+0000 7f8d88c00640 1 --2- 172.21.15.52:0/1617060311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d84160170 0x7f8d8406af30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c009870 tx=0x7f8d7c0046c0 comp rx=0 tx=0).stop 2024-03-12T17:07:30.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.785+0000 7f8d88c00640 1 -- 172.21.15.52:0/1617060311 shutdown_connections 2024-03-12T17:07:30.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.785+0000 7f8d88c00640 1 --2- 172.21.15.52:0/1617060311 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f8d6006ac00 0x7f8d6006d0c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:30.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.785+0000 7f8d88c00640 1 --2- 172.21.15.52:0/1617060311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d84160170 0x7f8d8406af30 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:30.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.785+0000 7f8d88c00640 1 --2- 172.21.15.52:0/1617060311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d840d22d0 0x7f8d84159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:30.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.785+0000 7f8d88c00640 1 --2- 172.21.15.52:0/1617060311 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8d840cf600 0x7f8d840b79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:30.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.786+0000 7f8d88c00640 1 -- 172.21.15.52:0/1617060311 >> 172.21.15.52:0/1617060311 conn(0x7f8d84089fd0 msgr2=0x7f8d8415d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:30.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.786+0000 7f8d88c00640 1 -- 172.21.15.52:0/1617060311 shutdown_connections 2024-03-12T17:07:30.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:30.786+0000 7f8d88c00640 1 -- 172.21.15.52:0/1617060311 wait complete. 2024-03-12T17:07:30.801 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-12T17:07:31.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.033+0000 7f51fe800640 1 Processor -- start 2024-03-12T17:07:31.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.033+0000 7f51fe800640 1 -- start start 2024-03-12T17:07:31.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.035+0000 7f51fe800640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f51f8160bd0 0x7f51f80cad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:31.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.035+0000 7f51fe800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f51f80cce70 0x7f51f80cb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:31.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.035+0000 7f51fe800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f51f80cd650 0x7f51f8069c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:31.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.035+0000 7f51fe800640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51f80c9a90 con 0x7f51f8160bd0 2024-03-12T17:07:31.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.035+0000 7f51fe800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51f81621d0 con 0x7f51f80cce70 2024-03-12T17:07:31.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.035+0000 7f51fe800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f80cd370 con 0x7f51f80cd650 2024-03-12T17:07:31.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.035+0000 7f51fd400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f51f8160bd0 0x7f51f80cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:31.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.035+0000 7f51fd400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f51f8160bd0 0x7f51f80cad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51112/0 (socket says 172.21.15.52:51112) 2024-03-12T17:07:31.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.035+0000 7f51fd400640 1 -- 172.21.15.52:0/3022717991 learned_addr learned my addr 172.21.15.52:0/3022717991 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:31.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.035+0000 7f51fca00640 1 --2- 172.21.15.52:0/3022717991 >> v2:172.21.15.102:3300/0 conn(0x7f51f80cce70 0x7f51f80cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:31.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.035+0000 7f51fde00640 1 --2- 172.21.15.52:0/3022717991 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f51f80cd650 0x7f51f8069c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:31.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.035+0000 7f51fd400640 1 -- 172.21.15.52:0/3022717991 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f51f80cd650 msgr2=0x7f51f8069c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:31.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.036+0000 7f51fd400640 1 --2- 172.21.15.52:0/3022717991 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f51f80cd650 0x7f51f8069c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:31.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.036+0000 7f51fd400640 1 -- 172.21.15.52:0/3022717991 >> v2:172.21.15.102:3300/0 conn(0x7f51f80cce70 msgr2=0x7f51f80cb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:31.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.036+0000 7f51fd400640 1 --2- 172.21.15.52:0/3022717991 >> v2:172.21.15.102:3300/0 conn(0x7f51f80cce70 0x7f51f80cb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:31.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.036+0000 7f51fd400640 1 -- 172.21.15.52:0/3022717991 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f8161410 con 0x7f51f8160bd0 2024-03-12T17:07:31.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.037+0000 7f51fd400640 1 --2- 172.21.15.52:0/3022717991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51f8160bd0 0x7f51f80cad10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f51f80b3510 tx=0x7f51ec02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22015207815c32d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:31.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.037+0000 7f51f7e00640 1 -- 172.21.15.52:0/3022717991 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec031870 con 0x7f51f8160bd0 2024-03-12T17:07:31.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.037+0000 7f51f7e00640 1 -- 172.21.15.52:0/3022717991 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ec0319d0 con 0x7f51f8160bd0 2024-03-12T17:07:31.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.037+0000 7f51f7e00640 1 -- 172.21.15.52:0/3022717991 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec031d60 con 0x7f51f8160bd0 2024-03-12T17:07:31.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.038+0000 7f51fe800640 1 -- 172.21.15.52:0/3022717991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51f8160bd0 msgr2=0x7f51f80cad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:31.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.038+0000 7f51fe800640 1 --2- 172.21.15.52:0/3022717991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51f8160bd0 0x7f51f80cad10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f51f80b3510 tx=0x7f51ec02b910 comp rx=0 tx=0).stop 2024-03-12T17:07:31.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.038+0000 7f51fe800640 1 -- 172.21.15.52:0/3022717991 shutdown_connections 2024-03-12T17:07:31.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.038+0000 7f51fe800640 1 --2- 172.21.15.52:0/3022717991 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f51f80cd650 0x7f51f8069c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:31.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.038+0000 7f51fe800640 1 --2- 172.21.15.52:0/3022717991 >> v2:172.21.15.102:3300/0 conn(0x7f51f80cce70 0x7f51f80cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:31.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.038+0000 7f51fe800640 1 --2- 172.21.15.52:0/3022717991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51f8160bd0 0x7f51f80cad10 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:31.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.038+0000 7f51fe800640 1 -- 172.21.15.52:0/3022717991 >> 172.21.15.52:0/3022717991 conn(0x7f51f80b01b0 msgr2=0x7f51f80b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:31.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.038+0000 7f51fe800640 1 -- 172.21.15.52:0/3022717991 shutdown_connections 2024-03-12T17:07:31.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.038+0000 7f51fe800640 1 -- 172.21.15.52:0/3022717991 wait complete. 2024-03-12T17:07:31.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.040+0000 7f51fe800640 1 Processor -- start 2024-03-12T17:07:31.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.040+0000 7f51fe800640 1 -- start start 2024-03-12T17:07:31.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.041+0000 7f51fe800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51f80cce70 0x7f51f806d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:31.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.041+0000 7f51fe800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f51f80cd650 0x7f51f80af200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:31.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.041+0000 7f51fe800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51f8160bd0 0x7f51f806c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:31.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.041+0000 7f51fe800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f809ac60 con 0x7f51f8160bd0 2024-03-12T17:07:31.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.041+0000 7f51fe800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f80987a0 con 0x7f51f80cce70 2024-03-12T17:07:31.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.041+0000 7f51fe800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f8098a80 con 0x7f51f80cd650 2024-03-12T17:07:31.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.041+0000 7f51fde00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51f8160bd0 0x7f51f806c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:31.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.041+0000 7f51fde00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51f8160bd0 0x7f51f806c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51114/0 (socket says 172.21.15.52:51114) 2024-03-12T17:07:31.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.041+0000 7f51fde00640 1 -- 172.21.15.52:0/392173470 learned_addr learned my addr 172.21.15.52:0/392173470 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:31.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.041+0000 7f51fd400640 1 --2- 172.21.15.52:0/392173470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51f80cce70 0x7f51f806d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:31.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.041+0000 7f51fca00640 1 --2- 172.21.15.52:0/392173470 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f51f80cd650 0x7f51f80af200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:31.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.042+0000 7f51fde00640 1 -- 172.21.15.52:0/392173470 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f51f80cd650 msgr2=0x7f51f80af200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:31.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.042+0000 7f51fde00640 1 --2- 172.21.15.52:0/392173470 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f51f80cd650 0x7f51f80af200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:31.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.042+0000 7f51fde00640 1 -- 172.21.15.52:0/392173470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51f80cce70 msgr2=0x7f51f806d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:31.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.042+0000 7f51fde00640 1 --2- 172.21.15.52:0/392173470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51f80cce70 0x7f51f806d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:31.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.042+0000 7f51fde00640 1 -- 172.21.15.52:0/392173470 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ec0094d0 con 0x7f51f8160bd0 2024-03-12T17:07:31.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.042+0000 7f51fca00640 1 --2- 172.21.15.52:0/392173470 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f51f80cd650 0x7f51f80af200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:31.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.042+0000 7f51fde00640 1 --2- 172.21.15.52:0/392173470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51f8160bd0 0x7f51f806c040 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f51f0007e30 tx=0x7f51f000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:31.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.042+0000 7f51f5600640 1 -- 172.21.15.52:0/392173470 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f0014620 con 0x7f51f8160bd0 2024-03-12T17:07:31.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.042+0000 7f51f5600640 1 -- 172.21.15.52:0/392173470 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f000ee90 con 0x7f51f8160bd0 2024-03-12T17:07:31.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.042+0000 7f51fe800640 1 -- 172.21.15.52:0/392173470 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f8098d60 con 0x7f51f8160bd0 2024-03-12T17:07:31.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.042+0000 7f51f5600640 1 -- 172.21.15.52:0/392173470 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f00149c0 con 0x7f51f8160bd0 2024-03-12T17:07:31.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.042+0000 7f51fe800640 1 -- 172.21.15.52:0/392173470 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f8099320 con 0x7f51f8160bd0 2024-03-12T17:07:31.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.044+0000 7f51fe800640 1 -- 172.21.15.52:0/392173470 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51f804e030 con 0x7f51f8160bd0 2024-03-12T17:07:31.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.045+0000 7f51f5600640 1 -- 172.21.15.52:0/392173470 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f51f0014b20 con 0x7f51f8160bd0 2024-03-12T17:07:31.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.046+0000 7f51f5600640 1 --2- 172.21.15.52:0/392173470 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f51d806ac00 0x7f51d806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:31.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.046+0000 7f51f5600640 1 -- 172.21.15.52:0/392173470 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f51f008a140 con 0x7f51f8160bd0 2024-03-12T17:07:31.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.046+0000 7f51fd400640 1 --2- 172.21.15.52:0/392173470 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f51d806ac00 0x7f51d806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:31.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.047+0000 7f51fd400640 1 --2- 172.21.15.52:0/392173470 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f51d806ac00 0x7f51d806d0c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f51ec036000 tx=0x7f51ec03c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:31.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.051+0000 7f51f5600640 1 -- 172.21.15.52:0/392173470 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f51f0054ec0 con 0x7f51f8160bd0 2024-03-12T17:07:31.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.250+0000 7f51fe800640 1 -- 172.21.15.52:0/392173470 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f51f80b3320 con 0x7f51f8160bd0 2024-03-12T17:07:31.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.251+0000 7f51f5600640 1 -- 172.21.15.52:0/392173470 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f51f00584e0 con 0x7f51f8160bd0 2024-03-12T17:07:31.245 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-03-12T17:07:31.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.253+0000 7f51fe800640 1 -- 172.21.15.52:0/392173470 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f51d806ac00 msgr2=0x7f51d806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:31.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.253+0000 7f51fe800640 1 --2- 172.21.15.52:0/392173470 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f51d806ac00 0x7f51d806d0c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f51ec036000 tx=0x7f51ec03c040 comp rx=0 tx=0).stop 2024-03-12T17:07:31.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.253+0000 7f51fe800640 1 -- 172.21.15.52:0/392173470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51f8160bd0 msgr2=0x7f51f806c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:31.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.253+0000 7f51fe800640 1 --2- 172.21.15.52:0/392173470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51f8160bd0 0x7f51f806c040 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f51f0007e30 tx=0x7f51f000e4b0 comp rx=0 tx=0).stop 2024-03-12T17:07:31.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.253+0000 7f51fe800640 1 -- 172.21.15.52:0/392173470 shutdown_connections 2024-03-12T17:07:31.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.253+0000 7f51fe800640 1 --2- 172.21.15.52:0/392173470 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f51d806ac00 0x7f51d806d0c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:31.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.253+0000 7f51fe800640 1 --2- 172.21.15.52:0/392173470 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f51f80cd650 0x7f51f80af200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:31.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.253+0000 7f51fe800640 1 --2- 172.21.15.52:0/392173470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51f8160bd0 0x7f51f806c040 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:31.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.253+0000 7f51fe800640 1 --2- 172.21.15.52:0/392173470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51f80cce70 0x7f51f806d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:31.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.254+0000 7f51fe800640 1 -- 172.21.15.52:0/392173470 >> 172.21.15.52:0/392173470 conn(0x7f51f80b01b0 msgr2=0x7f51f80b0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:31.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.254+0000 7f51fe800640 1 -- 172.21.15.52:0/392173470 shutdown_connections 2024-03-12T17:07:31.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.254+0000 7f51fe800640 1 -- 172.21.15.52:0/392173470 wait complete. 2024-03-12T17:07:31.268 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-12T17:07:31.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.503+0000 7f0c05e00640 1 Processor -- start 2024-03-12T17:07:31.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.504+0000 7f0c05e00640 1 -- start start 2024-03-12T17:07:31.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.505+0000 7f0c05e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f0c000cf5c0 0x7f0c000b8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:31.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.505+0000 7f0c05e00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c0015ce90 con 0x7f0c000d0c20 2024-03-12T17:07:31.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.505+0000 7f0c05e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c000cddf0 con 0x7f0c0015cab0 2024-03-12T17:07:31.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.505+0000 7f0c05e00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c000cd570 con 0x7f0c000cf5c0 2024-03-12T17:07:31.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.506+0000 7f0bffe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f0c000cf5c0 0x7f0c000b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:31.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.506+0000 7f0c05400640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f0c000d0c20 0x7f0c000d1070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:52378/0 (socket says 172.21.15.52:52378) 2024-03-12T17:07:31.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.506+0000 7f0c05400640 1 -- 172.21.15.52:0/3228404556 learned_addr learned my addr 172.21.15.52:0/3228404556 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:31.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.506+0000 7f0bffe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f0c000cf5c0 0x7f0c000b8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51128/0 (socket says 172.21.15.52:51128) 2024-03-12T17:07:31.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.506+0000 7f0bffe00640 1 -- 172.21.15.52:0/3228404556 >> v1:172.21.15.102:6789/0 conn(0x7f0c0015cab0 legacy=0x7f0c000ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:31.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.506+0000 7f0bffe00640 1 -- 172.21.15.52:0/3228404556 >> v1:172.21.15.52:6789/0 conn(0x7f0c000d0c20 legacy=0x7f0c000d1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:31.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.506+0000 7f0bffe00640 1 -- 172.21.15.52:0/3228404556 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c00161370 con 0x7f0c000cf5c0 2024-03-12T17:07:31.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.508+0000 7f0bffe00640 1 --2- 172.21.15.52:0/3228404556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c000cf5c0 0x7f0c000b8050 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0bec0097a0 tx=0x7f0bec02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3241ae1e2316e8e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:31.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.508+0000 7f0bff400640 1 -- 172.21.15.52:0/3228404556 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bec030610 con 0x7f0c000cf5c0 2024-03-12T17:07:31.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.508+0000 7f0bff400640 1 -- 172.21.15.52:0/3228404556 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bec030770 con 0x7f0c000cf5c0 2024-03-12T17:07:31.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.508+0000 7f0bff400640 1 -- 172.21.15.52:0/3228404556 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bec030b00 con 0x7f0c000cf5c0 2024-03-12T17:07:31.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.508+0000 7f0c05e00640 1 -- 172.21.15.52:0/3228404556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c000cf5c0 msgr2=0x7f0c000b8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:31.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.508+0000 7f0c05e00640 1 --2- 172.21.15.52:0/3228404556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c000cf5c0 0x7f0c000b8050 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0bec0097a0 tx=0x7f0bec02b960 comp rx=0 tx=0).stop 2024-03-12T17:07:31.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.509+0000 7f0c05e00640 1 -- 172.21.15.52:0/3228404556 shutdown_connections 2024-03-12T17:07:31.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.509+0000 7f0c05e00640 1 --2- 172.21.15.52:0/3228404556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c000cf5c0 0x7f0c000b8050 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:31.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.509+0000 7f0c05e00640 1 -- 172.21.15.52:0/3228404556 >> 172.21.15.52:0/3228404556 conn(0x7f0c0006ae90 msgr2=0x7f0c0006b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:31.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.509+0000 7f0c05e00640 1 -- 172.21.15.52:0/3228404556 shutdown_connections 2024-03-12T17:07:31.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.509+0000 7f0c05e00640 1 -- 172.21.15.52:0/3228404556 wait complete. 2024-03-12T17:07:31.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.510+0000 7f0c05e00640 1 Processor -- start 2024-03-12T17:07:31.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.511+0000 7f0c05e00640 1 -- start start 2024-03-12T17:07:31.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.512+0000 7f0c05e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c000cf5c0 0x7f0c000c7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:31.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.512+0000 7f0c05e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0c000d0c20 0x7f0c00151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:31.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.512+0000 7f0c05e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c0015cab0 0x7f0c0015de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:31.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.512+0000 7f0c05e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c000c8740 con 0x7f0c0015cab0 2024-03-12T17:07:31.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.512+0000 7f0c05e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c000c66e0 con 0x7f0c000cf5c0 2024-03-12T17:07:31.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.512+0000 7f0c05e00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c000c8180 con 0x7f0c000d0c20 2024-03-12T17:07:31.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.512+0000 7f0c05400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c0015cab0 0x7f0c0015de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:31.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.512+0000 7f0c05400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c0015cab0 0x7f0c0015de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51134/0 (socket says 172.21.15.52:51134) 2024-03-12T17:07:31.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.512+0000 7f0c05400640 1 -- 172.21.15.52:0/908505706 learned_addr learned my addr 172.21.15.52:0/908505706 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:31.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.512+0000 7f0c04a00640 1 --2- 172.21.15.52:0/908505706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c000cf5c0 0x7f0c000c7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:31.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.512+0000 7f0bffe00640 1 --2- 172.21.15.52:0/908505706 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0c000d0c20 0x7f0c00151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:31.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.512+0000 7f0c05400640 1 -- 172.21.15.52:0/908505706 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0c000d0c20 msgr2=0x7f0c00151b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:31.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.512+0000 7f0c05400640 1 --2- 172.21.15.52:0/908505706 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0c000d0c20 0x7f0c00151b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:31.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.512+0000 7f0c05400640 1 -- 172.21.15.52:0/908505706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c000cf5c0 msgr2=0x7f0c000c7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:31.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.512+0000 7f0c05400640 1 --2- 172.21.15.52:0/908505706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c000cf5c0 0x7f0c000c7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:31.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.513+0000 7f0c05400640 1 -- 172.21.15.52:0/908505706 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bec009400 con 0x7f0c0015cab0 2024-03-12T17:07:31.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.513+0000 7f0c05400640 1 --2- 172.21.15.52:0/908505706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c0015cab0 0x7f0c0015de40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800cbc0 tx=0x7f0bf800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:31.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.513+0000 7f0bfcc00640 1 -- 172.21.15.52:0/908505706 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf800ad40 con 0x7f0c0015cab0 2024-03-12T17:07:31.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.513+0000 7f0bfcc00640 1 -- 172.21.15.52:0/908505706 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf8012400 con 0x7f0c0015cab0 2024-03-12T17:07:31.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.513+0000 7f0bfcc00640 1 -- 172.21.15.52:0/908505706 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf800aea0 con 0x7f0c0015cab0 2024-03-12T17:07:31.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.513+0000 7f0c05e00640 1 -- 172.21.15.52:0/908505706 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c000c8460 con 0x7f0c0015cab0 2024-03-12T17:07:31.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.513+0000 7f0c05e00640 1 -- 172.21.15.52:0/908505706 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c0015e380 con 0x7f0c0015cab0 2024-03-12T17:07:31.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.515+0000 7f0c05e00640 1 -- 172.21.15.52:0/908505706 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c000c61d0 con 0x7f0c0015cab0 2024-03-12T17:07:31.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.516+0000 7f0bfcc00640 1 -- 172.21.15.52:0/908505706 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0bf8007070 con 0x7f0c0015cab0 2024-03-12T17:07:31.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.517+0000 7f0bfcc00640 1 --2- 172.21.15.52:0/908505706 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f0be006ac00 0x7f0be006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:31.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.517+0000 7f0bfcc00640 1 -- 172.21.15.52:0/908505706 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0bf8089580 con 0x7f0c0015cab0 2024-03-12T17:07:31.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.518+0000 7f0c04a00640 1 --2- 172.21.15.52:0/908505706 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f0be006ac00 0x7f0be006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:31.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.519+0000 7f0c04a00640 1 --2- 172.21.15.52:0/908505706 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f0be006ac00 0x7f0be006d0c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4000f60 tx=0x7f0bf4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:31.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.522+0000 7f0bfcc00640 1 -- 172.21.15.52:0/908505706 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f0c000c61d0 con 0x7f0c0015cab0 2024-03-12T17:07:31.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.724+0000 7f0c05e00640 1 -- 172.21.15.52:0/908505706 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0c000c61d0 con 0x7f0c0015cab0 2024-03-12T17:07:31.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.725+0000 7f0bfcc00640 1 -- 172.21.15.52:0/908505706 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0c000c61d0 con 0x7f0c0015cab0 2024-03-12T17:07:31.719 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-12T17:07:31.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.728+0000 7f0c05e00640 1 -- 172.21.15.52:0/908505706 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f0be006ac00 msgr2=0x7f0be006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:31.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.728+0000 7f0c05e00640 1 --2- 172.21.15.52:0/908505706 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f0be006ac00 0x7f0be006d0c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4000f60 tx=0x7f0bf4009040 comp rx=0 tx=0).stop 2024-03-12T17:07:31.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.728+0000 7f0c05e00640 1 -- 172.21.15.52:0/908505706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c0015cab0 msgr2=0x7f0c0015de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:31.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.728+0000 7f0c05e00640 1 --2- 172.21.15.52:0/908505706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c0015cab0 0x7f0c0015de40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800cbc0 tx=0x7f0bf800a430 comp rx=0 tx=0).stop 2024-03-12T17:07:31.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.728+0000 7f0c05e00640 1 -- 172.21.15.52:0/908505706 shutdown_connections 2024-03-12T17:07:31.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.728+0000 7f0c05e00640 1 --2- 172.21.15.52:0/908505706 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f0be006ac00 0x7f0be006d0c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:31.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.728+0000 7f0c05e00640 1 --2- 172.21.15.52:0/908505706 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0c000d0c20 0x7f0c00151b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:31.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.728+0000 7f0c05e00640 1 --2- 172.21.15.52:0/908505706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c0015cab0 0x7f0c0015de40 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:31.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.728+0000 7f0c05e00640 1 --2- 172.21.15.52:0/908505706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c000cf5c0 0x7f0c000c7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:31.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.728+0000 7f0c05e00640 1 -- 172.21.15.52:0/908505706 >> 172.21.15.52:0/908505706 conn(0x7f0c0006ae90 msgr2=0x7f0c0006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:31.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.728+0000 7f0c05e00640 1 -- 172.21.15.52:0/908505706 shutdown_connections 2024-03-12T17:07:31.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:31.728+0000 7f0c05e00640 1 -- 172.21.15.52:0/908505706 wait complete. 2024-03-12T17:07:31.796 INFO:tasks.cephfs_test_runner:Starting test: test_acls (tasks.cephfs.test_acls.TestACLs) 2024-03-12T17:07:31.797 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_acls.TestACLs.test_acls' 2024-03-12T17:07:32.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.028+0000 7f2ea3600640 1 Processor -- start 2024-03-12T17:07:32.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.029+0000 7f2ea3600640 1 -- start start 2024-03-12T17:07:32.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.030+0000 7f2ea3600640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2e9c15cd70 0x7f2e9c0d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:32.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.030+0000 7f2ea3600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2e9c0d1330 0x7f2e9c0b4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:32.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.030+0000 7f2ea3600640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e9c0d17d0 con 0x7f2e9c0b4880 2024-03-12T17:07:32.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.030+0000 7f2ea3600640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e9c15d150 con 0x7f2e9c15cd70 2024-03-12T17:07:32.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.030+0000 7f2ea3600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e9c0b4d10 con 0x7f2e9c0d1330 2024-03-12T17:07:32.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.030+0000 7f2ea0c00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2e9c15cd70 0x7f2e9c0d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:32.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.030+0000 7f2ea0c00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2e9c15cd70 0x7f2e9c0d0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51150/0 (socket says 172.21.15.52:51150) 2024-03-12T17:07:32.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.030+0000 7f2ea0c00640 1 -- 172.21.15.52:0/1036120990 learned_addr learned my addr 172.21.15.52:0/1036120990 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:32.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.031+0000 7f2e9be00640 1 --2- 172.21.15.52:0/1036120990 >> v2:172.21.15.102:3300/0 conn(0x7f2e9c0d1330 0x7f2e9c0b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:32.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.031+0000 7f2ea0c00640 1 -- 172.21.15.52:0/1036120990 >> v2:172.21.15.102:3300/0 conn(0x7f2e9c0d1330 msgr2=0x7f2e9c0b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:32.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.031+0000 7f2ea0c00640 1 --2- 172.21.15.52:0/1036120990 >> v2:172.21.15.102:3300/0 conn(0x7f2e9c0d1330 0x7f2e9c0b4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:32.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.031+0000 7f2ea0c00640 1 -- 172.21.15.52:0/1036120990 >> v1:172.21.15.102:6789/0 conn(0x7f2e9c0b4880 legacy=0x7f2e9c06b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:32.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.031+0000 7f2ea0c00640 1 -- 172.21.15.52:0/1036120990 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c0d1ab0 con 0x7f2e9c15cd70 2024-03-12T17:07:32.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.032+0000 7f2ea0c00640 1 --2- 172.21.15.52:0/1036120990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e9c15cd70 0x7f2e9c0d0df0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c0c9940 tx=0x7f2e8c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9557675fe52c667b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:32.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.032+0000 7f2e9b400640 1 -- 172.21.15.52:0/1036120990 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c031870 con 0x7f2e9c15cd70 2024-03-12T17:07:32.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.032+0000 7f2e9b400640 1 -- 172.21.15.52:0/1036120990 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e8c0319d0 con 0x7f2e9c15cd70 2024-03-12T17:07:32.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.032+0000 7f2e9b400640 1 -- 172.21.15.52:0/1036120990 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c031d60 con 0x7f2e9c15cd70 2024-03-12T17:07:32.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.033+0000 7f2ea3600640 1 -- 172.21.15.52:0/1036120990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e9c15cd70 msgr2=0x7f2e9c0d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:32.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.033+0000 7f2ea3600640 1 --2- 172.21.15.52:0/1036120990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e9c15cd70 0x7f2e9c0d0df0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c0c9940 tx=0x7f2e8c02b910 comp rx=0 tx=0).stop 2024-03-12T17:07:32.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.033+0000 7f2ea3600640 1 -- 172.21.15.52:0/1036120990 shutdown_connections 2024-03-12T17:07:32.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.033+0000 7f2ea3600640 1 --2- 172.21.15.52:0/1036120990 >> v2:172.21.15.102:3300/0 conn(0x7f2e9c0d1330 0x7f2e9c0b4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:32.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.033+0000 7f2ea3600640 1 --2- 172.21.15.52:0/1036120990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e9c15cd70 0x7f2e9c0d0df0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:32.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.033+0000 7f2ea3600640 1 -- 172.21.15.52:0/1036120990 >> 172.21.15.52:0/1036120990 conn(0x7f2e9c0cf340 msgr2=0x7f2e9c0cf750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:32.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.033+0000 7f2ea3600640 1 -- 172.21.15.52:0/1036120990 shutdown_connections 2024-03-12T17:07:32.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.033+0000 7f2ea3600640 1 -- 172.21.15.52:0/1036120990 wait complete. 2024-03-12T17:07:32.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.035+0000 7f2ea3600640 1 Processor -- start 2024-03-12T17:07:32.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.035+0000 7f2ea3600640 1 -- start start 2024-03-12T17:07:32.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.036+0000 7f2ea3600640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2e9c0b4880 0x7f2e9c0ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:32.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.036+0000 7f2ea3600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e9c0d1330 0x7f2e9c158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:32.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.036+0000 7f2ea3600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e9c15cd70 0x7f2e9c15e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:32.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.036+0000 7f2ea3600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c15f750 con 0x7f2e9c0d1330 2024-03-12T17:07:32.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.036+0000 7f2ea3600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c0c9990 con 0x7f2e9c15cd70 2024-03-12T17:07:32.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.036+0000 7f2ea3600640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c0ca230 con 0x7f2e9c0b4880 2024-03-12T17:07:32.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.037+0000 7f2e9be00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e9c0d1330 0x7f2e9c158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:32.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.037+0000 7f2e9be00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e9c0d1330 0x7f2e9c158d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51154/0 (socket says 172.21.15.52:51154) 2024-03-12T17:07:32.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.037+0000 7f2e9be00640 1 -- 172.21.15.52:0/2084725934 learned_addr learned my addr 172.21.15.52:0/2084725934 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:32.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.037+0000 7f2ea1600640 1 --2- 172.21.15.52:0/2084725934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e9c15cd70 0x7f2e9c15e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:32.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.037+0000 7f2ea0c00640 1 --2- 172.21.15.52:0/2084725934 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2e9c0b4880 0x7f2e9c0ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:32.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.037+0000 7f2e9be00640 1 -- 172.21.15.52:0/2084725934 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2e9c0b4880 msgr2=0x7f2e9c0ca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:32.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.037+0000 7f2e9be00640 1 --2- 172.21.15.52:0/2084725934 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2e9c0b4880 0x7f2e9c0ca6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:32.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.037+0000 7f2e9be00640 1 -- 172.21.15.52:0/2084725934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e9c15cd70 msgr2=0x7f2e9c15e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:32.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.037+0000 7f2e9be00640 1 --2- 172.21.15.52:0/2084725934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e9c15cd70 0x7f2e9c15e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:32.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.037+0000 7f2e9be00640 1 -- 172.21.15.52:0/2084725934 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e8c0094d0 con 0x7f2e9c0d1330 2024-03-12T17:07:32.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.037+0000 7f2ea0c00640 1 --2- 172.21.15.52:0/2084725934 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2e9c0b4880 0x7f2e9c0ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:32.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.038+0000 7f2e9be00640 1 --2- 172.21.15.52:0/2084725934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e9c0d1330 0x7f2e9c158d10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2e8400d7d0 tx=0x7f2e8400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:32.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.038+0000 7f2e98c00640 1 -- 172.21.15.52:0/2084725934 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8400b6c0 con 0x7f2e9c0d1330 2024-03-12T17:07:32.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.038+0000 7f2e98c00640 1 -- 172.21.15.52:0/2084725934 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e8400b820 con 0x7f2e9c0d1330 2024-03-12T17:07:32.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.038+0000 7f2e98c00640 1 -- 172.21.15.52:0/2084725934 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8400bbb0 con 0x7f2e9c0d1330 2024-03-12T17:07:32.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.038+0000 7f2ea3600640 1 -- 172.21.15.52:0/2084725934 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e9c15ee90 con 0x7f2e9c0d1330 2024-03-12T17:07:32.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.038+0000 7f2ea3600640 1 -- 172.21.15.52:0/2084725934 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e9c0c9ec0 con 0x7f2e9c0d1330 2024-03-12T17:07:32.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.039+0000 7f2ea3600640 1 -- 172.21.15.52:0/2084725934 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e9c0ca3a0 con 0x7f2e9c0d1330 2024-03-12T17:07:32.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.040+0000 7f2e98c00640 1 -- 172.21.15.52:0/2084725934 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2e8400bd10 con 0x7f2e9c0d1330 2024-03-12T17:07:32.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.042+0000 7f2e98c00640 1 --2- 172.21.15.52:0/2084725934 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2e7806ac30 0x7f2e7806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:32.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.042+0000 7f2e98c00640 1 -- 172.21.15.52:0/2084725934 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2e84089cc0 con 0x7f2e9c0d1330 2024-03-12T17:07:32.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.042+0000 7f2ea0c00640 1 --2- 172.21.15.52:0/2084725934 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2e7806ac30 0x7f2e7806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:32.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.043+0000 7f2ea0c00640 1 --2- 172.21.15.52:0/2084725934 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2e7806ac30 0x7f2e7806d0f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c036000 tx=0x7f2e8c03c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:32.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.046+0000 7f2e98c00640 1 -- 172.21.15.52:0/2084725934 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f2e84090080 con 0x7f2e9c0d1330 2024-03-12T17:07:32.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:32.320+0000 7f2ea3600640 1 -- 172.21.15.52:0/2084725934 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_acls.TestACLs.test_acls"]} v 0) v1 -- 0x7f2e9c073390 con 0x7f2e9c0d1330 2024-03-12T17:07:33.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.089+0000 7f2e98c00640 1 -- 172.21.15.52:0/2084725934 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_acls.TestACLs.test_acls"]}]=0 v72) v1 ==== 123+0+0 (secure 0 0 0) 0x7f2e84054a40 con 0x7f2e9c0d1330 2024-03-12T17:07:33.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.092+0000 7f2ea3600640 1 -- 172.21.15.52:0/2084725934 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2e7806ac30 msgr2=0x7f2e7806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:33.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.092+0000 7f2ea3600640 1 --2- 172.21.15.52:0/2084725934 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2e7806ac30 0x7f2e7806d0f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c036000 tx=0x7f2e8c03c040 comp rx=0 tx=0).stop 2024-03-12T17:07:33.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.092+0000 7f2ea3600640 1 -- 172.21.15.52:0/2084725934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e9c0d1330 msgr2=0x7f2e9c158d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:33.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.092+0000 7f2ea3600640 1 --2- 172.21.15.52:0/2084725934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e9c0d1330 0x7f2e9c158d10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2e8400d7d0 tx=0x7f2e8400dc90 comp rx=0 tx=0).stop 2024-03-12T17:07:33.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.092+0000 7f2ea3600640 1 -- 172.21.15.52:0/2084725934 shutdown_connections 2024-03-12T17:07:33.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.092+0000 7f2ea3600640 1 --2- 172.21.15.52:0/2084725934 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2e7806ac30 0x7f2e7806d0f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:33.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.092+0000 7f2ea3600640 1 --2- 172.21.15.52:0/2084725934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e9c15cd70 0x7f2e9c15e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:33.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.093+0000 7f2ea3600640 1 --2- 172.21.15.52:0/2084725934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e9c0d1330 0x7f2e9c158d10 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:33.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.093+0000 7f2ea3600640 1 --2- 172.21.15.52:0/2084725934 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2e9c0b4880 0x7f2e9c0ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:33.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.093+0000 7f2ea3600640 1 -- 172.21.15.52:0/2084725934 >> 172.21.15.52:0/2084725934 conn(0x7f2e9c0cf340 msgr2=0x7f2e9c0b6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:33.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.093+0000 7f2ea3600640 1 -- 172.21.15.52:0/2084725934 shutdown_connections 2024-03-12T17:07:33.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.093+0000 7f2ea3600640 1 -- 172.21.15.52:0/2084725934 wait complete. 2024-03-12T17:07:33.109 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-12T17:07:33.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.348+0000 7f11ccc00640 1 Processor -- start 2024-03-12T17:07:33.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.348+0000 7f11ccc00640 1 -- start start 2024-03-12T17:07:33.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.349+0000 7f11ccc00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f11c80add30 0x7f11c80ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:33.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.349+0000 7f11ccc00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f11c806ce90 0x7f11c806d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:33.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.349+0000 7f11ccc00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11c80cf250 con 0x7f11c80ce930 2024-03-12T17:07:33.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.349+0000 7f11ccc00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11c80ae1c0 con 0x7f11c80add30 2024-03-12T17:07:33.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.349+0000 7f11ccc00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c806c760 con 0x7f11c806ce90 2024-03-12T17:07:33.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.349+0000 7f11c5e00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f11c80ce930 0x7f11c80cd3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:52380/0 (socket says 172.21.15.52:52380) 2024-03-12T17:07:33.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.349+0000 7f11c5e00640 1 -- 172.21.15.52:0/2603830786 learned_addr learned my addr 172.21.15.52:0/2603830786 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:33.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.349+0000 7f11c5400640 1 --2- 172.21.15.52:0/2603830786 >> v2:172.21.15.52:3300/0 conn(0x7f11c80add30 0x7f11c80ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:33.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.350+0000 7f11c6800640 1 --2- 172.21.15.52:0/2603830786 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f11c806ce90 0x7f11c806d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:33.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.350+0000 7f11c5400640 1 -- 172.21.15.52:0/2603830786 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f11c806ce90 msgr2=0x7f11c806d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:33.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.350+0000 7f11c5400640 1 --2- 172.21.15.52:0/2603830786 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f11c806ce90 0x7f11c806d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:33.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.350+0000 7f11c5400640 1 -- 172.21.15.52:0/2603830786 >> v1:172.21.15.52:6789/0 conn(0x7f11c80ce930 legacy=0x7f11c80cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:33.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.350+0000 7f11c5400640 1 -- 172.21.15.52:0/2603830786 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c80cdba0 con 0x7f11c80add30 2024-03-12T17:07:33.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.350+0000 7f11c4a00640 1 -- 172.21.15.52:0/2603830786 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2395418675 0 0) 0x7f11c80cf250 con 0x7f11c80ce930 2024-03-12T17:07:33.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.350+0000 7f11c6800640 1 --2- 172.21.15.52:0/2603830786 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f11c806ce90 0x7f11c806d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:33.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.351+0000 7f11c5400640 1 --2- 172.21.15.52:0/2603830786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11c80add30 0x7f11c80ced10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f11b40097a0 tx=0x7f11b402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34c25686b2bead server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:33.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.351+0000 7f11c4a00640 1 -- 172.21.15.52:0/2603830786 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b402d040 con 0x7f11c80add30 2024-03-12T17:07:33.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.351+0000 7f11c4a00640 1 -- 172.21.15.52:0/2603830786 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b4031810 con 0x7f11c80add30 2024-03-12T17:07:33.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.352+0000 7f11c4a00640 1 -- 172.21.15.52:0/2603830786 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b4031ba0 con 0x7f11c80add30 2024-03-12T17:07:33.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.352+0000 7f11ccc00640 1 -- 172.21.15.52:0/2603830786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11c80add30 msgr2=0x7f11c80ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:33.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.352+0000 7f11ccc00640 1 --2- 172.21.15.52:0/2603830786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11c80add30 0x7f11c80ced10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f11b40097a0 tx=0x7f11b402b8b0 comp rx=0 tx=0).stop 2024-03-12T17:07:33.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.352+0000 7f11ccc00640 1 -- 172.21.15.52:0/2603830786 shutdown_connections 2024-03-12T17:07:33.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.352+0000 7f11ccc00640 1 --2- 172.21.15.52:0/2603830786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11c80add30 0x7f11c80ced10 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:33.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.352+0000 7f11ccc00640 1 --2- 172.21.15.52:0/2603830786 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f11c806ce90 0x7f11c806d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:33.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.352+0000 7f11ccc00640 1 -- 172.21.15.52:0/2603830786 >> 172.21.15.52:0/2603830786 conn(0x7f11c806a0b0 msgr2=0x7f11c806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:33.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.352+0000 7f11ccc00640 1 -- 172.21.15.52:0/2603830786 shutdown_connections 2024-03-12T17:07:33.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.352+0000 7f11ccc00640 1 -- 172.21.15.52:0/2603830786 wait complete. 2024-03-12T17:07:33.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.354+0000 7f11ccc00640 1 Processor -- start 2024-03-12T17:07:33.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.354+0000 7f11ccc00640 1 -- start start 2024-03-12T17:07:33.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.355+0000 7f11ccc00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f11c806ce90 0x7f11c813f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:33.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.355+0000 7f11ccc00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11c80add30 0x7f11c8152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:33.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.355+0000 7f11ccc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11c80ce930 0x7f11c8152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:33.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.355+0000 7f11ccc00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c8143d40 con 0x7f11c80add30 2024-03-12T17:07:33.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.355+0000 7f11ccc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c813f8c0 con 0x7f11c80ce930 2024-03-12T17:07:33.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.355+0000 7f11ccc00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c81430c0 con 0x7f11c806ce90 2024-03-12T17:07:33.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.355+0000 7f11c5400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11c80add30 0x7f11c8152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:33.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.355+0000 7f11c5400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11c80add30 0x7f11c8152110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51174/0 (socket says 172.21.15.52:51174) 2024-03-12T17:07:33.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.355+0000 7f11c5400640 1 -- 172.21.15.52:0/3570828897 learned_addr learned my addr 172.21.15.52:0/3570828897 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:33.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.356+0000 7f11c5400640 1 -- 172.21.15.52:0/3570828897 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f11c806ce90 msgr2=0x7f11c813f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:33.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.356+0000 7f11c5400640 1 --2- 172.21.15.52:0/3570828897 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f11c806ce90 0x7f11c813f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:33.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.356+0000 7f11c5400640 1 -- 172.21.15.52:0/3570828897 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11c80ce930 msgr2=0x7f11c8152650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:33.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.356+0000 7f11c5400640 1 --2- 172.21.15.52:0/3570828897 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11c80ce930 0x7f11c8152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:33.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.356+0000 7f11c5400640 1 -- 172.21.15.52:0/3570828897 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b4009400 con 0x7f11c80add30 2024-03-12T17:07:33.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.356+0000 7f11c5400640 1 --2- 172.21.15.52:0/3570828897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11c80add30 0x7f11c8152110 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f11b40098c0 tx=0x7f11b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:33.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.356+0000 7f11be000640 1 -- 172.21.15.52:0/3570828897 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b403ad60 con 0x7f11c80add30 2024-03-12T17:07:33.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.356+0000 7f11be000640 1 -- 172.21.15.52:0/3570828897 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b4044400 con 0x7f11c80add30 2024-03-12T17:07:33.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.356+0000 7f11be000640 1 -- 172.21.15.52:0/3570828897 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b40447b0 con 0x7f11c80add30 2024-03-12T17:07:33.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.356+0000 7f11ccc00640 1 -- 172.21.15.52:0/3570828897 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c81433a0 con 0x7f11c80add30 2024-03-12T17:07:33.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.357+0000 7f11ccc00640 1 -- 172.21.15.52:0/3570828897 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c8143960 con 0x7f11c80add30 2024-03-12T17:07:33.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.358+0000 7f11ccc00640 1 -- 172.21.15.52:0/3570828897 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c8143e80 con 0x7f11c80add30 2024-03-12T17:07:33.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.359+0000 7f11be000640 1 -- 172.21.15.52:0/3570828897 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f11b404d440 con 0x7f11c80add30 2024-03-12T17:07:33.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.360+0000 7f11be000640 1 --2- 172.21.15.52:0/3570828897 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f11a806ac00 0x7f11a806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:33.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.360+0000 7f11be000640 1 -- 172.21.15.52:0/3570828897 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f11b40abb60 con 0x7f11c80add30 2024-03-12T17:07:33.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.360+0000 7f11c5e00640 1 --2- 172.21.15.52:0/3570828897 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f11a806ac00 0x7f11a806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:33.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.361+0000 7f11c5e00640 1 --2- 172.21.15.52:0/3570828897 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f11a806ac00 0x7f11a806d0c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f11b8007fd0 tx=0x7f11b8007960 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:33.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.364+0000 7f11be000640 1 -- 172.21.15.52:0/3570828897 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f11c8143e80 con 0x7f11c80add30 2024-03-12T17:07:33.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.563+0000 7f11ccc00640 1 -- 172.21.15.52:0/3570828897 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f11c8143e80 con 0x7f11c80add30 2024-03-12T17:07:33.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.568+0000 7f11be000640 1 -- 172.21.15.52:0/3570828897 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f11c8143e80 con 0x7f11c80add30 2024-03-12T17:07:33.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.571+0000 7f11ccc00640 1 -- 172.21.15.52:0/3570828897 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f11a806ac00 msgr2=0x7f11a806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:33.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.571+0000 7f11ccc00640 1 --2- 172.21.15.52:0/3570828897 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f11a806ac00 0x7f11a806d0c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f11b8007fd0 tx=0x7f11b8007960 comp rx=0 tx=0).stop 2024-03-12T17:07:33.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.571+0000 7f11ccc00640 1 -- 172.21.15.52:0/3570828897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11c80add30 msgr2=0x7f11c8152110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:33.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.571+0000 7f11ccc00640 1 --2- 172.21.15.52:0/3570828897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11c80add30 0x7f11c8152110 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f11b40098c0 tx=0x7f11b40046c0 comp rx=0 tx=0).stop 2024-03-12T17:07:33.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.571+0000 7f11ccc00640 1 -- 172.21.15.52:0/3570828897 shutdown_connections 2024-03-12T17:07:33.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.571+0000 7f11ccc00640 1 --2- 172.21.15.52:0/3570828897 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f11a806ac00 0x7f11a806d0c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:33.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.571+0000 7f11ccc00640 1 --2- 172.21.15.52:0/3570828897 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11c80ce930 0x7f11c8152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:33.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.571+0000 7f11ccc00640 1 --2- 172.21.15.52:0/3570828897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11c80add30 0x7f11c8152110 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:33.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.571+0000 7f11ccc00640 1 --2- 172.21.15.52:0/3570828897 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f11c806ce90 0x7f11c813f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:33.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.571+0000 7f11ccc00640 1 -- 172.21.15.52:0/3570828897 >> 172.21.15.52:0/3570828897 conn(0x7f11c806a0b0 msgr2=0x7f11c806a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:33.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.571+0000 7f11ccc00640 1 -- 172.21.15.52:0/3570828897 shutdown_connections 2024-03-12T17:07:33.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:33.571+0000 7f11ccc00640 1 -- 172.21.15.52:0/3570828897 wait complete. 2024-03-12T17:07:33.584 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-12T17:07:33.584 DEBUG:teuthology.orchestra.run.smithi052:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-12T17:07:33.693 DEBUG:teuthology.orchestra.run.smithi102:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-12T17:07:33.815 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-12T17:07:33.815 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:07:33.815 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-12T17:07:33.876 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:07:33.876 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-12T17:07:33.931 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:07:33.931 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-12T17:07:33.988 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:07:34.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.207+0000 7f63ab200640 1 Processor -- start 2024-03-12T17:07:34.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.207+0000 7f63ab200640 1 -- start start 2024-03-12T17:07:34.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.208+0000 7f63ab200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f63ac098a40 0x7f63ac06ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:34.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.209+0000 7f63ab200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63ac099780 0x7f63ac098e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:34.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.209+0000 7f63ab200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f63ac15c160 0x7f63ac15c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:34.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.209+0000 7f63ab200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63ac0cf2d0 con 0x7f63ac098a40 2024-03-12T17:07:34.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.209+0000 7f63ab200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63ac099be0 con 0x7f63ac15c160 2024-03-12T17:07:34.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.209+0000 7f63ab200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac099360 con 0x7f63ac099780 2024-03-12T17:07:34.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.209+0000 7f63a9e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f63ac098a40 0x7f63ac06ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:34.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.209+0000 7f63a9e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f63ac098a40 0x7f63ac06ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51182/0 (socket says 172.21.15.52:51182) 2024-03-12T17:07:34.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.209+0000 7f63a9e00640 1 -- 172.21.15.52:0/1466007459 learned_addr learned my addr 172.21.15.52:0/1466007459 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:34.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.209+0000 7f63a9400640 1 --2- 172.21.15.52:0/1466007459 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63ac099780 0x7f63ac098e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:34.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.210+0000 7f63a9e00640 1 -- 172.21.15.52:0/1466007459 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63ac099780 msgr2=0x7f63ac098e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:34.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.210+0000 7f63a9e00640 1 --2- 172.21.15.52:0/1466007459 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63ac099780 0x7f63ac098e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:34.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.210+0000 7f63a9e00640 1 -- 172.21.15.52:0/1466007459 >> v2:172.21.15.102:3300/0 conn(0x7f63ac15c160 msgr2=0x7f63ac15c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:34.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.210+0000 7f63aa800640 1 --2- 172.21.15.52:0/1466007459 >> v2:172.21.15.102:3300/0 conn(0x7f63ac15c160 0x7f63ac15c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:34.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.210+0000 7f63a9e00640 1 --2- 172.21.15.52:0/1466007459 >> v2:172.21.15.102:3300/0 conn(0x7f63ac15c160 0x7f63ac15c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:34.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.210+0000 7f63a9e00640 1 -- 172.21.15.52:0/1466007459 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ac06f260 con 0x7f63ac098a40 2024-03-12T17:07:34.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.210+0000 7f63aa800640 1 --2- 172.21.15.52:0/1466007459 >> v2:172.21.15.102:3300/0 conn(0x7f63ac15c160 0x7f63ac15c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:34.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.211+0000 7f63a9e00640 1 --2- 172.21.15.52:0/1466007459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63ac098a40 0x7f63ac06ec10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f63ac071cd0 tx=0x7f639402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=849089e9485c6bdf server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:34.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.211+0000 7f63a8a00640 1 -- 172.21.15.52:0/1466007459 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6394031860 con 0x7f63ac098a40 2024-03-12T17:07:34.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.212+0000 7f63a8a00640 1 -- 172.21.15.52:0/1466007459 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63940319c0 con 0x7f63ac098a40 2024-03-12T17:07:34.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.212+0000 7f63a8a00640 1 -- 172.21.15.52:0/1466007459 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6394031d50 con 0x7f63ac098a40 2024-03-12T17:07:34.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.212+0000 7f63ab200640 1 -- 172.21.15.52:0/1466007459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63ac098a40 msgr2=0x7f63ac06ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:34.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.212+0000 7f63ab200640 1 --2- 172.21.15.52:0/1466007459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63ac098a40 0x7f63ac06ec10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f63ac071cd0 tx=0x7f639402b900 comp rx=0 tx=0).stop 2024-03-12T17:07:34.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.212+0000 7f63ab200640 1 -- 172.21.15.52:0/1466007459 shutdown_connections 2024-03-12T17:07:34.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.212+0000 7f63ab200640 1 --2- 172.21.15.52:0/1466007459 >> v2:172.21.15.102:3300/0 conn(0x7f63ac15c160 0x7f63ac15c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:34.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.212+0000 7f63ab200640 1 --2- 172.21.15.52:0/1466007459 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63ac099780 0x7f63ac098e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:34.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.212+0000 7f63ab200640 1 --2- 172.21.15.52:0/1466007459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63ac098a40 0x7f63ac06ec10 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:34.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.212+0000 7f63ab200640 1 -- 172.21.15.52:0/1466007459 >> 172.21.15.52:0/1466007459 conn(0x7f63ac0b1fe0 msgr2=0x7f63ac0b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:34.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.212+0000 7f63ab200640 1 -- 172.21.15.52:0/1466007459 shutdown_connections 2024-03-12T17:07:34.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.213+0000 7f63ab200640 1 -- 172.21.15.52:0/1466007459 wait complete. 2024-03-12T17:07:34.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.215+0000 7f63ab200640 1 Processor -- start 2024-03-12T17:07:34.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.215+0000 7f63ab200640 1 -- start start 2024-03-12T17:07:34.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.216+0000 7f63ab200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63ac098a40 0x7f63ac14ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:34.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.216+0000 7f63ab200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63ac099780 0x7f63ac0b5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:34.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.216+0000 7f63ab200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63ac15c160 0x7f63ac15b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:34.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.216+0000 7f63ab200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac0b5eb0 con 0x7f63ac098a40 2024-03-12T17:07:34.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.216+0000 7f63ab200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac0c99c0 con 0x7f63ac15c160 2024-03-12T17:07:34.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.216+0000 7f63ab200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac0c9ca0 con 0x7f63ac099780 2024-03-12T17:07:34.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.216+0000 7f63a9e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63ac098a40 0x7f63ac14ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:34.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.216+0000 7f63a9e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63ac098a40 0x7f63ac14ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51190/0 (socket says 172.21.15.52:51190) 2024-03-12T17:07:34.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.216+0000 7f63a9e00640 1 -- 172.21.15.52:0/1759954026 learned_addr learned my addr 172.21.15.52:0/1759954026 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:34.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.216+0000 7f63a9400640 1 --2- 172.21.15.52:0/1759954026 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63ac099780 0x7f63ac0b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:34.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.216+0000 7f63aa800640 1 --2- 172.21.15.52:0/1759954026 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63ac15c160 0x7f63ac15b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:34.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.216+0000 7f63a9e00640 1 -- 172.21.15.52:0/1759954026 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63ac099780 msgr2=0x7f63ac0b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:34.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.217+0000 7f63a9e00640 1 --2- 172.21.15.52:0/1759954026 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63ac099780 0x7f63ac0b5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:34.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.217+0000 7f63a9e00640 1 -- 172.21.15.52:0/1759954026 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63ac15c160 msgr2=0x7f63ac15b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:34.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.217+0000 7f63a9e00640 1 --2- 172.21.15.52:0/1759954026 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63ac15c160 0x7f63ac15b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:34.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.217+0000 7f63a9e00640 1 -- 172.21.15.52:0/1759954026 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63940094d0 con 0x7f63ac098a40 2024-03-12T17:07:34.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.217+0000 7f63a9e00640 1 --2- 172.21.15.52:0/1759954026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63ac098a40 0x7f63ac14ae80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6394000c00 tx=0x7f63940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:34.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.217+0000 7f63a2000640 1 -- 172.21.15.52:0/1759954026 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6394031830 con 0x7f63ac098a40 2024-03-12T17:07:34.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.217+0000 7f63a2000640 1 -- 172.21.15.52:0/1759954026 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63940443f0 con 0x7f63ac098a40 2024-03-12T17:07:34.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.217+0000 7f63a2000640 1 -- 172.21.15.52:0/1759954026 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63940447a0 con 0x7f63ac098a40 2024-03-12T17:07:34.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.217+0000 7f63ab200640 1 -- 172.21.15.52:0/1759954026 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63ac0c9f80 con 0x7f63ac098a40 2024-03-12T17:07:34.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.218+0000 7f63ab200640 1 -- 172.21.15.52:0/1759954026 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63ac0ca540 con 0x7f63ac098a40 2024-03-12T17:07:34.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.219+0000 7f63ab200640 1 -- 172.21.15.52:0/1759954026 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63ac0c9de0 con 0x7f63ac098a40 2024-03-12T17:07:34.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.220+0000 7f63a2000640 1 -- 172.21.15.52:0/1759954026 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f639404d450 con 0x7f63ac098a40 2024-03-12T17:07:34.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.220+0000 7f63a2000640 1 --2- 172.21.15.52:0/1759954026 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f638806ac00 0x7f638806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:34.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.220+0000 7f63a2000640 1 -- 172.21.15.52:0/1759954026 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f63940abf50 con 0x7f63ac098a40 2024-03-12T17:07:34.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.221+0000 7f63a9400640 1 --2- 172.21.15.52:0/1759954026 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f638806ac00 0x7f638806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:34.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.222+0000 7f63a9400640 1 --2- 172.21.15.52:0/1759954026 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f638806ac00 0x7f638806d0c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f63ac1f3cf0 tx=0x7f639c0083a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:34.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.225+0000 7f63a2000640 1 -- 172.21.15.52:0/1759954026 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f63ac0c9de0 con 0x7f63ac098a40 2024-03-12T17:07:34.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.424+0000 7f63ab200640 1 -- 172.21.15.52:0/1759954026 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63ac0c9de0 con 0x7f63ac098a40 2024-03-12T17:07:34.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.427+0000 7f63a2000640 1 -- 172.21.15.52:0/1759954026 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18207 (secure 0 0 0) 0x7f63ac0c9de0 con 0x7f63ac098a40 2024-03-12T17:07:34.422 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:07:34.422 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":27,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:07:12.969614+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T17:06:19.218543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T17:06:23.525474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T17:06:24.911574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-12T17:06:28.017172+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.733031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.775178+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.591522+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.820838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.829252+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.582481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.18","mappings":[{"from":3,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:07:34.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.430+0000 7f63ab200640 1 -- 172.21.15.52:0/1759954026 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f638806ac00 msgr2=0x7f638806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:34.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.430+0000 7f63ab200640 1 --2- 172.21.15.52:0/1759954026 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f638806ac00 0x7f638806d0c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f63ac1f3cf0 tx=0x7f639c0083a0 comp rx=0 tx=0).stop 2024-03-12T17:07:34.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.430+0000 7f63ab200640 1 -- 172.21.15.52:0/1759954026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63ac098a40 msgr2=0x7f63ac14ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:34.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.430+0000 7f63ab200640 1 --2- 172.21.15.52:0/1759954026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63ac098a40 0x7f63ac14ae80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6394000c00 tx=0x7f63940046c0 comp rx=0 tx=0).stop 2024-03-12T17:07:34.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.430+0000 7f63ab200640 1 -- 172.21.15.52:0/1759954026 shutdown_connections 2024-03-12T17:07:34.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.430+0000 7f63ab200640 1 --2- 172.21.15.52:0/1759954026 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f638806ac00 0x7f638806d0c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:34.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.430+0000 7f63ab200640 1 --2- 172.21.15.52:0/1759954026 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63ac099780 0x7f63ac0b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:34.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.430+0000 7f63ab200640 1 --2- 172.21.15.52:0/1759954026 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63ac15c160 0x7f63ac15b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:34.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.430+0000 7f63ab200640 1 --2- 172.21.15.52:0/1759954026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63ac098a40 0x7f63ac14ae80 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:34.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.430+0000 7f63ab200640 1 -- 172.21.15.52:0/1759954026 >> 172.21.15.52:0/1759954026 conn(0x7f63ac0b1fe0 msgr2=0x7f63ac06ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:34.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.430+0000 7f63ab200640 1 -- 172.21.15.52:0/1759954026 shutdown_connections 2024-03-12T17:07:34.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.431+0000 7f63ab200640 1 -- 172.21.15.52:0/1759954026 wait complete. 2024-03-12T17:07:34.445 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T17:06:19.218543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-12T17:06:23.525474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-12T17:06:24.911574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-12T17:06:28.017172+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-12T17:07:34.446 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T17:07:34.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.679+0000 7f3576c00640 1 Processor -- start 2024-03-12T17:07:34.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.679+0000 7f3576c00640 1 -- start start 2024-03-12T17:07:34.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.680+0000 7f3576c00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f35700d0d40 0x7f35700b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:34.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.680+0000 7f3576c00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35700d20c0 con 0x7f35700cd740 2024-03-12T17:07:34.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.680+0000 7f3576c00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35700b8940 con 0x7f35700b84b0 2024-03-12T17:07:34.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.680+0000 7f3576c00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35700d14b0 con 0x7f35700d0d40 2024-03-12T17:07:34.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.681+0000 7f356fe00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f35700cd740 0x7f35700cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:52390/0 (socket says 172.21.15.52:52390) 2024-03-12T17:07:34.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.681+0000 7f356fe00640 1 -- 172.21.15.52:0/1539125805 learned_addr learned my addr 172.21.15.52:0/1539125805 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:34.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.681+0000 7f3574c00640 1 --2- 172.21.15.52:0/1539125805 >> v2:172.21.15.52:3300/0 conn(0x7f35700d0d40 0x7f35700b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:34.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.681+0000 7f3574c00640 1 -- 172.21.15.52:0/1539125805 >> v1:172.21.15.102:6789/0 conn(0x7f35700b84b0 legacy=0x7f35700b5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:34.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.681+0000 7f3574c00640 1 -- 172.21.15.52:0/1539125805 >> v1:172.21.15.52:6789/0 conn(0x7f35700cd740 legacy=0x7f35700cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:34.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.681+0000 7f356f400640 1 -- 172.21.15.52:0/1539125805 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 765756386 0 0) 0x7f35700d20c0 con 0x7f35700cd740 2024-03-12T17:07:34.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.681+0000 7f3574c00640 1 -- 172.21.15.52:0/1539125805 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35700d1790 con 0x7f35700d0d40 2024-03-12T17:07:34.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.683+0000 7f3574c00640 1 --2- 172.21.15.52:0/1539125805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35700d0d40 0x7f35700b61b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3568009d90 tx=0x7f356802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e8743f5cb186d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:34.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.683+0000 7f356f400640 1 -- 172.21.15.52:0/1539125805 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356802cc60 con 0x7f35700d0d40 2024-03-12T17:07:34.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.683+0000 7f356f400640 1 -- 172.21.15.52:0/1539125805 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f356802cdc0 con 0x7f35700d0d40 2024-03-12T17:07:34.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.683+0000 7f356f400640 1 -- 172.21.15.52:0/1539125805 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3568034660 con 0x7f35700d0d40 2024-03-12T17:07:34.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.684+0000 7f3576c00640 1 -- 172.21.15.52:0/1539125805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35700d0d40 msgr2=0x7f35700b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:34.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.684+0000 7f3576c00640 1 --2- 172.21.15.52:0/1539125805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35700d0d40 0x7f35700b61b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3568009d90 tx=0x7f356802beb0 comp rx=0 tx=0).stop 2024-03-12T17:07:34.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.684+0000 7f3576c00640 1 -- 172.21.15.52:0/1539125805 shutdown_connections 2024-03-12T17:07:34.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.684+0000 7f3576c00640 1 --2- 172.21.15.52:0/1539125805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35700d0d40 0x7f35700b61b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:34.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.684+0000 7f3576c00640 1 -- 172.21.15.52:0/1539125805 >> 172.21.15.52:0/1539125805 conn(0x7f35700cf5d0 msgr2=0x7f35700cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:34.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.684+0000 7f3576c00640 1 -- 172.21.15.52:0/1539125805 shutdown_connections 2024-03-12T17:07:34.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.684+0000 7f3576c00640 1 -- 172.21.15.52:0/1539125805 wait complete. 2024-03-12T17:07:34.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.686+0000 7f3576c00640 1 Processor -- start 2024-03-12T17:07:34.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.686+0000 7f3576c00640 1 -- start start 2024-03-12T17:07:34.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.687+0000 7f3576c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35700b84b0 0x7f3570153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:34.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.687+0000 7f3576c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35700cd740 0x7f35700cbc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:34.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.687+0000 7f3576c00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35700d0d40 0x7f35700cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:34.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.687+0000 7f3576c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35701542f0 con 0x7f35700cd740 2024-03-12T17:07:34.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.687+0000 7f3576c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35700cb460 con 0x7f35700b84b0 2024-03-12T17:07:34.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.687+0000 7f3576c00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3570154720 con 0x7f35700d0d40 2024-03-12T17:07:34.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.687+0000 7f3567e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35700cd740 0x7f35700cbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:34.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.687+0000 7f3567e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35700cd740 0x7f35700cbc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51204/0 (socket says 172.21.15.52:51204) 2024-03-12T17:07:34.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.687+0000 7f3567e00640 1 -- 172.21.15.52:0/493194674 learned_addr learned my addr 172.21.15.52:0/493194674 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:34.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.688+0000 7f356fe00640 1 --2- 172.21.15.52:0/493194674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35700b84b0 0x7f3570153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:34.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.688+0000 7f3574c00640 1 --2- 172.21.15.52:0/493194674 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35700d0d40 0x7f35700cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:34.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.688+0000 7f3567e00640 1 -- 172.21.15.52:0/493194674 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35700d0d40 msgr2=0x7f35700cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:34.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.688+0000 7f3567e00640 1 --2- 172.21.15.52:0/493194674 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35700d0d40 0x7f35700cc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:34.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.688+0000 7f3567e00640 1 -- 172.21.15.52:0/493194674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35700b84b0 msgr2=0x7f3570153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:34.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.688+0000 7f3567e00640 1 --2- 172.21.15.52:0/493194674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35700b84b0 0x7f3570153db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:34.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.688+0000 7f3567e00640 1 -- 172.21.15.52:0/493194674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3568009a70 con 0x7f35700cd740 2024-03-12T17:07:34.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.688+0000 7f356fe00640 1 --2- 172.21.15.52:0/493194674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35700b84b0 0x7f3570153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:07:34.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.688+0000 7f3567e00640 1 --2- 172.21.15.52:0/493194674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35700cd740 0x7f35700cbc60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f355800c430 tx=0x7f355800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:34.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.689+0000 7f356cc00640 1 -- 172.21.15.52:0/493194674 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3558011790 con 0x7f35700cd740 2024-03-12T17:07:34.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.689+0000 7f356cc00640 1 -- 172.21.15.52:0/493194674 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35580118f0 con 0x7f35700cd740 2024-03-12T17:07:34.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.689+0000 7f356cc00640 1 -- 172.21.15.52:0/493194674 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3558011ce0 con 0x7f35700cd740 2024-03-12T17:07:34.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.689+0000 7f3576c00640 1 -- 172.21.15.52:0/493194674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35700cc7b0 con 0x7f35700cd740 2024-03-12T17:07:34.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.689+0000 7f3576c00640 1 -- 172.21.15.52:0/493194674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35700cae10 con 0x7f35700cd740 2024-03-12T17:07:34.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.690+0000 7f3576c00640 1 -- 172.21.15.52:0/493194674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3570073390 con 0x7f35700cd740 2024-03-12T17:07:34.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.691+0000 7f356cc00640 1 -- 172.21.15.52:0/493194674 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f355801c460 con 0x7f35700cd740 2024-03-12T17:07:34.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.693+0000 7f356cc00640 1 --2- 172.21.15.52:0/493194674 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f354c06ac00 0x7f354c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:34.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.693+0000 7f356cc00640 1 -- 172.21.15.52:0/493194674 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3558089710 con 0x7f35700cd740 2024-03-12T17:07:34.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.693+0000 7f356fe00640 1 --2- 172.21.15.52:0/493194674 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f354c06ac00 0x7f354c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:34.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.695+0000 7f356fe00640 1 --2- 172.21.15.52:0/493194674 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f354c06ac00 0x7f354c06d0c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3560000f60 tx=0x7f3560007c10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:34.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.696+0000 7f356cc00640 1 -- 172.21.15.52:0/493194674 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f3558054490 con 0x7f35700cd740 2024-03-12T17:07:34.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.896+0000 7f3576c00640 1 -- 172.21.15.52:0/493194674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3570158060 con 0x7f35700cd740 2024-03-12T17:07:34.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.897+0000 7f356cc00640 1 -- 172.21.15.52:0/493194674 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3558057ab0 con 0x7f35700cd740 2024-03-12T17:07:34.891 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-03-12T17:07:34.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.900+0000 7f3576c00640 1 -- 172.21.15.52:0/493194674 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f354c06ac00 msgr2=0x7f354c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:34.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.900+0000 7f3576c00640 1 --2- 172.21.15.52:0/493194674 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f354c06ac00 0x7f354c06d0c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3560000f60 tx=0x7f3560007c10 comp rx=0 tx=0).stop 2024-03-12T17:07:34.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.900+0000 7f3576c00640 1 -- 172.21.15.52:0/493194674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35700cd740 msgr2=0x7f35700cbc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:34.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.900+0000 7f3576c00640 1 --2- 172.21.15.52:0/493194674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35700cd740 0x7f35700cbc60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f355800c430 tx=0x7f355800c8f0 comp rx=0 tx=0).stop 2024-03-12T17:07:34.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.900+0000 7f3576c00640 1 -- 172.21.15.52:0/493194674 shutdown_connections 2024-03-12T17:07:34.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.900+0000 7f3576c00640 1 --2- 172.21.15.52:0/493194674 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f354c06ac00 0x7f354c06d0c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:34.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.900+0000 7f3576c00640 1 --2- 172.21.15.52:0/493194674 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35700d0d40 0x7f35700cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:34.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.900+0000 7f3576c00640 1 --2- 172.21.15.52:0/493194674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35700cd740 0x7f35700cbc60 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:34.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.900+0000 7f3576c00640 1 --2- 172.21.15.52:0/493194674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35700b84b0 0x7f3570153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:34.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.900+0000 7f3576c00640 1 -- 172.21.15.52:0/493194674 >> 172.21.15.52:0/493194674 conn(0x7f35700cf5d0 msgr2=0x7f35700c8de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:34.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.900+0000 7f3576c00640 1 -- 172.21.15.52:0/493194674 shutdown_connections 2024-03-12T17:07:34.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:34.900+0000 7f3576c00640 1 -- 172.21.15.52:0/493194674 wait complete. 2024-03-12T17:07:34.913 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-12T17:07:35.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.138+0000 7f0089200640 1 Processor -- start 2024-03-12T17:07:35.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.138+0000 7f0089200640 1 -- start start 2024-03-12T17:07:35.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.140+0000 7f0089200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f00841441a0 0x7f00840ae750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:35.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.140+0000 7f0089200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f00840ac3a0 0x7f0084069c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:35.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.140+0000 7f0089200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f008406a1d0 0x7f00840cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:35.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.140+0000 7f0089200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f008406a660 con 0x7f008406a1d0 2024-03-12T17:07:35.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.140+0000 7f0089200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00840cbd40 con 0x7f00840ac3a0 2024-03-12T17:07:35.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.140+0000 7f0089200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00840aed60 con 0x7f00841441a0 2024-03-12T17:07:35.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.140+0000 7f0083e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f008406a1d0 0x7f00840cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:35.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.140+0000 7f0083e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f008406a1d0 0x7f00840cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51596/0 (socket says 172.21.15.52:51596) 2024-03-12T17:07:35.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.140+0000 7f0083e00640 1 -- 172.21.15.52:0/1329929257 learned_addr learned my addr 172.21.15.52:0/1329929257 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:35.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.140+0000 7f0083400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f00841441a0 0x7f00840ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:35.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.141+0000 7f0082a00640 1 --2- 172.21.15.52:0/1329929257 >> v2:172.21.15.102:3300/0 conn(0x7f00840ac3a0 0x7f0084069c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:35.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.141+0000 7f0083e00640 1 -- 172.21.15.52:0/1329929257 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f00841441a0 msgr2=0x7f00840ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:35.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.141+0000 7f0083e00640 1 --2- 172.21.15.52:0/1329929257 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f00841441a0 0x7f00840ae750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:35.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.141+0000 7f0083e00640 1 -- 172.21.15.52:0/1329929257 >> v2:172.21.15.102:3300/0 conn(0x7f00840ac3a0 msgr2=0x7f0084069c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:35.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.141+0000 7f0083e00640 1 --2- 172.21.15.52:0/1329929257 >> v2:172.21.15.102:3300/0 conn(0x7f00840ac3a0 0x7f0084069c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:35.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.141+0000 7f0083e00640 1 -- 172.21.15.52:0/1329929257 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00840af040 con 0x7f008406a1d0 2024-03-12T17:07:35.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.141+0000 7f0083400640 1 --2- 172.21.15.52:0/1329929257 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f00841441a0 0x7f00840ae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:07:35.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.142+0000 7f0083e00640 1 --2- 172.21.15.52:0/1329929257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f008406a1d0 0x7f00840cf330 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f00840d0410 tx=0x7f007402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d56f622d31731513 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:35.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.142+0000 7f0082000640 1 -- 172.21.15.52:0/1329929257 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0074031870 con 0x7f008406a1d0 2024-03-12T17:07:35.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.143+0000 7f0082000640 1 -- 172.21.15.52:0/1329929257 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00740319d0 con 0x7f008406a1d0 2024-03-12T17:07:35.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.143+0000 7f0082000640 1 -- 172.21.15.52:0/1329929257 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0074031d60 con 0x7f008406a1d0 2024-03-12T17:07:35.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.143+0000 7f0089200640 1 -- 172.21.15.52:0/1329929257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f008406a1d0 msgr2=0x7f00840cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:35.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.143+0000 7f0089200640 1 --2- 172.21.15.52:0/1329929257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f008406a1d0 0x7f00840cf330 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f00840d0410 tx=0x7f007402b910 comp rx=0 tx=0).stop 2024-03-12T17:07:35.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.143+0000 7f0089200640 1 -- 172.21.15.52:0/1329929257 shutdown_connections 2024-03-12T17:07:35.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.143+0000 7f0089200640 1 --2- 172.21.15.52:0/1329929257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f008406a1d0 0x7f00840cf330 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:35.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.143+0000 7f0089200640 1 --2- 172.21.15.52:0/1329929257 >> v2:172.21.15.102:3300/0 conn(0x7f00840ac3a0 0x7f0084069c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:35.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.144+0000 7f0089200640 1 --2- 172.21.15.52:0/1329929257 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f00841441a0 0x7f00840ae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:35.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.144+0000 7f0089200640 1 -- 172.21.15.52:0/1329929257 >> 172.21.15.52:0/1329929257 conn(0x7f00840b48e0 msgr2=0x7f00840b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:35.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.144+0000 7f0089200640 1 -- 172.21.15.52:0/1329929257 shutdown_connections 2024-03-12T17:07:35.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.144+0000 7f0089200640 1 -- 172.21.15.52:0/1329929257 wait complete. 2024-03-12T17:07:35.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.145+0000 7f0089200640 1 Processor -- start 2024-03-12T17:07:35.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.146+0000 7f0089200640 1 -- start start 2024-03-12T17:07:35.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.146+0000 7f0089200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f008406a1d0 0x7f008415a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:35.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.147+0000 7f0089200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00840ac3a0 0x7f00841f2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:35.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.147+0000 7f0089200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00841441a0 0x7f00840ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:35.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.147+0000 7f0089200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00840cabd0 con 0x7f00840ac3a0 2024-03-12T17:07:35.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.147+0000 7f0089200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00841f2130 con 0x7f00841441a0 2024-03-12T17:07:35.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.147+0000 7f0089200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00841f2410 con 0x7f008406a1d0 2024-03-12T17:07:35.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.147+0000 7f0082a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00840ac3a0 0x7f00841f2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:35.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.147+0000 7f0082a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00840ac3a0 0x7f00841f2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51598/0 (socket says 172.21.15.52:51598) 2024-03-12T17:07:35.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.147+0000 7f0082a00640 1 -- 172.21.15.52:0/2000677908 learned_addr learned my addr 172.21.15.52:0/2000677908 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:35.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.147+0000 7f0083e00640 1 --2- 172.21.15.52:0/2000677908 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00841441a0 0x7f00840ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:35.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.147+0000 7f0083400640 1 --2- 172.21.15.52:0/2000677908 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f008406a1d0 0x7f008415a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:35.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.147+0000 7f0082a00640 1 -- 172.21.15.52:0/2000677908 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f008406a1d0 msgr2=0x7f008415a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:35.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.147+0000 7f0082a00640 1 --2- 172.21.15.52:0/2000677908 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f008406a1d0 0x7f008415a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:35.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.147+0000 7f0082a00640 1 -- 172.21.15.52:0/2000677908 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00841441a0 msgr2=0x7f00840ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:35.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.147+0000 7f0082a00640 1 --2- 172.21.15.52:0/2000677908 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00841441a0 0x7f00840ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:35.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.147+0000 7f0082a00640 1 -- 172.21.15.52:0/2000677908 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00740094d0 con 0x7f00840ac3a0 2024-03-12T17:07:35.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.147+0000 7f0083e00640 1 --2- 172.21.15.52:0/2000677908 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00841441a0 0x7f00840ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:07:35.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.148+0000 7f0082a00640 1 --2- 172.21.15.52:0/2000677908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00840ac3a0 0x7f00841f2c10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f006c00d800 tx=0x7f006c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:35.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.148+0000 7f007b400640 1 -- 172.21.15.52:0/2000677908 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c0096a0 con 0x7f00840ac3a0 2024-03-12T17:07:35.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.148+0000 7f007b400640 1 -- 172.21.15.52:0/2000677908 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006c009800 con 0x7f00840ac3a0 2024-03-12T17:07:35.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.148+0000 7f007b400640 1 -- 172.21.15.52:0/2000677908 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c009b90 con 0x7f00840ac3a0 2024-03-12T17:07:35.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.148+0000 7f0089200640 1 -- 172.21.15.52:0/2000677908 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00841f26f0 con 0x7f00840ac3a0 2024-03-12T17:07:35.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.148+0000 7f0089200640 1 -- 172.21.15.52:0/2000677908 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00840caeb0 con 0x7f00840ac3a0 2024-03-12T17:07:35.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.150+0000 7f0089200640 1 -- 172.21.15.52:0/2000677908 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f008404e030 con 0x7f00840ac3a0 2024-03-12T17:07:35.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.150+0000 7f007b400640 1 -- 172.21.15.52:0/2000677908 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f006c009cf0 con 0x7f00840ac3a0 2024-03-12T17:07:35.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.152+0000 7f007b400640 1 --2- 172.21.15.52:0/2000677908 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f006006ac30 0x7f006006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:35.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.152+0000 7f007b400640 1 -- 172.21.15.52:0/2000677908 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f006c089b90 con 0x7f00840ac3a0 2024-03-12T17:07:35.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.153+0000 7f0083400640 1 --2- 172.21.15.52:0/2000677908 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f006006ac30 0x7f006006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:35.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.154+0000 7f0083400640 1 --2- 172.21.15.52:0/2000677908 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f006006ac30 0x7f006006d0f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f007c003fd0 tx=0x7f007c003990 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:35.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.156+0000 7f007b400640 1 -- 172.21.15.52:0/2000677908 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f006c0549a0 con 0x7f00840ac3a0 2024-03-12T17:07:35.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.355+0000 7f0089200640 1 -- 172.21.15.52:0/2000677908 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f00840b5040 con 0x7f00840ac3a0 2024-03-12T17:07:35.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.356+0000 7f007b400640 1 -- 172.21.15.52:0/2000677908 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f006c00b440 con 0x7f00840ac3a0 2024-03-12T17:07:35.349 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-12T17:07:35.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.358+0000 7f0089200640 1 -- 172.21.15.52:0/2000677908 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f006006ac30 msgr2=0x7f006006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:35.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.358+0000 7f0089200640 1 --2- 172.21.15.52:0/2000677908 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f006006ac30 0x7f006006d0f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f007c003fd0 tx=0x7f007c003990 comp rx=0 tx=0).stop 2024-03-12T17:07:35.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.358+0000 7f0089200640 1 -- 172.21.15.52:0/2000677908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00840ac3a0 msgr2=0x7f00841f2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:35.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.358+0000 7f0089200640 1 --2- 172.21.15.52:0/2000677908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00840ac3a0 0x7f00841f2c10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f006c00d800 tx=0x7f006c00dcc0 comp rx=0 tx=0).stop 2024-03-12T17:07:35.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.358+0000 7f0089200640 1 -- 172.21.15.52:0/2000677908 shutdown_connections 2024-03-12T17:07:35.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.358+0000 7f0089200640 1 --2- 172.21.15.52:0/2000677908 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f006006ac30 0x7f006006d0f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:35.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.358+0000 7f0089200640 1 --2- 172.21.15.52:0/2000677908 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00841441a0 0x7f00840ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:35.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.358+0000 7f0089200640 1 --2- 172.21.15.52:0/2000677908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00840ac3a0 0x7f00841f2c10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:35.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.358+0000 7f0089200640 1 --2- 172.21.15.52:0/2000677908 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f008406a1d0 0x7f008415a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:35.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.358+0000 7f0089200640 1 -- 172.21.15.52:0/2000677908 >> 172.21.15.52:0/2000677908 conn(0x7f00840b48e0 msgr2=0x7f0084142dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:35.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.358+0000 7f0089200640 1 -- 172.21.15.52:0/2000677908 shutdown_connections 2024-03-12T17:07:35.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.359+0000 7f0089200640 1 -- 172.21.15.52:0/2000677908 wait complete. 2024-03-12T17:07:35.372 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-12T17:07:35.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.606+0000 7eff73600640 1 Processor -- start 2024-03-12T17:07:35.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.606+0000 7eff73600640 1 -- start start 2024-03-12T17:07:35.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.607+0000 7eff73600640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7eff6c155090 0x7eff6c0712a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:35.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.608+0000 7eff73600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7eff6c0d0290 0x7eff6c06dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:35.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.608+0000 7eff73600640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff6c15ae60 con 0x7eff6c06d860 2024-03-12T17:07:35.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.608+0000 7eff73600640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff6c06e250 con 0x7eff6c155090 2024-03-12T17:07:35.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.608+0000 7eff73600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff6c0ce8c0 con 0x7eff6c0d0290 2024-03-12T17:07:35.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.608+0000 7eff70c00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7eff6c155090 0x7eff6c0712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:35.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.608+0000 7eff70c00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7eff6c155090 0x7eff6c0712a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51610/0 (socket says 172.21.15.52:51610) 2024-03-12T17:07:35.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.608+0000 7eff70c00640 1 -- 172.21.15.52:0/1039570106 learned_addr learned my addr 172.21.15.52:0/1039570106 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:35.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.608+0000 7eff71600640 1 --2- 172.21.15.52:0/1039570106 >> v2:172.21.15.102:3300/0 conn(0x7eff6c0d0290 0x7eff6c06dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:35.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.608+0000 7eff70c00640 1 -- 172.21.15.52:0/1039570106 >> v2:172.21.15.102:3300/0 conn(0x7eff6c0d0290 msgr2=0x7eff6c06dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:35.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.608+0000 7eff70c00640 1 --2- 172.21.15.52:0/1039570106 >> v2:172.21.15.102:3300/0 conn(0x7eff6c0d0290 0x7eff6c06dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:35.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.608+0000 7eff70c00640 1 -- 172.21.15.52:0/1039570106 >> v1:172.21.15.102:6789/0 conn(0x7eff6c06d860 legacy=0x7eff6c0cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:35.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.609+0000 7eff70c00640 1 -- 172.21.15.52:0/1039570106 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff6c0ceba0 con 0x7eff6c155090 2024-03-12T17:07:35.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.610+0000 7eff70c00640 1 --2- 172.21.15.52:0/1039570106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff6c155090 0x7eff6c0712a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7eff6c06a570 tx=0x7eff6002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af14e1154b39d17f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:35.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.610+0000 7eff6b400640 1 -- 172.21.15.52:0/1039570106 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff60031870 con 0x7eff6c155090 2024-03-12T17:07:35.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.610+0000 7eff6b400640 1 -- 172.21.15.52:0/1039570106 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff600319d0 con 0x7eff6c155090 2024-03-12T17:07:35.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.610+0000 7eff6b400640 1 -- 172.21.15.52:0/1039570106 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff60031d60 con 0x7eff6c155090 2024-03-12T17:07:35.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.611+0000 7eff73600640 1 -- 172.21.15.52:0/1039570106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff6c155090 msgr2=0x7eff6c0712a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:35.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.611+0000 7eff73600640 1 --2- 172.21.15.52:0/1039570106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff6c155090 0x7eff6c0712a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7eff6c06a570 tx=0x7eff6002b910 comp rx=0 tx=0).stop 2024-03-12T17:07:35.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.611+0000 7eff73600640 1 -- 172.21.15.52:0/1039570106 shutdown_connections 2024-03-12T17:07:35.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.611+0000 7eff73600640 1 --2- 172.21.15.52:0/1039570106 >> v2:172.21.15.102:3300/0 conn(0x7eff6c0d0290 0x7eff6c06dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:35.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.611+0000 7eff73600640 1 --2- 172.21.15.52:0/1039570106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff6c155090 0x7eff6c0712a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:35.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.611+0000 7eff73600640 1 -- 172.21.15.52:0/1039570106 >> 172.21.15.52:0/1039570106 conn(0x7eff6c0b4c90 msgr2=0x7eff6c0b50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:35.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.611+0000 7eff73600640 1 -- 172.21.15.52:0/1039570106 shutdown_connections 2024-03-12T17:07:35.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.611+0000 7eff73600640 1 -- 172.21.15.52:0/1039570106 wait complete. 2024-03-12T17:07:35.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.613+0000 7eff73600640 1 Processor -- start 2024-03-12T17:07:35.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.613+0000 7eff73600640 1 -- start start 2024-03-12T17:07:35.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.614+0000 7eff73600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff6c06d860 0x7eff6c0ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:35.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.614+0000 7eff73600640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7eff6c0d0290 0x7eff6c146f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:35.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.614+0000 7eff73600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff6c155090 0x7eff6c0cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:35.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.614+0000 7eff73600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c0c9620 con 0x7eff6c155090 2024-03-12T17:07:35.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.614+0000 7eff73600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c0c9900 con 0x7eff6c06d860 2024-03-12T17:07:35.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.614+0000 7eff73600640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c0c9be0 con 0x7eff6c0d0290 2024-03-12T17:07:35.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.614+0000 7eff71600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff6c155090 0x7eff6c0cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:35.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.614+0000 7eff71600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff6c155090 0x7eff6c0cbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51626/0 (socket says 172.21.15.52:51626) 2024-03-12T17:07:35.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.614+0000 7eff71600640 1 -- 172.21.15.52:0/3434293625 learned_addr learned my addr 172.21.15.52:0/3434293625 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:35.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.614+0000 7eff70c00640 1 --2- 172.21.15.52:0/3434293625 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff6c06d860 0x7eff6c0ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:35.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.614+0000 7eff6be00640 1 --2- 172.21.15.52:0/3434293625 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7eff6c0d0290 0x7eff6c146f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:35.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.615+0000 7eff71600640 1 -- 172.21.15.52:0/3434293625 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7eff6c0d0290 msgr2=0x7eff6c146f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:35.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.615+0000 7eff71600640 1 --2- 172.21.15.52:0/3434293625 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7eff6c0d0290 0x7eff6c146f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:35.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.615+0000 7eff71600640 1 -- 172.21.15.52:0/3434293625 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff6c06d860 msgr2=0x7eff6c0ca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:35.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.615+0000 7eff71600640 1 --2- 172.21.15.52:0/3434293625 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff6c06d860 0x7eff6c0ca6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:35.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.615+0000 7eff71600640 1 -- 172.21.15.52:0/3434293625 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff600094d0 con 0x7eff6c155090 2024-03-12T17:07:35.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.615+0000 7eff71600640 1 --2- 172.21.15.52:0/3434293625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff6c155090 0x7eff6c0cbe70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7eff64007e40 tx=0x7eff6400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:35.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.615+0000 7eff68c00640 1 -- 172.21.15.52:0/3434293625 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff64014620 con 0x7eff6c155090 2024-03-12T17:07:35.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.615+0000 7eff68c00640 1 -- 172.21.15.52:0/3434293625 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff6400ee90 con 0x7eff6c155090 2024-03-12T17:07:35.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.615+0000 7eff68c00640 1 -- 172.21.15.52:0/3434293625 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff640149e0 con 0x7eff6c155090 2024-03-12T17:07:35.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.616+0000 7eff73600640 1 -- 172.21.15.52:0/3434293625 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff6c0c9ec0 con 0x7eff6c155090 2024-03-12T17:07:35.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.616+0000 7eff73600640 1 -- 172.21.15.52:0/3434293625 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff6c147450 con 0x7eff6c155090 2024-03-12T17:07:35.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.617+0000 7eff73600640 1 -- 172.21.15.52:0/3434293625 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff6c0ca3e0 con 0x7eff6c155090 2024-03-12T17:07:35.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.618+0000 7eff68c00640 1 -- 172.21.15.52:0/3434293625 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7eff64014b40 con 0x7eff6c155090 2024-03-12T17:07:35.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.618+0000 7eff68c00640 1 --2- 172.21.15.52:0/3434293625 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7eff4c06ac00 0x7eff4c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:35.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.619+0000 7eff68c00640 1 -- 172.21.15.52:0/3434293625 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7eff6408a140 con 0x7eff6c155090 2024-03-12T17:07:35.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.619+0000 7eff70c00640 1 --2- 172.21.15.52:0/3434293625 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7eff4c06ac00 0x7eff4c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:35.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.620+0000 7eff70c00640 1 --2- 172.21.15.52:0/3434293625 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7eff4c06ac00 0x7eff4c06d0c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7eff60036000 tx=0x7eff6003c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:35.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.623+0000 7eff68c00640 1 -- 172.21.15.52:0/3434293625 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7eff6c0ca3e0 con 0x7eff6c155090 2024-03-12T17:07:35.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.820+0000 7eff73600640 1 -- 172.21.15.52:0/3434293625 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7eff6c0ca3e0 con 0x7eff6c155090 2024-03-12T17:07:35.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.821+0000 7eff68c00640 1 -- 172.21.15.52:0/3434293625 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7eff6c0ca3e0 con 0x7eff6c155090 2024-03-12T17:07:35.815 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-03-12T17:07:35.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.824+0000 7eff73600640 1 -- 172.21.15.52:0/3434293625 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7eff4c06ac00 msgr2=0x7eff4c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:35.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.824+0000 7eff73600640 1 --2- 172.21.15.52:0/3434293625 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7eff4c06ac00 0x7eff4c06d0c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7eff60036000 tx=0x7eff6003c040 comp rx=0 tx=0).stop 2024-03-12T17:07:35.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.824+0000 7eff73600640 1 -- 172.21.15.52:0/3434293625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff6c155090 msgr2=0x7eff6c0cbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:35.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.824+0000 7eff73600640 1 --2- 172.21.15.52:0/3434293625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff6c155090 0x7eff6c0cbe70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7eff64007e40 tx=0x7eff6400e4b0 comp rx=0 tx=0).stop 2024-03-12T17:07:35.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.824+0000 7eff73600640 1 -- 172.21.15.52:0/3434293625 shutdown_connections 2024-03-12T17:07:35.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.824+0000 7eff73600640 1 --2- 172.21.15.52:0/3434293625 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7eff4c06ac00 0x7eff4c06d0c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:35.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.824+0000 7eff73600640 1 --2- 172.21.15.52:0/3434293625 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7eff6c0d0290 0x7eff6c146f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:35.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.824+0000 7eff73600640 1 --2- 172.21.15.52:0/3434293625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff6c155090 0x7eff6c0cbe70 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:35.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.824+0000 7eff73600640 1 --2- 172.21.15.52:0/3434293625 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff6c06d860 0x7eff6c0ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:35.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.824+0000 7eff73600640 1 -- 172.21.15.52:0/3434293625 >> 172.21.15.52:0/3434293625 conn(0x7eff6c0b4c90 msgr2=0x7eff6c15cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:35.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.824+0000 7eff73600640 1 -- 172.21.15.52:0/3434293625 shutdown_connections 2024-03-12T17:07:35.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:35.825+0000 7eff73600640 1 -- 172.21.15.52:0/3434293625 wait complete. 2024-03-12T17:07:35.838 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-12T17:07:36.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.067+0000 7f6c21a00640 1 Processor -- start 2024-03-12T17:07:36.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.067+0000 7f6c21a00640 1 -- start start 2024-03-12T17:07:36.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.069+0000 7f6c21a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c1c143930 0x7f6c1c0cd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:36.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.069+0000 7f6c21a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6c1c0cdc50 0x7f6c1c070020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:36.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.069+0000 7f6c21a00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c1c15b2b0 con 0x7f6c1c143460 2024-03-12T17:07:36.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.069+0000 7f6c21a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c1c0cc6b0 con 0x7f6c1c0cdc50 2024-03-12T17:07:36.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.069+0000 7f6c21a00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c0cc950 con 0x7f6c1c143930 2024-03-12T17:07:36.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.069+0000 7f6c1be00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f6c1c143460 0x7f6c1c15aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:44792/0 (socket says 172.21.15.52:44792) 2024-03-12T17:07:36.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.069+0000 7f6c1be00640 1 -- 172.21.15.52:0/1910377619 learned_addr learned my addr 172.21.15.52:0/1910377619 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:36.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.070+0000 7f6c21000640 1 --2- 172.21.15.52:0/1910377619 >> v2:172.21.15.102:3300/0 conn(0x7f6c1c0cdc50 0x7f6c1c070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:36.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.070+0000 7f6c13400640 1 --2- 172.21.15.52:0/1910377619 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c1c143930 0x7f6c1c0cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:36.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.070+0000 7f6c1b400640 1 -- 172.21.15.52:0/1910377619 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 50878279 0 0) 0x7f6c1c15b2b0 con 0x7f6c1c143460 2024-03-12T17:07:36.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.070+0000 7f6c1b400640 1 -- 172.21.15.52:0/1910377619 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c00003520 con 0x7f6c1c143460 2024-03-12T17:07:36.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.070+0000 7f6c1b400640 1 -- 172.21.15.52:0/1910377619 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3396731985 0 0) 0x7f6c00003520 con 0x7f6c1c143460 2024-03-12T17:07:36.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.070+0000 7f6c1b400640 1 -- 172.21.15.52:0/1910377619 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c1c143930 msgr2=0x7f6c1c0cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:36.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.070+0000 7f6c1b400640 1 --2- 172.21.15.52:0/1910377619 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c1c143930 0x7f6c1c0cd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:36.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.070+0000 7f6c1b400640 1 -- 172.21.15.52:0/1910377619 >> v2:172.21.15.102:3300/0 conn(0x7f6c1c0cdc50 msgr2=0x7f6c1c070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:36.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.070+0000 7f6c1b400640 1 --2- 172.21.15.52:0/1910377619 >> v2:172.21.15.102:3300/0 conn(0x7f6c1c0cdc50 0x7f6c1c070020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:36.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.070+0000 7f6c1b400640 1 -- 172.21.15.52:0/1910377619 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c0ccc30 con 0x7f6c1c143460 2024-03-12T17:07:36.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.070+0000 7f6c1b400640 1 -- 172.21.15.52:0/1910377619 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f6c0c001670 con 0x7f6c1c143460 2024-03-12T17:07:36.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.071+0000 7f6c1b400640 1 -- 172.21.15.52:0/1910377619 >> v1:172.21.15.52:6789/0 conn(0x7f6c1c143460 legacy=0x7f6c1c15aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:36.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.071+0000 7f6c1b400640 1 --2- 172.21.15.52:0/1910377619 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c1c143930 0x7f6c00003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:36.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.071+0000 7f6c1b400640 1 --2- 172.21.15.52:0/1910377619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c00003f40 0x7f6c000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:36.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.071+0000 7f6c1b400640 1 --2- 172.21.15.52:0/1910377619 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c00004d00 0x7f6c00005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:36.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.071+0000 7f6c1b400640 1 -- 172.21.15.52:0/1910377619 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c0cc6b0 con 0x7f6c00003f40 2024-03-12T17:07:36.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.071+0000 7f6c1b400640 1 -- 172.21.15.52:0/1910377619 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c0cc950 con 0x7f6c00004d00 2024-03-12T17:07:36.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.071+0000 7f6c1b400640 1 -- 172.21.15.52:0/1910377619 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0000ba00 con 0x7f6c1c143930 2024-03-12T17:07:36.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.071+0000 7f6c21000640 1 --2- 172.21.15.52:0/1910377619 >> v2:172.21.15.102:3300/0 conn(0x7f6c1c0cdc50 0x7f6c1c070020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T17:07:36.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.071+0000 7f6c13400640 1 --2- 172.21.15.52:0/1910377619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c00003f40 0x7f6c000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:36.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.071+0000 7f6c1be00640 1 --2- 172.21.15.52:0/1910377619 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c1c143930 0x7f6c00003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:36.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.071+0000 7f6c13400640 1 -- 172.21.15.52:0/1910377619 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c1c143930 msgr2=0x7f6c00003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:36.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.071+0000 7f6c13400640 1 --2- 172.21.15.52:0/1910377619 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c1c143930 0x7f6c00003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:36.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.071+0000 7f6c21000640 1 --2- 172.21.15.52:0/1910377619 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c00004d00 0x7f6c00005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:36.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.071+0000 7f6c13400640 1 -- 172.21.15.52:0/1910377619 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c00004d00 msgr2=0x7f6c00005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:36.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.071+0000 7f6c13400640 1 --2- 172.21.15.52:0/1910377619 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c00004d00 0x7f6c00005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:36.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.071+0000 7f6c13400640 1 -- 172.21.15.52:0/1910377619 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c00003520 con 0x7f6c00003f40 2024-03-12T17:07:36.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.073+0000 7f6c13400640 1 --2- 172.21.15.52:0/1910377619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c00003f40 0x7f6c000047c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6c04009ac0 tx=0x7f6c000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d1a1fcb27e07b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:36.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.073+0000 7f6c1b400640 1 -- 172.21.15.52:0/1910377619 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0402c6b0 con 0x7f6c00003f40 2024-03-12T17:07:36.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.073+0000 7f6c1b400640 1 -- 172.21.15.52:0/1910377619 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c0402c810 con 0x7f6c00003f40 2024-03-12T17:07:36.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.073+0000 7f6c1b400640 1 -- 172.21.15.52:0/1910377619 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0402cba0 con 0x7f6c00003f40 2024-03-12T17:07:36.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.074+0000 7f6c21a00640 1 -- 172.21.15.52:0/1910377619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c00003f40 msgr2=0x7f6c000047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:36.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.074+0000 7f6c21a00640 1 --2- 172.21.15.52:0/1910377619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c00003f40 0x7f6c000047c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6c04009ac0 tx=0x7f6c000057f0 comp rx=0 tx=0).stop 2024-03-12T17:07:36.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.074+0000 7f6c21000640 1 -- 172.21.15.52:0/1910377619 reap_dead start 2024-03-12T17:07:36.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.074+0000 7f6c21a00640 1 -- 172.21.15.52:0/1910377619 shutdown_connections 2024-03-12T17:07:36.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.074+0000 7f6c21a00640 1 -- 172.21.15.52:0/1910377619 >> 172.21.15.52:0/1910377619 conn(0x7f6c1c065df0 msgr2=0x7f6c1c069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:36.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.074+0000 7f6c21a00640 1 -- 172.21.15.52:0/1910377619 shutdown_connections 2024-03-12T17:07:36.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.074+0000 7f6c21a00640 1 -- 172.21.15.52:0/1910377619 wait complete. 2024-03-12T17:07:36.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.076+0000 7f6c21a00640 1 Processor -- start 2024-03-12T17:07:36.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.076+0000 7f6c21a00640 1 -- start start 2024-03-12T17:07:36.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.077+0000 7f6c21a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c00003f40 0x7f6c1c159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:36.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.077+0000 7f6c21a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c1c1eecd0 0x7f6c1c1ef110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:36.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.077+0000 7f6c21a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c1c0c4460 0x7f6c1c0c4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:36.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.077+0000 7f6c21a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c15a0c0 con 0x7f6c00003f40 2024-03-12T17:07:36.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.077+0000 7f6c21a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c0c3fd0 con 0x7f6c1c1eecd0 2024-03-12T17:07:36.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.077+0000 7f6c21a00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c0c33e0 con 0x7f6c1c0c4460 2024-03-12T17:07:36.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.077+0000 7f6c1be00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c00003f40 0x7f6c1c159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:36.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.077+0000 7f6c1be00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c00003f40 0x7f6c1c159b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51636/0 (socket says 172.21.15.52:51636) 2024-03-12T17:07:36.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.077+0000 7f6c1be00640 1 -- 172.21.15.52:0/1589569376 learned_addr learned my addr 172.21.15.52:0/1589569376 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:36.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.077+0000 7f6c21000640 1 --2- 172.21.15.52:0/1589569376 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c1c0c4460 0x7f6c1c0c4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:36.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.077+0000 7f6c13400640 1 --2- 172.21.15.52:0/1589569376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c1c1eecd0 0x7f6c1c1ef110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:36.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.077+0000 7f6c1be00640 1 -- 172.21.15.52:0/1589569376 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c1c0c4460 msgr2=0x7f6c1c0c4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:36.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.077+0000 7f6c1be00640 1 --2- 172.21.15.52:0/1589569376 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c1c0c4460 0x7f6c1c0c4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:36.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.077+0000 7f6c1be00640 1 -- 172.21.15.52:0/1589569376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c1c1eecd0 msgr2=0x7f6c1c1ef110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:36.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.077+0000 7f6c1be00640 1 --2- 172.21.15.52:0/1589569376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c1c1eecd0 0x7f6c1c1ef110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:36.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.077+0000 7f6c1be00640 1 -- 172.21.15.52:0/1589569376 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c0c36a0 con 0x7f6c00003f40 2024-03-12T17:07:36.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.077+0000 7f6c21000640 1 --2- 172.21.15.52:0/1589569376 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c1c0c4460 0x7f6c1c0c4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:36.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.077+0000 7f6c13400640 1 --2- 172.21.15.52:0/1589569376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c1c1eecd0 0x7f6c1c1ef110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:36.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.078+0000 7f6c1be00640 1 --2- 172.21.15.52:0/1589569376 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c00003f40 0x7f6c1c159b80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c001200 tx=0x7f6c0c00ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:36.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.078+0000 7f6c18c00640 1 -- 172.21.15.52:0/1589569376 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c00c5f0 con 0x7f6c00003f40 2024-03-12T17:07:36.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.078+0000 7f6c18c00640 1 -- 172.21.15.52:0/1589569376 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c0c00c770 con 0x7f6c00003f40 2024-03-12T17:07:36.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.078+0000 7f6c18c00640 1 -- 172.21.15.52:0/1589569376 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c00cb20 con 0x7f6c00003f40 2024-03-12T17:07:36.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.078+0000 7f6c21a00640 1 -- 172.21.15.52:0/1589569376 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c1c0c3980 con 0x7f6c00003f40 2024-03-12T17:07:36.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.078+0000 7f6c21a00640 1 -- 172.21.15.52:0/1589569376 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c1c0c85c0 con 0x7f6c00003f40 2024-03-12T17:07:36.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.079+0000 7f6c21a00640 1 -- 172.21.15.52:0/1589569376 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c1c04e030 con 0x7f6c00003f40 2024-03-12T17:07:36.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.080+0000 7f6c18c00640 1 -- 172.21.15.52:0/1589569376 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6c0c00cc80 con 0x7f6c00003f40 2024-03-12T17:07:36.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.081+0000 7f6c18c00640 1 --2- 172.21.15.52:0/1589569376 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f6bf806ac30 0x7f6bf806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:36.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.081+0000 7f6c18c00640 1 -- 172.21.15.52:0/1589569376 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6c0c0892e0 con 0x7f6c00003f40 2024-03-12T17:07:36.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.082+0000 7f6c13400640 1 --2- 172.21.15.52:0/1589569376 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f6bf806ac30 0x7f6bf806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:36.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.083+0000 7f6c13400640 1 --2- 172.21.15.52:0/1589569376 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f6bf806ac30 0x7f6bf806d0f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6c04036000 tx=0x7f6c04030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:36.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.086+0000 7f6c18c00640 1 -- 172.21.15.52:0/1589569376 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f6c0c054060 con 0x7f6c00003f40 2024-03-12T17:07:36.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.285+0000 7f6c21a00640 1 -- 172.21.15.52:0/1589569376 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6c1c06d8f0 con 0x7f6c00003f40 2024-03-12T17:07:36.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.286+0000 7f6c18c00640 1 -- 172.21.15.52:0/1589569376 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6c0c057680 con 0x7f6c00003f40 2024-03-12T17:07:36.279 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-12T17:07:36.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.288+0000 7f6c21a00640 1 -- 172.21.15.52:0/1589569376 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f6bf806ac30 msgr2=0x7f6bf806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:36.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.288+0000 7f6c21a00640 1 --2- 172.21.15.52:0/1589569376 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f6bf806ac30 0x7f6bf806d0f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6c04036000 tx=0x7f6c04030000 comp rx=0 tx=0).stop 2024-03-12T17:07:36.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.288+0000 7f6c21a00640 1 -- 172.21.15.52:0/1589569376 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c00003f40 msgr2=0x7f6c1c159b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:36.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.288+0000 7f6c21a00640 1 --2- 172.21.15.52:0/1589569376 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c00003f40 0x7f6c1c159b80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c001200 tx=0x7f6c0c00ead0 comp rx=0 tx=0).stop 2024-03-12T17:07:36.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.288+0000 7f6c21a00640 1 -- 172.21.15.52:0/1589569376 shutdown_connections 2024-03-12T17:07:36.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.288+0000 7f6c21a00640 1 --2- 172.21.15.52:0/1589569376 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f6bf806ac30 0x7f6bf806d0f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:36.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.288+0000 7f6c21a00640 1 --2- 172.21.15.52:0/1589569376 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c1c0c4460 0x7f6c1c0c4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:36.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.288+0000 7f6c21a00640 1 --2- 172.21.15.52:0/1589569376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c1c1eecd0 0x7f6c1c1ef110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:36.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.288+0000 7f6c21a00640 1 --2- 172.21.15.52:0/1589569376 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c00003f40 0x7f6c1c159b80 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:36.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.288+0000 7f6c21a00640 1 -- 172.21.15.52:0/1589569376 >> 172.21.15.52:0/1589569376 conn(0x7f6c1c065df0 msgr2=0x7f6c1c13e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:36.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.288+0000 7f6c21a00640 1 -- 172.21.15.52:0/1589569376 shutdown_connections 2024-03-12T17:07:36.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.289+0000 7f6c21a00640 1 -- 172.21.15.52:0/1589569376 wait complete. 2024-03-12T17:07:36.301 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-12T17:07:36.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.529+0000 7f09e9600640 1 Processor -- start 2024-03-12T17:07:36.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.529+0000 7f09e9600640 1 -- start start 2024-03-12T17:07:36.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.530+0000 7f09e9600640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f09e40b5a30 0x7f09e40b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:36.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.530+0000 7f09e9600640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09e40d1830 con 0x7f09e415cda0 2024-03-12T17:07:36.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.531+0000 7f09e9600640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09e40b1f20 con 0x7f09e40b4350 2024-03-12T17:07:36.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.531+0000 7f09e9600640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09e40d0c20 con 0x7f09e40b5a30 2024-03-12T17:07:36.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.531+0000 7f09e2000640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f09e40b5a30 0x7f09e40b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:36.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.531+0000 7f09e2a00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f09e415cda0 0x7f09e40cdb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:44800/0 (socket says 172.21.15.52:44800) 2024-03-12T17:07:36.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.531+0000 7f09e2000640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f09e40b5a30 0x7f09e40b5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51638/0 (socket says 172.21.15.52:51638) 2024-03-12T17:07:36.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.531+0000 7f09e2a00640 1 -- 172.21.15.52:0/4255479301 learned_addr learned my addr 172.21.15.52:0/4255479301 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:36.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.531+0000 7f09e1600640 1 -- 172.21.15.52:0/4255479301 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 893823061 0 0) 0x7f09e40d1830 con 0x7f09e415cda0 2024-03-12T17:07:36.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.531+0000 7f09e2000640 1 -- 172.21.15.52:0/4255479301 >> v1:172.21.15.102:6789/0 conn(0x7f09e40b4350 legacy=0x7f09e40d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:36.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.531+0000 7f09e2000640 1 -- 172.21.15.52:0/4255479301 >> v1:172.21.15.52:6789/0 conn(0x7f09e415cda0 legacy=0x7f09e40cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:36.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.531+0000 7f09e2000640 1 -- 172.21.15.52:0/4255479301 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e40d0f00 con 0x7f09e40b5a30 2024-03-12T17:07:36.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.533+0000 7f09e2000640 1 --2- 172.21.15.52:0/4255479301 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09e40b5a30 0x7f09e40b5e90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f09cc009d90 tx=0x7f09cc02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a23d6c9a581702 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:36.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.533+0000 7f09e1600640 1 -- 172.21.15.52:0/4255479301 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc02cc60 con 0x7f09e40b5a30 2024-03-12T17:07:36.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.533+0000 7f09e1600640 1 -- 172.21.15.52:0/4255479301 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09cc02cdc0 con 0x7f09e40b5a30 2024-03-12T17:07:36.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.533+0000 7f09e1600640 1 -- 172.21.15.52:0/4255479301 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc034660 con 0x7f09e40b5a30 2024-03-12T17:07:36.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.533+0000 7f09e9600640 1 -- 172.21.15.52:0/4255479301 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09e40b5a30 msgr2=0x7f09e40b5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:36.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.533+0000 7f09e9600640 1 --2- 172.21.15.52:0/4255479301 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09e40b5a30 0x7f09e40b5e90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f09cc009d90 tx=0x7f09cc02beb0 comp rx=0 tx=0).stop 2024-03-12T17:07:36.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.534+0000 7f09e9600640 1 -- 172.21.15.52:0/4255479301 shutdown_connections 2024-03-12T17:07:36.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.534+0000 7f09e9600640 1 --2- 172.21.15.52:0/4255479301 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09e40b5a30 0x7f09e40b5e90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:36.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.534+0000 7f09e9600640 1 -- 172.21.15.52:0/4255479301 >> 172.21.15.52:0/4255479301 conn(0x7f09e40cf350 msgr2=0x7f09e40cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:36.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.534+0000 7f09e9600640 1 -- 172.21.15.52:0/4255479301 shutdown_connections 2024-03-12T17:07:36.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.534+0000 7f09e9600640 1 -- 172.21.15.52:0/4255479301 wait complete. 2024-03-12T17:07:36.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.535+0000 7f09e9600640 1 Processor -- start 2024-03-12T17:07:36.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.536+0000 7f09e9600640 1 -- start start 2024-03-12T17:07:36.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.537+0000 7f09e9600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09e40b4350 0x7f09e4158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:36.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.537+0000 7f09e9600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09e40b5a30 0x7f09e40ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:36.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.537+0000 7f09e9600640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f09e415cda0 0x7f09e40c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:36.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.537+0000 7f09e9600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e415ee20 con 0x7f09e40b5a30 2024-03-12T17:07:36.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.537+0000 7f09e9600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e4159380 con 0x7f09e40b4350 2024-03-12T17:07:36.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.537+0000 7f09e9600640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e40cb4b0 con 0x7f09e415cda0 2024-03-12T17:07:36.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.537+0000 7f09e2000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09e40b5a30 0x7f09e40ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:36.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.537+0000 7f09e2000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09e40b5a30 0x7f09e40ca9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51650/0 (socket says 172.21.15.52:51650) 2024-03-12T17:07:36.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.537+0000 7f09e2000640 1 -- 172.21.15.52:0/2130990877 learned_addr learned my addr 172.21.15.52:0/2130990877 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:36.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.537+0000 7f09e2a00640 1 --2- 172.21.15.52:0/2130990877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09e40b4350 0x7f09e4158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:36.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.537+0000 7f09e3400640 1 --2- 172.21.15.52:0/2130990877 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f09e415cda0 0x7f09e40c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:36.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.538+0000 7f09e2000640 1 -- 172.21.15.52:0/2130990877 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f09e415cda0 msgr2=0x7f09e40c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:36.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.538+0000 7f09e2000640 1 --2- 172.21.15.52:0/2130990877 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f09e415cda0 0x7f09e40c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:36.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.538+0000 7f09e2000640 1 -- 172.21.15.52:0/2130990877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09e40b4350 msgr2=0x7f09e4158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:36.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.538+0000 7f09e2000640 1 --2- 172.21.15.52:0/2130990877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09e40b4350 0x7f09e4158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:36.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.538+0000 7f09e2000640 1 -- 172.21.15.52:0/2130990877 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09cc009a70 con 0x7f09e40b5a30 2024-03-12T17:07:36.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.538+0000 7f09e2000640 1 --2- 172.21.15.52:0/2130990877 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09e40b5a30 0x7f09e40ca9f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f09cc037040 tx=0x7f09cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:36.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.538+0000 7f09daa00640 1 -- 172.21.15.52:0/2130990877 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc040650 con 0x7f09e40b5a30 2024-03-12T17:07:36.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.538+0000 7f09daa00640 1 -- 172.21.15.52:0/2130990877 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09cc0407b0 con 0x7f09e40b5a30 2024-03-12T17:07:36.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.538+0000 7f09daa00640 1 -- 172.21.15.52:0/2130990877 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc040b60 con 0x7f09e40b5a30 2024-03-12T17:07:36.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.538+0000 7f09e9600640 1 -- 172.21.15.52:0/2130990877 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09e415e480 con 0x7f09e40b5a30 2024-03-12T17:07:36.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.539+0000 7f09e9600640 1 -- 172.21.15.52:0/2130990877 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09e415ea40 con 0x7f09e40b5a30 2024-03-12T17:07:36.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.540+0000 7f09e9600640 1 -- 172.21.15.52:0/2130990877 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09e415ebd0 con 0x7f09e40b5a30 2024-03-12T17:07:36.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.541+0000 7f09daa00640 1 -- 172.21.15.52:0/2130990877 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f09cc040cc0 con 0x7f09e40b5a30 2024-03-12T17:07:36.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.542+0000 7f09daa00640 1 --2- 172.21.15.52:0/2130990877 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f09c006ac00 0x7f09c006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:36.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.542+0000 7f09daa00640 1 -- 172.21.15.52:0/2130990877 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f09cc0ac6d0 con 0x7f09e40b5a30 2024-03-12T17:07:36.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.543+0000 7f09e2a00640 1 --2- 172.21.15.52:0/2130990877 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f09c006ac00 0x7f09c006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:36.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.544+0000 7f09e2a00640 1 --2- 172.21.15.52:0/2130990877 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f09c006ac00 0x7f09c006d0c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f09d4000f60 tx=0x7f09d4007c00 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:36.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.547+0000 7f09daa00640 1 -- 172.21.15.52:0/2130990877 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f09cc0774f0 con 0x7f09e40b5a30 2024-03-12T17:07:36.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.758+0000 7f09e9600640 1 -- 172.21.15.52:0/2130990877 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f09e415ef60 con 0x7f09e40b5a30 2024-03-12T17:07:36.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.759+0000 7f09daa00640 1 -- 172.21.15.52:0/2130990877 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f09e415ef60 con 0x7f09e40b5a30 2024-03-12T17:07:36.753 INFO:teuthology.orchestra.run.smithi052.stderr:listed 0 entries 2024-03-12T17:07:36.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.762+0000 7f09e9600640 1 -- 172.21.15.52:0/2130990877 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f09c006ac00 msgr2=0x7f09c006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:36.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.762+0000 7f09e9600640 1 --2- 172.21.15.52:0/2130990877 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f09c006ac00 0x7f09c006d0c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f09d4000f60 tx=0x7f09d4007c00 comp rx=0 tx=0).stop 2024-03-12T17:07:36.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.762+0000 7f09e9600640 1 -- 172.21.15.52:0/2130990877 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09e40b5a30 msgr2=0x7f09e40ca9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:36.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.762+0000 7f09e9600640 1 --2- 172.21.15.52:0/2130990877 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09e40b5a30 0x7f09e40ca9f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f09cc037040 tx=0x7f09cc004a40 comp rx=0 tx=0).stop 2024-03-12T17:07:36.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.762+0000 7f09e9600640 1 -- 172.21.15.52:0/2130990877 shutdown_connections 2024-03-12T17:07:36.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.762+0000 7f09e9600640 1 --2- 172.21.15.52:0/2130990877 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f09c006ac00 0x7f09c006d0c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:36.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.762+0000 7f09e9600640 1 --2- 172.21.15.52:0/2130990877 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f09e415cda0 0x7f09e40c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:36.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.762+0000 7f09e9600640 1 --2- 172.21.15.52:0/2130990877 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09e40b5a30 0x7f09e40ca9f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:36.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.762+0000 7f09e9600640 1 --2- 172.21.15.52:0/2130990877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09e40b4350 0x7f09e4158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:36.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.762+0000 7f09e9600640 1 -- 172.21.15.52:0/2130990877 >> 172.21.15.52:0/2130990877 conn(0x7f09e40cf350 msgr2=0x7f09e40cdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:36.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.762+0000 7f09e9600640 1 -- 172.21.15.52:0/2130990877 shutdown_connections 2024-03-12T17:07:36.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:36.762+0000 7f09e9600640 1 -- 172.21.15.52:0/2130990877 wait complete. 2024-03-12T17:07:36.775 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-12T17:07:36.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:36.775 DEBUG:teuthology.orchestra.run.smithi052:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-03-12T17:07:36.883 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi052.front.sepia.ceph.com 2024-03-12T17:07:36.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:36.884 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-12T17:07:36.934 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T17:07:36.934 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-12T17:07:36.992 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T17:07:36.992 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-12T17:07:37.047 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T17:07:37.047 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-12T17:07:37.098 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:07:37.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.332+0000 7f4ef6e00640 1 Processor -- start 2024-03-12T17:07:37.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.332+0000 7f4ef6e00640 1 -- start start 2024-03-12T17:07:37.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.333+0000 7f4ef6e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4ef0098a40 0x7f4ef006be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:37.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.333+0000 7f4ef6e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4ef00cf3c0 0x7f4ef00cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:37.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.333+0000 7f4ef6e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ef006c380 con 0x7f4ef015ccd0 2024-03-12T17:07:37.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.333+0000 7f4ef6e00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ef006ada0 con 0x7f4ef0098a40 2024-03-12T17:07:37.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.333+0000 7f4ef6e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ef006b080 con 0x7f4ef00cf3c0 2024-03-12T17:07:37.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.334+0000 7f4eefe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4ef0098a40 0x7f4ef006be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:37.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.334+0000 7f4eefe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4ef0098a40 0x7f4ef006be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51660/0 (socket says 172.21.15.52:51660) 2024-03-12T17:07:37.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.334+0000 7f4eefe00640 1 -- 172.21.15.52:0/3957465598 learned_addr learned my addr 172.21.15.52:0/3957465598 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:37.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.334+0000 7f4ef4e00640 1 --2- 172.21.15.52:0/3957465598 >> v2:172.21.15.102:3300/0 conn(0x7f4ef00cf3c0 0x7f4ef00cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:37.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.334+0000 7f4eefe00640 1 -- 172.21.15.52:0/3957465598 >> v2:172.21.15.102:3300/0 conn(0x7f4ef00cf3c0 msgr2=0x7f4ef00cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:37.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.334+0000 7f4eefe00640 1 --2- 172.21.15.52:0/3957465598 >> v2:172.21.15.102:3300/0 conn(0x7f4ef00cf3c0 0x7f4ef00cf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:37.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.334+0000 7f4eefe00640 1 -- 172.21.15.52:0/3957465598 >> v1:172.21.15.102:6789/0 conn(0x7f4ef015ccd0 legacy=0x7f4ef0098e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:37.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.334+0000 7f4eefe00640 1 -- 172.21.15.52:0/3957465598 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef006b360 con 0x7f4ef0098a40 2024-03-12T17:07:37.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.336+0000 7f4eefe00640 1 --2- 172.21.15.52:0/3957465598 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ef0098a40 0x7f4ef006be40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4ef00c4150 tx=0x7f4ee002bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6006baf0339c841e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:37.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.336+0000 7f4eeea00640 1 -- 172.21.15.52:0/3957465598 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee002c960 con 0x7f4ef0098a40 2024-03-12T17:07:37.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.336+0000 7f4eeea00640 1 -- 172.21.15.52:0/3957465598 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee002cac0 con 0x7f4ef0098a40 2024-03-12T17:07:37.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.336+0000 7f4eeea00640 1 -- 172.21.15.52:0/3957465598 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee002ce50 con 0x7f4ef0098a40 2024-03-12T17:07:37.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.337+0000 7f4ef6e00640 1 -- 172.21.15.52:0/3957465598 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ef0098a40 msgr2=0x7f4ef006be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:37.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.337+0000 7f4ef6e00640 1 --2- 172.21.15.52:0/3957465598 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ef0098a40 0x7f4ef006be40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4ef00c4150 tx=0x7f4ee002bdc0 comp rx=0 tx=0).stop 2024-03-12T17:07:37.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.337+0000 7f4ef6e00640 1 -- 172.21.15.52:0/3957465598 shutdown_connections 2024-03-12T17:07:37.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.337+0000 7f4ef6e00640 1 --2- 172.21.15.52:0/3957465598 >> v2:172.21.15.102:3300/0 conn(0x7f4ef00cf3c0 0x7f4ef00cf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:37.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.337+0000 7f4ef6e00640 1 --2- 172.21.15.52:0/3957465598 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ef0098a40 0x7f4ef006be40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:37.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.337+0000 7f4ef6e00640 1 -- 172.21.15.52:0/3957465598 >> 172.21.15.52:0/3957465598 conn(0x7f4ef00b2160 msgr2=0x7f4ef00b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:37.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.337+0000 7f4ef6e00640 1 -- 172.21.15.52:0/3957465598 shutdown_connections 2024-03-12T17:07:37.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.337+0000 7f4ef6e00640 1 -- 172.21.15.52:0/3957465598 wait complete. 2024-03-12T17:07:37.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.339+0000 7f4ef6e00640 1 Processor -- start 2024-03-12T17:07:37.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.340+0000 7f4ef6e00640 1 -- start start 2024-03-12T17:07:37.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.341+0000 7f4ef6e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ef0098a40 0x7f4ef014b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:37.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.341+0000 7f4ef6e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4ef00cf3c0 0x7f4ef00b3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:37.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.341+0000 7f4ef6e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ef015ccd0 0x7f4ef00b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:37.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.341+0000 7f4ef6e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef014c110 con 0x7f4ef0098a40 2024-03-12T17:07:37.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.341+0000 7f4ef6e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef014bdd0 con 0x7f4ef015ccd0 2024-03-12T17:07:37.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.341+0000 7f4ef6e00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef00c6120 con 0x7f4ef00cf3c0 2024-03-12T17:07:37.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.341+0000 7f4eefe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ef0098a40 0x7f4ef014b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:37.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.341+0000 7f4eefe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ef0098a40 0x7f4ef014b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51666/0 (socket says 172.21.15.52:51666) 2024-03-12T17:07:37.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.341+0000 7f4eefe00640 1 -- 172.21.15.52:0/2641680961 learned_addr learned my addr 172.21.15.52:0/2641680961 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:37.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.342+0000 7f4eefe00640 1 -- 172.21.15.52:0/2641680961 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4ef00cf3c0 msgr2=0x7f4ef00b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:37.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.342+0000 7f4eef400640 1 --2- 172.21.15.52:0/2641680961 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4ef00cf3c0 0x7f4ef00b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:37.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.342+0000 7f4eefe00640 1 --2- 172.21.15.52:0/2641680961 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4ef00cf3c0 0x7f4ef00b3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:37.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.342+0000 7f4eefe00640 1 -- 172.21.15.52:0/2641680961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ef015ccd0 msgr2=0x7f4ef00b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:37.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.342+0000 7f4ef4e00640 1 --2- 172.21.15.52:0/2641680961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ef015ccd0 0x7f4ef00b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:37.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.342+0000 7f4eefe00640 1 --2- 172.21.15.52:0/2641680961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ef015ccd0 0x7f4ef00b37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:37.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.342+0000 7f4eefe00640 1 -- 172.21.15.52:0/2641680961 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee0009b30 con 0x7f4ef0098a40 2024-03-12T17:07:37.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.342+0000 7f4ef4e00640 1 --2- 172.21.15.52:0/2641680961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ef015ccd0 0x7f4ef00b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:37.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.342+0000 7f4eefe00640 1 --2- 172.21.15.52:0/2641680961 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ef0098a40 0x7f4ef014b890 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0036040 tx=0x7f4ee0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:37.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.342+0000 7f4ee7e00640 1 -- 172.21.15.52:0/2641680961 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee003e450 con 0x7f4ef0098a40 2024-03-12T17:07:37.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.342+0000 7f4ee7e00640 1 -- 172.21.15.52:0/2641680961 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee003e5b0 con 0x7f4ef0098a40 2024-03-12T17:07:37.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.342+0000 7f4ee7e00640 1 -- 172.21.15.52:0/2641680961 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee003e960 con 0x7f4ef0098a40 2024-03-12T17:07:37.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.343+0000 7f4ef6e00640 1 -- 172.21.15.52:0/2641680961 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef00c6400 con 0x7f4ef0098a40 2024-03-12T17:07:37.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.343+0000 7f4ef6e00640 1 -- 172.21.15.52:0/2641680961 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef00c69c0 con 0x7f4ef0098a40 2024-03-12T17:07:37.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.344+0000 7f4ef6e00640 1 -- 172.21.15.52:0/2641680961 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ef0073390 con 0x7f4ef0098a40 2024-03-12T17:07:37.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.345+0000 7f4ee7e00640 1 -- 172.21.15.52:0/2641680961 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4ee003eac0 con 0x7f4ef0098a40 2024-03-12T17:07:37.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.346+0000 7f4ee7e00640 1 --2- 172.21.15.52:0/2641680961 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4ecc06ac00 0x7f4ecc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:37.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.346+0000 7f4ee7e00640 1 -- 172.21.15.52:0/2641680961 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4ee00ac390 con 0x7f4ef0098a40 2024-03-12T17:07:37.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.346+0000 7f4eef400640 1 --2- 172.21.15.52:0/2641680961 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4ecc06ac00 0x7f4ecc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:37.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.347+0000 7f4eef400640 1 --2- 172.21.15.52:0/2641680961 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4ecc06ac00 0x7f4ecc06d0c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4ef00cee30 tx=0x7f4ed8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:37.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.350+0000 7f4ee7e00640 1 -- 172.21.15.52:0/2641680961 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f4ee0077360 con 0x7f4ef0098a40 2024-03-12T17:07:37.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.555+0000 7f4ef6e00640 1 -- 172.21.15.52:0/2641680961 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ef006c380 con 0x7f4ef0098a40 2024-03-12T17:07:37.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.556+0000 7f4ee7e00640 1 -- 172.21.15.52:0/2641680961 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18207 (secure 0 0 0) 0x7f4ef006c380 con 0x7f4ef0098a40 2024-03-12T17:07:37.551 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:07:37.551 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":27,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:07:12.969614+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T17:06:19.218543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T17:06:23.525474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T17:06:24.911574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-12T17:06:28.017172+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.733031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.775178+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.591522+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.820838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.829252+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.582481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.18","mappings":[{"from":3,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:07:37.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.559+0000 7f4ef6e00640 1 -- 172.21.15.52:0/2641680961 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4ecc06ac00 msgr2=0x7f4ecc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:37.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.559+0000 7f4ef6e00640 1 --2- 172.21.15.52:0/2641680961 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4ecc06ac00 0x7f4ecc06d0c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4ef00cee30 tx=0x7f4ed8008040 comp rx=0 tx=0).stop 2024-03-12T17:07:37.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.559+0000 7f4ef6e00640 1 -- 172.21.15.52:0/2641680961 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ef0098a40 msgr2=0x7f4ef014b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:37.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.559+0000 7f4ef6e00640 1 --2- 172.21.15.52:0/2641680961 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ef0098a40 0x7f4ef014b890 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0036040 tx=0x7f4ee0004a40 comp rx=0 tx=0).stop 2024-03-12T17:07:37.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.559+0000 7f4ef6e00640 1 -- 172.21.15.52:0/2641680961 shutdown_connections 2024-03-12T17:07:37.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.559+0000 7f4ef6e00640 1 --2- 172.21.15.52:0/2641680961 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4ecc06ac00 0x7f4ecc06d0c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:37.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.559+0000 7f4ef6e00640 1 --2- 172.21.15.52:0/2641680961 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4ef00cf3c0 0x7f4ef00b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:37.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.559+0000 7f4ef6e00640 1 --2- 172.21.15.52:0/2641680961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ef015ccd0 0x7f4ef00b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:37.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.559+0000 7f4ef6e00640 1 --2- 172.21.15.52:0/2641680961 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ef0098a40 0x7f4ef014b890 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:37.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.559+0000 7f4ef6e00640 1 -- 172.21.15.52:0/2641680961 >> 172.21.15.52:0/2641680961 conn(0x7f4ef00b2160 msgr2=0x7f4ef0099790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:37.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.559+0000 7f4ef6e00640 1 -- 172.21.15.52:0/2641680961 shutdown_connections 2024-03-12T17:07:37.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.559+0000 7f4ef6e00640 1 -- 172.21.15.52:0/2641680961 wait complete. 2024-03-12T17:07:37.574 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T17:06:19.218543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-12T17:06:23.525474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-12T17:06:24.911574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-12T17:06:28.017172+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-12T17:07:37.574 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T17:07:37.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.808+0000 7f7606600640 1 Processor -- start 2024-03-12T17:07:37.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.808+0000 7f7606600640 1 -- start start 2024-03-12T17:07:37.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.809+0000 7f7606600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f76000b5b90 0x7f76000b5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:37.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.809+0000 7f7606600640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7600160ae0 con 0x7f76000cf510 2024-03-12T17:07:37.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.809+0000 7f7606600640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76000b6510 con 0x7f760015cc30 2024-03-12T17:07:37.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.809+0000 7f7606600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76000d0c00 con 0x7f76000b5b90 2024-03-12T17:07:37.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.810+0000 7f7605c00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f76000cf510 0x7f76000cd200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:44810/0 (socket says 172.21.15.52:44810) 2024-03-12T17:07:37.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.810+0000 7f7605c00640 1 -- 172.21.15.52:0/2672582106 learned_addr learned my addr 172.21.15.52:0/2672582106 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:37.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.810+0000 7f75ffe00640 1 --2- 172.21.15.52:0/2672582106 >> v2:172.21.15.102:3300/0 conn(0x7f76000b5b90 0x7f76000b5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:37.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.810+0000 7f75ff400640 1 -- 172.21.15.52:0/2672582106 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1387588054 0 0) 0x7f7600160ae0 con 0x7f76000cf510 2024-03-12T17:07:37.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.810+0000 7f75ff400640 1 -- 172.21.15.52:0/2672582106 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75e4003520 con 0x7f76000cf510 2024-03-12T17:07:37.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.811+0000 7f75ff400640 1 -- 172.21.15.52:0/2672582106 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 274603456 0 0) 0x7f75e4003520 con 0x7f76000cf510 2024-03-12T17:07:37.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.811+0000 7f75ff400640 1 -- 172.21.15.52:0/2672582106 >> v2:172.21.15.102:3300/0 conn(0x7f76000b5b90 msgr2=0x7f76000b5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:37.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.811+0000 7f75ff400640 1 --2- 172.21.15.52:0/2672582106 >> v2:172.21.15.102:3300/0 conn(0x7f76000b5b90 0x7f76000b5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:37.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.811+0000 7f75ff400640 1 -- 172.21.15.52:0/2672582106 >> v1:172.21.15.102:6789/0 conn(0x7f760015cc30 legacy=0x7f76000d16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:37.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.811+0000 7f75ff400640 1 -- 172.21.15.52:0/2672582106 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76000cdde0 con 0x7f76000cf510 2024-03-12T17:07:37.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.811+0000 7f75ff400640 1 -- 172.21.15.52:0/2672582106 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f75f80019f0 con 0x7f76000cf510 2024-03-12T17:07:37.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.811+0000 7f75ff400640 1 -- 172.21.15.52:0/2672582106 >> v1:172.21.15.52:6789/0 conn(0x7f76000cf510 legacy=0x7f76000cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:37.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.811+0000 7f75ff400640 1 --2- 172.21.15.52:0/2672582106 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f75e40039c0 0x7f75e4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:37.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.811+0000 7f75ff400640 1 --2- 172.21.15.52:0/2672582106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75e40047c0 0x7f75e4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:37.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.811+0000 7f75ff400640 1 --2- 172.21.15.52:0/2672582106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75e4005180 0x7f75e400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:37.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.811+0000 7f75ff400640 1 -- 172.21.15.52:0/2672582106 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76000d0c00 con 0x7f75e4005180 2024-03-12T17:07:37.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.811+0000 7f75ff400640 1 -- 172.21.15.52:0/2672582106 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e400c080 con 0x7f75e40047c0 2024-03-12T17:07:37.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.811+0000 7f75ff400640 1 -- 172.21.15.52:0/2672582106 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e400c240 con 0x7f75e40039c0 2024-03-12T17:07:37.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.811+0000 7f7605c00640 1 --2- 172.21.15.52:0/2672582106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75e4005180 0x7f75e400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:37.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.812+0000 7f75ffe00640 1 --2- 172.21.15.52:0/2672582106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75e40047c0 0x7f75e4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:37.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.812+0000 7f7605200640 1 --2- 172.21.15.52:0/2672582106 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f75e40039c0 0x7f75e4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:37.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.812+0000 7f7605c00640 1 -- 172.21.15.52:0/2672582106 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f75e40039c0 msgr2=0x7f75e4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:37.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.812+0000 7f7605c00640 1 --2- 172.21.15.52:0/2672582106 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f75e40039c0 0x7f75e4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:37.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.812+0000 7f7605c00640 1 -- 172.21.15.52:0/2672582106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75e40047c0 msgr2=0x7f75e4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:37.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.812+0000 7f7605c00640 1 --2- 172.21.15.52:0/2672582106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75e40047c0 0x7f75e4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:37.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.812+0000 7f7605c00640 1 -- 172.21.15.52:0/2672582106 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e4003520 con 0x7f75e4005180 2024-03-12T17:07:37.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.812+0000 7f7605200640 1 --2- 172.21.15.52:0/2672582106 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f75e40039c0 0x7f75e4003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:37.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.813+0000 7f7605c00640 1 -- 172.21.15.52:0/2672582106 reap_dead start 2024-03-12T17:07:37.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.813+0000 7f7605c00640 1 --2- 172.21.15.52:0/2672582106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75e4005180 0x7f75e400b980 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f75f800a760 tx=0x7f75e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e02a9368971769 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:37.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.813+0000 7f75ff400640 1 -- 172.21.15.52:0/2672582106 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f8034630 con 0x7f75e4005180 2024-03-12T17:07:37.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.813+0000 7f75ff400640 1 -- 172.21.15.52:0/2672582106 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f802f070 con 0x7f75e4005180 2024-03-12T17:07:37.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.813+0000 7f75ff400640 1 -- 172.21.15.52:0/2672582106 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f80349d0 con 0x7f75e4005180 2024-03-12T17:07:37.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.814+0000 7f7606600640 1 -- 172.21.15.52:0/2672582106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75e4005180 msgr2=0x7f75e400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:37.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.814+0000 7f7606600640 1 --2- 172.21.15.52:0/2672582106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75e4005180 0x7f75e400b980 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f75f800a760 tx=0x7f75e40057f0 comp rx=0 tx=0).stop 2024-03-12T17:07:37.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.814+0000 7f7606600640 1 -- 172.21.15.52:0/2672582106 shutdown_connections 2024-03-12T17:07:37.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.814+0000 7f7606600640 1 --2- 172.21.15.52:0/2672582106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75e4005180 0x7f75e400b980 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:37.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.814+0000 7f7606600640 1 -- 172.21.15.52:0/2672582106 >> 172.21.15.52:0/2672582106 conn(0x7f760006b100 msgr2=0x7f760006b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:37.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.814+0000 7f7606600640 1 -- 172.21.15.52:0/2672582106 shutdown_connections 2024-03-12T17:07:37.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.814+0000 7f7606600640 1 -- 172.21.15.52:0/2672582106 wait complete. 2024-03-12T17:07:37.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.816+0000 7f7606600640 1 Processor -- start 2024-03-12T17:07:37.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.816+0000 7f7606600640 1 -- start start 2024-03-12T17:07:37.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.817+0000 7f7606600640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f75e4005180 0x7f76001f2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:37.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.817+0000 7f7606600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76000b2290 0x7f7600159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:37.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.817+0000 7f7606600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76000b26c0 0x7f76000cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:37.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.817+0000 7f7606600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f760015efc0 con 0x7f76000b26c0 2024-03-12T17:07:37.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.817+0000 7f7606600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f760015e3b0 con 0x7f76000b2290 2024-03-12T17:07:37.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.817+0000 7f7606600640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f760015e690 con 0x7f75e4005180 2024-03-12T17:07:37.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.817+0000 7f7605c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76000b26c0 0x7f76000cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:37.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.817+0000 7f7605c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76000b26c0 0x7f76000cb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51676/0 (socket says 172.21.15.52:51676) 2024-03-12T17:07:37.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.817+0000 7f7605c00640 1 -- 172.21.15.52:0/655589687 learned_addr learned my addr 172.21.15.52:0/655589687 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:37.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.818+0000 7f7605200640 1 --2- 172.21.15.52:0/655589687 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f75e4005180 0x7f76001f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:37.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.818+0000 7f75ffe00640 1 --2- 172.21.15.52:0/655589687 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76000b2290 0x7f7600159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:37.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.818+0000 7f7605c00640 1 -- 172.21.15.52:0/655589687 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f75e4005180 msgr2=0x7f76001f2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:37.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.818+0000 7f7605c00640 1 --2- 172.21.15.52:0/655589687 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f75e4005180 0x7f76001f2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:37.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.818+0000 7f7605c00640 1 -- 172.21.15.52:0/655589687 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76000b2290 msgr2=0x7f7600159840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:37.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.818+0000 7f7605c00640 1 --2- 172.21.15.52:0/655589687 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76000b2290 0x7f7600159840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:37.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.818+0000 7f7605c00640 1 -- 172.21.15.52:0/655589687 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f760015e970 con 0x7f76000b26c0 2024-03-12T17:07:37.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.818+0000 7f75ffe00640 1 --2- 172.21.15.52:0/655589687 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76000b2290 0x7f7600159840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:37.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.818+0000 7f7605c00640 1 --2- 172.21.15.52:0/655589687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76000b26c0 0x7f76000cb7d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f75f800a760 tx=0x7f75f8001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:37.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.818+0000 7f75fcc00640 1 -- 172.21.15.52:0/655589687 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f80018b0 con 0x7f76000b26c0 2024-03-12T17:07:37.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.818+0000 7f75fcc00640 1 -- 172.21.15.52:0/655589687 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f8034430 con 0x7f76000b26c0 2024-03-12T17:07:37.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.818+0000 7f75fcc00640 1 -- 172.21.15.52:0/655589687 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f80347e0 con 0x7f76000b26c0 2024-03-12T17:07:37.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.818+0000 7f7606600640 1 -- 172.21.15.52:0/655589687 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f760015ec50 con 0x7f76000b26c0 2024-03-12T17:07:37.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.819+0000 7f7606600640 1 -- 172.21.15.52:0/655589687 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76000b2b30 con 0x7f76000b26c0 2024-03-12T17:07:37.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.820+0000 7f7606600640 1 -- 172.21.15.52:0/655589687 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f760015f170 con 0x7f76000b26c0 2024-03-12T17:07:37.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.821+0000 7f75fcc00640 1 -- 172.21.15.52:0/655589687 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f75f8031030 con 0x7f76000b26c0 2024-03-12T17:07:37.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.822+0000 7f75fcc00640 1 --2- 172.21.15.52:0/655589687 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f75e006ac00 0x7f75e006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:37.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.822+0000 7f75fcc00640 1 -- 172.21.15.52:0/655589687 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f75f80aafd0 con 0x7f76000b26c0 2024-03-12T17:07:37.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.822+0000 7f7605200640 1 --2- 172.21.15.52:0/655589687 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f75e006ac00 0x7f75e006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:37.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.823+0000 7f7605200640 1 --2- 172.21.15.52:0/655589687 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f75e006ac00 0x7f75e006d0c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f75ec0015f0 tx=0x7f75ec009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:37.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:37.826+0000 7f75fcc00640 1 -- 172.21.15.52:0/655589687 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f760015f170 con 0x7f76000b26c0 2024-03-12T17:07:38.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.030+0000 7f7606600640 1 -- 172.21.15.52:0/655589687 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f760004e030 con 0x7f76000b26c0 2024-03-12T17:07:38.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.031+0000 7f75fcc00640 1 -- 172.21.15.52:0/655589687 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f75f8075f10 con 0x7f76000b26c0 2024-03-12T17:07:38.025 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-03-12T17:07:38.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.034+0000 7f7606600640 1 -- 172.21.15.52:0/655589687 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f75e006ac00 msgr2=0x7f75e006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:38.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.034+0000 7f7606600640 1 --2- 172.21.15.52:0/655589687 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f75e006ac00 0x7f75e006d0c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f75ec0015f0 tx=0x7f75ec009040 comp rx=0 tx=0).stop 2024-03-12T17:07:38.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.034+0000 7f7606600640 1 -- 172.21.15.52:0/655589687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76000b26c0 msgr2=0x7f76000cb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:38.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.034+0000 7f7606600640 1 --2- 172.21.15.52:0/655589687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76000b26c0 0x7f76000cb7d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f75f800a760 tx=0x7f75f8001cb0 comp rx=0 tx=0).stop 2024-03-12T17:07:38.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.034+0000 7f7606600640 1 -- 172.21.15.52:0/655589687 shutdown_connections 2024-03-12T17:07:38.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.034+0000 7f7606600640 1 --2- 172.21.15.52:0/655589687 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f75e006ac00 0x7f75e006d0c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:38.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.034+0000 7f7606600640 1 --2- 172.21.15.52:0/655589687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76000b26c0 0x7f76000cb7d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:38.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.034+0000 7f7606600640 1 --2- 172.21.15.52:0/655589687 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76000b2290 0x7f7600159840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:38.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.034+0000 7f7606600640 1 --2- 172.21.15.52:0/655589687 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f75e4005180 0x7f76001f2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:38.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.034+0000 7f7606600640 1 -- 172.21.15.52:0/655589687 >> 172.21.15.52:0/655589687 conn(0x7f760006b100 msgr2=0x7f76000d14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:38.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.034+0000 7f7606600640 1 -- 172.21.15.52:0/655589687 shutdown_connections 2024-03-12T17:07:38.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.034+0000 7f7606600640 1 -- 172.21.15.52:0/655589687 wait complete. 2024-03-12T17:07:38.048 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-12T17:07:38.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.284+0000 7f8715e00640 1 Processor -- start 2024-03-12T17:07:38.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.284+0000 7f8715e00640 1 -- start start 2024-03-12T17:07:38.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.285+0000 7f8715e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f871015cab0 0x7f87100b8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:38.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.285+0000 7f8715e00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f871015ce90 con 0x7f87100d1c30 2024-03-12T17:07:38.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.285+0000 7f8715e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87100cddf0 con 0x7f87100ceeb0 2024-03-12T17:07:38.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.285+0000 7f8715e00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87100b4f40 con 0x7f871015cab0 2024-03-12T17:07:38.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.285+0000 7f8714a00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f871015cab0 0x7f87100b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:38.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.285+0000 7f8714a00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f871015cab0 0x7f87100b8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51684/0 (socket says 172.21.15.52:51684) 2024-03-12T17:07:38.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.285+0000 7f8714a00640 1 -- 172.21.15.52:0/1578708723 learned_addr learned my addr 172.21.15.52:0/1578708723 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:38.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.286+0000 7f8714a00640 1 -- 172.21.15.52:0/1578708723 >> v1:172.21.15.102:6789/0 conn(0x7f87100ceeb0 legacy=0x7f87100b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:38.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.286+0000 7f8714a00640 1 -- 172.21.15.52:0/1578708723 >> v1:172.21.15.52:6789/0 conn(0x7f87100d1c30 legacy=0x7f87100b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:38.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.286+0000 7f8714a00640 1 -- 172.21.15.52:0/1578708723 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87100cd570 con 0x7f871015cab0 2024-03-12T17:07:38.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.286+0000 7f870f400640 1 -- 172.21.15.52:0/1578708723 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2112217733 0 0) 0x7f871015ce90 con 0x7f87100d1c30 2024-03-12T17:07:38.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.287+0000 7f8714a00640 1 --2- 172.21.15.52:0/1578708723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f871015cab0 0x7f87100b8050 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8704009870 tx=0x7f870402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=762a6981dc4392a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:38.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.287+0000 7f870f400640 1 -- 172.21.15.52:0/1578708723 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8704031870 con 0x7f871015cab0 2024-03-12T17:07:38.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.288+0000 7f870f400640 1 -- 172.21.15.52:0/1578708723 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87040319d0 con 0x7f871015cab0 2024-03-12T17:07:38.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.288+0000 7f870f400640 1 -- 172.21.15.52:0/1578708723 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8704031d60 con 0x7f871015cab0 2024-03-12T17:07:38.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.288+0000 7f8715e00640 1 -- 172.21.15.52:0/1578708723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f871015cab0 msgr2=0x7f87100b8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:38.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.288+0000 7f8715e00640 1 --2- 172.21.15.52:0/1578708723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f871015cab0 0x7f87100b8050 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8704009870 tx=0x7f870402b910 comp rx=0 tx=0).stop 2024-03-12T17:07:38.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.288+0000 7f8715e00640 1 -- 172.21.15.52:0/1578708723 shutdown_connections 2024-03-12T17:07:38.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.288+0000 7f8715e00640 1 --2- 172.21.15.52:0/1578708723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f871015cab0 0x7f87100b8050 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:38.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.288+0000 7f8715e00640 1 -- 172.21.15.52:0/1578708723 >> 172.21.15.52:0/1578708723 conn(0x7f871006ae90 msgr2=0x7f871006b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:38.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.288+0000 7f8715e00640 1 -- 172.21.15.52:0/1578708723 shutdown_connections 2024-03-12T17:07:38.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.288+0000 7f8715e00640 1 -- 172.21.15.52:0/1578708723 wait complete. 2024-03-12T17:07:38.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.290+0000 7f8715e00640 1 Processor -- start 2024-03-12T17:07:38.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.290+0000 7f8715e00640 1 -- start start 2024-03-12T17:07:38.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.291+0000 7f8715e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f87100ceeb0 0x7f871015e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:38.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.291+0000 7f8715e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87100d1c30 0x7f871015f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:38.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.291+0000 7f8715e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f871015cab0 0x7f8710156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:38.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.291+0000 7f8715e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871015fed0 con 0x7f871015cab0 2024-03-12T17:07:38.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.291+0000 7f8715e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8710157030 con 0x7f87100d1c30 2024-03-12T17:07:38.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.291+0000 7f8715e00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f871015eff0 con 0x7f87100ceeb0 2024-03-12T17:07:38.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.291+0000 7f8715400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f871015cab0 0x7f8710156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:38.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.292+0000 7f8715400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f871015cab0 0x7f8710156af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51698/0 (socket says 172.21.15.52:51698) 2024-03-12T17:07:38.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.292+0000 7f8715400640 1 -- 172.21.15.52:0/3685189221 learned_addr learned my addr 172.21.15.52:0/3685189221 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:38.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.292+0000 7f8714a00640 1 --2- 172.21.15.52:0/3685189221 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f87100ceeb0 0x7f871015e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:38.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.292+0000 7f870fe00640 1 --2- 172.21.15.52:0/3685189221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87100d1c30 0x7f871015f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:38.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.292+0000 7f8715400640 1 -- 172.21.15.52:0/3685189221 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f87100ceeb0 msgr2=0x7f871015e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:38.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.292+0000 7f8715400640 1 --2- 172.21.15.52:0/3685189221 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f87100ceeb0 0x7f871015e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:38.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.292+0000 7f8715400640 1 -- 172.21.15.52:0/3685189221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87100d1c30 msgr2=0x7f871015f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:38.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.292+0000 7f8715400640 1 --2- 172.21.15.52:0/3685189221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87100d1c30 0x7f871015f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:38.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.292+0000 7f8715400640 1 -- 172.21.15.52:0/3685189221 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87040094d0 con 0x7f871015cab0 2024-03-12T17:07:38.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.292+0000 7f8715400640 1 --2- 172.21.15.52:0/3685189221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f871015cab0 0x7f8710156af0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f870800c760 tx=0x7f870800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:38.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.293+0000 7f870cc00640 1 -- 172.21.15.52:0/3685189221 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8708003da0 con 0x7f871015cab0 2024-03-12T17:07:38.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.293+0000 7f870cc00640 1 -- 172.21.15.52:0/3685189221 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870800ce70 con 0x7f871015cab0 2024-03-12T17:07:38.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.293+0000 7f870cc00640 1 -- 172.21.15.52:0/3685189221 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8708005290 con 0x7f871015cab0 2024-03-12T17:07:38.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.293+0000 7f8715e00640 1 -- 172.21.15.52:0/3685189221 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87100c6ec0 con 0x7f871015cab0 2024-03-12T17:07:38.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.293+0000 7f8715e00640 1 -- 172.21.15.52:0/3685189221 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87100c7480 con 0x7f871015cab0 2024-03-12T17:07:38.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.294+0000 7f8715e00640 1 -- 172.21.15.52:0/3685189221 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87100b8440 con 0x7f871015cab0 2024-03-12T17:07:38.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.295+0000 7f870cc00640 1 -- 172.21.15.52:0/3685189221 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f87080053f0 con 0x7f871015cab0 2024-03-12T17:07:38.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.296+0000 7f870cc00640 1 --2- 172.21.15.52:0/3685189221 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f86f006ac00 0x7f86f006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:38.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.296+0000 7f870cc00640 1 -- 172.21.15.52:0/3685189221 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f87080895e0 con 0x7f871015cab0 2024-03-12T17:07:38.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.297+0000 7f8714a00640 1 --2- 172.21.15.52:0/3685189221 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f86f006ac00 0x7f86f006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:38.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.298+0000 7f8714a00640 1 --2- 172.21.15.52:0/3685189221 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f86f006ac00 0x7f86f006d0c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8704036000 tx=0x7f8704008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:38.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.300+0000 7f870cc00640 1 -- 172.21.15.52:0/3685189221 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f8708054400 con 0x7f871015cab0 2024-03-12T17:07:38.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.497+0000 7f8715e00640 1 -- 172.21.15.52:0/3685189221 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f87100735a0 con 0x7f871015cab0 2024-03-12T17:07:38.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.498+0000 7f870cc00640 1 -- 172.21.15.52:0/3685189221 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8708057a20 con 0x7f871015cab0 2024-03-12T17:07:38.492 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-12T17:07:38.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.501+0000 7f8715e00640 1 -- 172.21.15.52:0/3685189221 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f86f006ac00 msgr2=0x7f86f006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:38.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.501+0000 7f8715e00640 1 --2- 172.21.15.52:0/3685189221 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f86f006ac00 0x7f86f006d0c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8704036000 tx=0x7f8704008030 comp rx=0 tx=0).stop 2024-03-12T17:07:38.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.501+0000 7f8715e00640 1 -- 172.21.15.52:0/3685189221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f871015cab0 msgr2=0x7f8710156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:38.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.501+0000 7f8715e00640 1 --2- 172.21.15.52:0/3685189221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f871015cab0 0x7f8710156af0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f870800c760 tx=0x7f870800cc20 comp rx=0 tx=0).stop 2024-03-12T17:07:38.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.501+0000 7f8715e00640 1 -- 172.21.15.52:0/3685189221 shutdown_connections 2024-03-12T17:07:38.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.501+0000 7f8715e00640 1 --2- 172.21.15.52:0/3685189221 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f86f006ac00 0x7f86f006d0c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:38.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.501+0000 7f8715e00640 1 --2- 172.21.15.52:0/3685189221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f871015cab0 0x7f8710156af0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:38.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.501+0000 7f8715e00640 1 --2- 172.21.15.52:0/3685189221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87100d1c30 0x7f871015f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:38.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.501+0000 7f8715e00640 1 --2- 172.21.15.52:0/3685189221 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f87100ceeb0 0x7f871015e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:38.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.501+0000 7f8715e00640 1 -- 172.21.15.52:0/3685189221 >> 172.21.15.52:0/3685189221 conn(0x7f871006ae90 msgr2=0x7f8710160710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:38.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.501+0000 7f8715e00640 1 -- 172.21.15.52:0/3685189221 shutdown_connections 2024-03-12T17:07:38.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.501+0000 7f8715e00640 1 -- 172.21.15.52:0/3685189221 wait complete. 2024-03-12T17:07:38.513 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-12T17:07:38.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.749+0000 7fa25e600640 1 Processor -- start 2024-03-12T17:07:38.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.749+0000 7fa25e600640 1 -- start start 2024-03-12T17:07:38.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.751+0000 7fa25e600640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa2580add30 0x7fa2580ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:38.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.751+0000 7fa25e600640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2580cf250 con 0x7fa2580ce930 2024-03-12T17:07:38.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.751+0000 7fa25e600640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2580cdba0 con 0x7fa25806ce90 2024-03-12T17:07:38.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.751+0000 7fa25e600640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2580ae1c0 con 0x7fa2580add30 2024-03-12T17:07:38.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.751+0000 7fa257400640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fa2580ce930 0x7fa2580cd3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:44830/0 (socket says 172.21.15.52:44830) 2024-03-12T17:07:38.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.751+0000 7fa257400640 1 -- 172.21.15.52:0/1593557603 learned_addr learned my addr 172.21.15.52:0/1593557603 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:38.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.751+0000 7fa256a00640 1 --2- 172.21.15.52:0/1593557603 >> v2:172.21.15.52:3300/0 conn(0x7fa2580add30 0x7fa2580ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:38.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.752+0000 7fa256a00640 1 -- 172.21.15.52:0/1593557603 >> v1:172.21.15.102:6789/0 conn(0x7fa25806ce90 legacy=0x7fa25815e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:38.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.752+0000 7fa256a00640 1 -- 172.21.15.52:0/1593557603 >> v1:172.21.15.52:6789/0 conn(0x7fa2580ce930 legacy=0x7fa2580cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:38.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.752+0000 7fa256a00640 1 -- 172.21.15.52:0/1593557603 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2580cde80 con 0x7fa2580add30 2024-03-12T17:07:38.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.753+0000 7fa256a00640 1 --2- 172.21.15.52:0/1593557603 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2580add30 0x7fa2580ced10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa2440097a0 tx=0x7fa24402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39575d51c802add2 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:38.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.753+0000 7fa256000640 1 -- 172.21.15.52:0/1593557603 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24402e070 con 0x7fa2580add30 2024-03-12T17:07:38.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.753+0000 7fa256000640 1 -- 172.21.15.52:0/1593557603 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2440315a0 con 0x7fa2580add30 2024-03-12T17:07:38.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.753+0000 7fa256000640 1 -- 172.21.15.52:0/1593557603 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa244031930 con 0x7fa2580add30 2024-03-12T17:07:38.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.754+0000 7fa25e600640 1 -- 172.21.15.52:0/1593557603 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2580add30 msgr2=0x7fa2580ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:38.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.754+0000 7fa25e600640 1 --2- 172.21.15.52:0/1593557603 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2580add30 0x7fa2580ced10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa2440097a0 tx=0x7fa24402b840 comp rx=0 tx=0).stop 2024-03-12T17:07:38.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.754+0000 7fa25e600640 1 -- 172.21.15.52:0/1593557603 shutdown_connections 2024-03-12T17:07:38.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.754+0000 7fa25e600640 1 --2- 172.21.15.52:0/1593557603 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2580add30 0x7fa2580ced10 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:38.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.754+0000 7fa25e600640 1 -- 172.21.15.52:0/1593557603 >> 172.21.15.52:0/1593557603 conn(0x7fa25806a0b0 msgr2=0x7fa25806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:38.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.754+0000 7fa25e600640 1 -- 172.21.15.52:0/1593557603 shutdown_connections 2024-03-12T17:07:38.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.754+0000 7fa25e600640 1 -- 172.21.15.52:0/1593557603 wait complete. 2024-03-12T17:07:38.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.756+0000 7fa25e600640 1 Processor -- start 2024-03-12T17:07:38.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.756+0000 7fa25e600640 1 -- start start 2024-03-12T17:07:38.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.757+0000 7fa25e600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa25806ce90 0x7fa258153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:38.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.757+0000 7fa25e600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2580add30 0x7fa258153270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:38.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.757+0000 7fa25e600640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa2580ce930 0x7fa25813ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:38.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.757+0000 7fa25e600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2581521d0 con 0x7fa2580add30 2024-03-12T17:07:38.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.757+0000 7fa25e600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2581524b0 con 0x7fa25806ce90 2024-03-12T17:07:38.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.757+0000 7fa25e600640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa258152790 con 0x7fa2580ce930 2024-03-12T17:07:38.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.757+0000 7fa256a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2580add30 0x7fa258153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:38.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.757+0000 7fa256a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2580add30 0x7fa258153270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51722/0 (socket says 172.21.15.52:51722) 2024-03-12T17:07:38.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.757+0000 7fa256a00640 1 -- 172.21.15.52:0/1159532086 learned_addr learned my addr 172.21.15.52:0/1159532086 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:38.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.757+0000 7fa257400640 1 --2- 172.21.15.52:0/1159532086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa25806ce90 0x7fa258153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:38.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.757+0000 7fa256a00640 1 -- 172.21.15.52:0/1159532086 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa2580ce930 msgr2=0x7fa25813ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:38.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.757+0000 7fa256a00640 1 --2- 172.21.15.52:0/1159532086 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa2580ce930 0x7fa25813ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:38.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.757+0000 7fa256a00640 1 -- 172.21.15.52:0/1159532086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa25806ce90 msgr2=0x7fa258153f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:38.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.757+0000 7fa256a00640 1 --2- 172.21.15.52:0/1159532086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa25806ce90 0x7fa258153f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:38.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.757+0000 7fa256a00640 1 -- 172.21.15.52:0/1159532086 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa244009400 con 0x7fa2580add30 2024-03-12T17:07:38.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.758+0000 7fa256a00640 1 --2- 172.21.15.52:0/1159532086 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2580add30 0x7fa258153270 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa244034040 tx=0x7fa2440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:38.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.758+0000 7fa24f400640 1 -- 172.21.15.52:0/1159532086 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa244032900 con 0x7fa2580add30 2024-03-12T17:07:38.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.758+0000 7fa24f400640 1 -- 172.21.15.52:0/1159532086 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa244032a80 con 0x7fa2580add30 2024-03-12T17:07:38.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.758+0000 7fa24f400640 1 -- 172.21.15.52:0/1159532086 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa244032e30 con 0x7fa2580add30 2024-03-12T17:07:38.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.758+0000 7fa25e600640 1 -- 172.21.15.52:0/1159532086 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa258152a70 con 0x7fa2580add30 2024-03-12T17:07:38.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.758+0000 7fa25e600640 1 -- 172.21.15.52:0/1159532086 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2581537b0 con 0x7fa2580add30 2024-03-12T17:07:38.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.760+0000 7fa25e600640 1 -- 172.21.15.52:0/1159532086 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa258152f90 con 0x7fa2580add30 2024-03-12T17:07:38.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.760+0000 7fa24f400640 1 -- 172.21.15.52:0/1159532086 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa24403b3f0 con 0x7fa2580add30 2024-03-12T17:07:38.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.762+0000 7fa24f400640 1 --2- 172.21.15.52:0/1159532086 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa23806ac50 0x7fa23806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:38.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.762+0000 7fa24f400640 1 -- 172.21.15.52:0/1159532086 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa2440ab790 con 0x7fa2580add30 2024-03-12T17:07:38.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.762+0000 7fa257400640 1 --2- 172.21.15.52:0/1159532086 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa23806ac50 0x7fa23806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:38.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.764+0000 7fa257400640 1 --2- 172.21.15.52:0/1159532086 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa23806ac50 0x7fa23806d110 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa248005880 tx=0x7fa248009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:38.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.767+0000 7fa24f400640 1 -- 172.21.15.52:0/1159532086 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fa258152f90 con 0x7fa2580add30 2024-03-12T17:07:38.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.966+0000 7fa25e600640 1 -- 172.21.15.52:0/1159532086 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa258152f90 con 0x7fa2580add30 2024-03-12T17:07:38.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.966+0000 7fa24f400640 1 -- 172.21.15.52:0/1159532086 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa258152f90 con 0x7fa2580add30 2024-03-12T17:07:38.960 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-03-12T17:07:38.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.969+0000 7fa25e600640 1 -- 172.21.15.52:0/1159532086 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa23806ac50 msgr2=0x7fa23806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:38.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.969+0000 7fa25e600640 1 --2- 172.21.15.52:0/1159532086 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa23806ac50 0x7fa23806d110 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa248005880 tx=0x7fa248009040 comp rx=0 tx=0).stop 2024-03-12T17:07:38.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.969+0000 7fa25e600640 1 -- 172.21.15.52:0/1159532086 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2580add30 msgr2=0x7fa258153270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:38.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.969+0000 7fa25e600640 1 --2- 172.21.15.52:0/1159532086 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2580add30 0x7fa258153270 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa244034040 tx=0x7fa2440046c0 comp rx=0 tx=0).stop 2024-03-12T17:07:38.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.969+0000 7fa25e600640 1 -- 172.21.15.52:0/1159532086 shutdown_connections 2024-03-12T17:07:38.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.969+0000 7fa25e600640 1 --2- 172.21.15.52:0/1159532086 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa23806ac50 0x7fa23806d110 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:38.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.969+0000 7fa25e600640 1 --2- 172.21.15.52:0/1159532086 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa2580ce930 0x7fa25813ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:38.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.970+0000 7fa25e600640 1 --2- 172.21.15.52:0/1159532086 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2580add30 0x7fa258153270 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:38.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.970+0000 7fa25e600640 1 --2- 172.21.15.52:0/1159532086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa25806ce90 0x7fa258153f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:38.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.970+0000 7fa25e600640 1 -- 172.21.15.52:0/1159532086 >> 172.21.15.52:0/1159532086 conn(0x7fa25806a0b0 msgr2=0x7fa2580cbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:38.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.970+0000 7fa25e600640 1 -- 172.21.15.52:0/1159532086 shutdown_connections 2024-03-12T17:07:38.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:38.970+0000 7fa25e600640 1 -- 172.21.15.52:0/1159532086 wait complete. 2024-03-12T17:07:38.984 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-12T17:07:39.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.215+0000 7fdbaaa00640 1 Processor -- start 2024-03-12T17:07:39.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.215+0000 7fdbaaa00640 1 -- start start 2024-03-12T17:07:39.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.217+0000 7fdbaaa00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdba40b22b0 0x7fdba415eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:39.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.217+0000 7fdbaaa00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdba415e320 con 0x7fdba415f020 2024-03-12T17:07:39.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.217+0000 7fdbaaa00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdba40cda20 con 0x7fdba415a850 2024-03-12T17:07:39.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.217+0000 7fdbaaa00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdba415ac30 con 0x7fdba40b22b0 2024-03-12T17:07:39.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.217+0000 7fdba8c00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdba40b22b0 0x7fdba415eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:39.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.218+0000 7fdba8c00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdba40b22b0 0x7fdba415eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51738/0 (socket says 172.21.15.52:51738) 2024-03-12T17:07:39.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.218+0000 7fdbaa000640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fdba415f020 0x7fdba40cacf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:44832/0 (socket says 172.21.15.52:44832) 2024-03-12T17:07:39.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.218+0000 7fdba8c00640 1 -- 172.21.15.52:0/763900591 learned_addr learned my addr 172.21.15.52:0/763900591 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:39.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.218+0000 7fdba8c00640 1 -- 172.21.15.52:0/763900591 >> v1:172.21.15.102:6789/0 conn(0x7fdba415a850 legacy=0x7fdba40cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:39.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.218+0000 7fdba8c00640 1 -- 172.21.15.52:0/763900591 >> v1:172.21.15.52:6789/0 conn(0x7fdba415f020 legacy=0x7fdba40cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:39.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.218+0000 7fdba8c00640 1 -- 172.21.15.52:0/763900591 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba40cdd00 con 0x7fdba40b22b0 2024-03-12T17:07:39.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.219+0000 7fdba8c00640 1 --2- 172.21.15.52:0/763900591 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba40b22b0 0x7fdba415eae0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009870 tx=0x7fdb9802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2facca57b37df7fe server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:39.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.220+0000 7fdba3e00640 1 -- 172.21.15.52:0/763900591 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9802f840 con 0x7fdba40b22b0 2024-03-12T17:07:39.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.220+0000 7fdba3e00640 1 -- 172.21.15.52:0/763900591 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb9802f9a0 con 0x7fdba40b22b0 2024-03-12T17:07:39.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.220+0000 7fdba3e00640 1 -- 172.21.15.52:0/763900591 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9802fd30 con 0x7fdba40b22b0 2024-03-12T17:07:39.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.220+0000 7fdbaaa00640 1 -- 172.21.15.52:0/763900591 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba40b22b0 msgr2=0x7fdba415eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:39.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.220+0000 7fdbaaa00640 1 --2- 172.21.15.52:0/763900591 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba40b22b0 0x7fdba415eae0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009870 tx=0x7fdb9802ba20 comp rx=0 tx=0).stop 2024-03-12T17:07:39.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.221+0000 7fdbaaa00640 1 -- 172.21.15.52:0/763900591 shutdown_connections 2024-03-12T17:07:39.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.221+0000 7fdbaaa00640 1 --2- 172.21.15.52:0/763900591 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba40b22b0 0x7fdba415eae0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:39.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.221+0000 7fdbaaa00640 1 -- 172.21.15.52:0/763900591 >> 172.21.15.52:0/763900591 conn(0x7fdba40b01b0 msgr2=0x7fdba40b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:39.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.221+0000 7fdbaaa00640 1 -- 172.21.15.52:0/763900591 shutdown_connections 2024-03-12T17:07:39.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.221+0000 7fdbaaa00640 1 -- 172.21.15.52:0/763900591 wait complete. 2024-03-12T17:07:39.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.222+0000 7fdbaaa00640 1 Processor -- start 2024-03-12T17:07:39.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.223+0000 7fdbaaa00640 1 -- start start 2024-03-12T17:07:39.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.223+0000 7fdbaaa00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba40b22b0 0x7fdba4147490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:39.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.223+0000 7fdbaaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdba415a850 0x7fdba41f4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:39.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.223+0000 7fdbaaa00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdba415f020 0x7fdba409be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:39.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.223+0000 7fdbaaa00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba4145650 con 0x7fdba40b22b0 2024-03-12T17:07:39.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.223+0000 7fdbaaa00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba41f4670 con 0x7fdba415a850 2024-03-12T17:07:39.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.223+0000 7fdbaaa00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba4148430 con 0x7fdba415f020 2024-03-12T17:07:39.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.224+0000 7fdba9600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba40b22b0 0x7fdba4147490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:39.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.224+0000 7fdba9600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba40b22b0 0x7fdba4147490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51748/0 (socket says 172.21.15.52:51748) 2024-03-12T17:07:39.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.224+0000 7fdba9600640 1 -- 172.21.15.52:0/3714477217 learned_addr learned my addr 172.21.15.52:0/3714477217 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:39.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.224+0000 7fdba8c00640 1 --2- 172.21.15.52:0/3714477217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdba415a850 0x7fdba41f4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:39.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.224+0000 7fdbaa000640 1 --2- 172.21.15.52:0/3714477217 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdba415f020 0x7fdba409be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:39.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.224+0000 7fdba9600640 1 -- 172.21.15.52:0/3714477217 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdba415f020 msgr2=0x7fdba409be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:39.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.224+0000 7fdba9600640 1 --2- 172.21.15.52:0/3714477217 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdba415f020 0x7fdba409be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:39.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.224+0000 7fdba9600640 1 -- 172.21.15.52:0/3714477217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdba415a850 msgr2=0x7fdba41f4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:39.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.224+0000 7fdba9600640 1 --2- 172.21.15.52:0/3714477217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdba415a850 0x7fdba41f4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:39.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.224+0000 7fdba9600640 1 -- 172.21.15.52:0/3714477217 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb980094d0 con 0x7fdba40b22b0 2024-03-12T17:07:39.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.224+0000 7fdbaa000640 1 --2- 172.21.15.52:0/3714477217 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdba415f020 0x7fdba409be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:39.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.225+0000 7fdba9600640 1 --2- 172.21.15.52:0/3714477217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba40b22b0 0x7fdba4147490 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdb9400c430 tx=0x7fdb9400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:39.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.225+0000 7fdba1600640 1 -- 172.21.15.52:0/3714477217 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb940052d0 con 0x7fdba40b22b0 2024-03-12T17:07:39.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.225+0000 7fdba1600640 1 -- 172.21.15.52:0/3714477217 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb94005430 con 0x7fdba40b22b0 2024-03-12T17:07:39.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.225+0000 7fdba1600640 1 -- 172.21.15.52:0/3714477217 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb94003830 con 0x7fdba40b22b0 2024-03-12T17:07:39.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.225+0000 7fdbaaa00640 1 -- 172.21.15.52:0/3714477217 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdba409c370 con 0x7fdba40b22b0 2024-03-12T17:07:39.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.225+0000 7fdbaaa00640 1 -- 172.21.15.52:0/3714477217 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba4147e70 con 0x7fdba40b22b0 2024-03-12T17:07:39.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.226+0000 7fdbaaa00640 1 -- 172.21.15.52:0/3714477217 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdba40735a0 con 0x7fdba40b22b0 2024-03-12T17:07:39.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.227+0000 7fdba1600640 1 -- 172.21.15.52:0/3714477217 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fdb94003990 con 0x7fdba40b22b0 2024-03-12T17:07:39.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.228+0000 7fdba1600640 1 --2- 172.21.15.52:0/3714477217 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fdb8806ac00 0x7fdb8806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:39.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.228+0000 7fdba1600640 1 -- 172.21.15.52:0/3714477217 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fdb94088cf0 con 0x7fdba40b22b0 2024-03-12T17:07:39.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.229+0000 7fdba8c00640 1 --2- 172.21.15.52:0/3714477217 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fdb8806ac00 0x7fdb8806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:39.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.230+0000 7fdba8c00640 1 --2- 172.21.15.52:0/3714477217 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fdb8806ac00 0x7fdb8806d0c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdb98036000 tx=0x7fdb98008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:39.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.233+0000 7fdba1600640 1 -- 172.21.15.52:0/3714477217 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fdb94053a70 con 0x7fdba40b22b0 2024-03-12T17:07:39.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.428+0000 7fdbaaa00640 1 -- 172.21.15.52:0/3714477217 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdba404e030 con 0x7fdba40b22b0 2024-03-12T17:07:39.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.428+0000 7fdba1600640 1 -- 172.21.15.52:0/3714477217 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdb940170a0 con 0x7fdba40b22b0 2024-03-12T17:07:39.422 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-12T17:07:39.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.431+0000 7fdbaaa00640 1 -- 172.21.15.52:0/3714477217 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fdb8806ac00 msgr2=0x7fdb8806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:39.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.431+0000 7fdbaaa00640 1 --2- 172.21.15.52:0/3714477217 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fdb8806ac00 0x7fdb8806d0c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdb98036000 tx=0x7fdb98008030 comp rx=0 tx=0).stop 2024-03-12T17:07:39.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.431+0000 7fdbaaa00640 1 -- 172.21.15.52:0/3714477217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba40b22b0 msgr2=0x7fdba4147490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:39.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.431+0000 7fdbaaa00640 1 --2- 172.21.15.52:0/3714477217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba40b22b0 0x7fdba4147490 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdb9400c430 tx=0x7fdb9400c8f0 comp rx=0 tx=0).stop 2024-03-12T17:07:39.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.431+0000 7fdbaaa00640 1 -- 172.21.15.52:0/3714477217 shutdown_connections 2024-03-12T17:07:39.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.431+0000 7fdbaaa00640 1 --2- 172.21.15.52:0/3714477217 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fdb8806ac00 0x7fdb8806d0c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:39.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.431+0000 7fdbaaa00640 1 --2- 172.21.15.52:0/3714477217 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdba415f020 0x7fdba409be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:39.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.431+0000 7fdbaaa00640 1 --2- 172.21.15.52:0/3714477217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdba415a850 0x7fdba41f4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:39.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.431+0000 7fdbaaa00640 1 --2- 172.21.15.52:0/3714477217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba40b22b0 0x7fdba4147490 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:39.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.431+0000 7fdbaaa00640 1 -- 172.21.15.52:0/3714477217 >> 172.21.15.52:0/3714477217 conn(0x7fdba40b01b0 msgr2=0x7fdba40aed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:39.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.431+0000 7fdbaaa00640 1 -- 172.21.15.52:0/3714477217 shutdown_connections 2024-03-12T17:07:39.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.431+0000 7fdbaaa00640 1 -- 172.21.15.52:0/3714477217 wait complete. 2024-03-12T17:07:39.444 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-12T17:07:39.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.665+0000 7f74da600640 1 Processor -- start 2024-03-12T17:07:39.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.666+0000 7f74da600640 1 -- start start 2024-03-12T17:07:39.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.667+0000 7f74da600640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f74d4160bf0 0x7f74d406c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:39.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.667+0000 7f74da600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f74d40ca820 0x7f74d406db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:39.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.667+0000 7f74da600640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74d406d060 con 0x7f74d406e0b0 2024-03-12T17:07:39.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.667+0000 7f74da600640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74d4160fd0 con 0x7f74d4160bf0 2024-03-12T17:07:39.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.667+0000 7f74da600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74d406caa0 con 0x7f74d40ca820 2024-03-12T17:07:39.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.668+0000 7f74d9c00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f74d406e0b0 0x7f74d40700d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:44836/0 (socket says 172.21.15.52:44836) 2024-03-12T17:07:39.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.668+0000 7f74d9c00640 1 -- 172.21.15.52:0/503816349 learned_addr learned my addr 172.21.15.52:0/503816349 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:39.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.668+0000 7f74d9200640 1 --2- 172.21.15.52:0/503816349 >> v2:172.21.15.52:3300/0 conn(0x7f74d4160bf0 0x7f74d406c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:39.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.668+0000 7f74d9200640 1 -- 172.21.15.52:0/503816349 >> v2:172.21.15.102:3300/0 conn(0x7f74d40ca820 msgr2=0x7f74d406db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:39.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.668+0000 7f74d9200640 1 --2- 172.21.15.52:0/503816349 >> v2:172.21.15.102:3300/0 conn(0x7f74d40ca820 0x7f74d406db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:39.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.668+0000 7f74d9200640 1 -- 172.21.15.52:0/503816349 >> v1:172.21.15.52:6789/0 conn(0x7f74d406e0b0 legacy=0x7f74d40700d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:39.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.668+0000 7f74d9200640 1 -- 172.21.15.52:0/503816349 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d406cd80 con 0x7f74d4160bf0 2024-03-12T17:07:39.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.669+0000 7f74d9200640 1 --2- 172.21.15.52:0/503816349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74d4160bf0 0x7f74d406c130 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f74d406a910 tx=0x7f74bc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b36674ea4974907f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:39.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.670+0000 7f74d3e00640 1 -- 172.21.15.52:0/503816349 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74bc031830 con 0x7f74d4160bf0 2024-03-12T17:07:39.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.670+0000 7f74d3e00640 1 -- 172.21.15.52:0/503816349 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74bc031990 con 0x7f74d4160bf0 2024-03-12T17:07:39.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.670+0000 7f74d3e00640 1 -- 172.21.15.52:0/503816349 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74bc031d20 con 0x7f74d4160bf0 2024-03-12T17:07:39.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.670+0000 7f74da600640 1 -- 172.21.15.52:0/503816349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74d4160bf0 msgr2=0x7f74d406c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:39.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.670+0000 7f74da600640 1 --2- 172.21.15.52:0/503816349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74d4160bf0 0x7f74d406c130 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f74d406a910 tx=0x7f74bc02b8d0 comp rx=0 tx=0).stop 2024-03-12T17:07:39.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.670+0000 7f74da600640 1 -- 172.21.15.52:0/503816349 shutdown_connections 2024-03-12T17:07:39.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.670+0000 7f74da600640 1 --2- 172.21.15.52:0/503816349 >> v2:172.21.15.102:3300/0 conn(0x7f74d40ca820 0x7f74d406db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:39.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.670+0000 7f74da600640 1 --2- 172.21.15.52:0/503816349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74d4160bf0 0x7f74d406c130 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:39.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.670+0000 7f74da600640 1 -- 172.21.15.52:0/503816349 >> 172.21.15.52:0/503816349 conn(0x7f74d4065df0 msgr2=0x7f74d4069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:39.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.670+0000 7f74da600640 1 -- 172.21.15.52:0/503816349 shutdown_connections 2024-03-12T17:07:39.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.670+0000 7f74da600640 1 -- 172.21.15.52:0/503816349 wait complete. 2024-03-12T17:07:39.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.672+0000 7f74da600640 1 Processor -- start 2024-03-12T17:07:39.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.672+0000 7f74da600640 1 -- start start 2024-03-12T17:07:39.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.673+0000 7f74da600640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74d406e0b0 0x7f74d4145450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:39.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.673+0000 7f74da600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74d40ca820 0x7f74d4158a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:39.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.673+0000 7f74da600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74d4160bf0 0x7f74d4153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:39.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.673+0000 7f74da600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d41596f0 con 0x7f74d4160bf0 2024-03-12T17:07:39.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.673+0000 7f74da600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d4157ef0 con 0x7f74d40ca820 2024-03-12T17:07:39.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.673+0000 7f74da600640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74d4157370 con 0x7f74d406e0b0 2024-03-12T17:07:39.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.673+0000 7f74d9c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74d4160bf0 0x7f74d4153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:39.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.674+0000 7f74d9c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74d4160bf0 0x7f74d4153020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51754/0 (socket says 172.21.15.52:51754) 2024-03-12T17:07:39.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.674+0000 7f74d9c00640 1 -- 172.21.15.52:0/2056569902 learned_addr learned my addr 172.21.15.52:0/2056569902 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:39.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.674+0000 7f74d9200640 1 --2- 172.21.15.52:0/2056569902 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74d406e0b0 0x7f74d4145450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:39.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.674+0000 7f74cbe00640 1 --2- 172.21.15.52:0/2056569902 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74d40ca820 0x7f74d4158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:39.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.674+0000 7f74d9c00640 1 -- 172.21.15.52:0/2056569902 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74d406e0b0 msgr2=0x7f74d4145450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:39.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.674+0000 7f74d9c00640 1 --2- 172.21.15.52:0/2056569902 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74d406e0b0 0x7f74d4145450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:39.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.674+0000 7f74d9c00640 1 -- 172.21.15.52:0/2056569902 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74d40ca820 msgr2=0x7f74d4158a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:39.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.674+0000 7f74d9c00640 1 --2- 172.21.15.52:0/2056569902 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74d40ca820 0x7f74d4158a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:39.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.674+0000 7f74d9c00640 1 -- 172.21.15.52:0/2056569902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74bc009400 con 0x7f74d4160bf0 2024-03-12T17:07:39.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.674+0000 7f74cbe00640 1 --2- 172.21.15.52:0/2056569902 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74d40ca820 0x7f74d4158a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:39.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.674+0000 7f74d9c00640 1 --2- 172.21.15.52:0/2056569902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74d4160bf0 0x7f74d4153020 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f74cc00ca80 tx=0x7f74cc00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:39.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.675+0000 7f74d1600640 1 -- 172.21.15.52:0/2056569902 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74cc00ac30 con 0x7f74d4160bf0 2024-03-12T17:07:39.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.675+0000 7f74d1600640 1 -- 172.21.15.52:0/2056569902 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74cc00ad90 con 0x7f74d4160bf0 2024-03-12T17:07:39.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.675+0000 7f74da600640 1 -- 172.21.15.52:0/2056569902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74d4157650 con 0x7f74d4160bf0 2024-03-12T17:07:39.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.675+0000 7f74d1600640 1 -- 172.21.15.52:0/2056569902 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74cc012600 con 0x7f74d4160bf0 2024-03-12T17:07:39.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.675+0000 7f74da600640 1 -- 172.21.15.52:0/2056569902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74d4157c10 con 0x7f74d4160bf0 2024-03-12T17:07:39.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.676+0000 7f74da600640 1 -- 172.21.15.52:0/2056569902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74d4071b10 con 0x7f74d4160bf0 2024-03-12T17:07:39.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.677+0000 7f74d1600640 1 -- 172.21.15.52:0/2056569902 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f74cc007070 con 0x7f74d4160bf0 2024-03-12T17:07:39.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.678+0000 7f74d1600640 1 --2- 172.21.15.52:0/2056569902 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f74b006ac00 0x7f74b006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:39.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.678+0000 7f74d1600640 1 -- 172.21.15.52:0/2056569902 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f74cc0895b0 con 0x7f74d4160bf0 2024-03-12T17:07:39.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.679+0000 7f74d9200640 1 --2- 172.21.15.52:0/2056569902 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f74b006ac00 0x7f74b006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:39.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.680+0000 7f74d9200640 1 --2- 172.21.15.52:0/2056569902 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f74b006ac00 0x7f74b006d0c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f74bc036000 tx=0x7f74bc03c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:39.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.683+0000 7f74d1600640 1 -- 172.21.15.52:0/2056569902 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f74cc0543e0 con 0x7f74d4160bf0 2024-03-12T17:07:39.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.895+0000 7f74da600640 1 -- 172.21.15.52:0/2056569902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f74d40b23a0 con 0x7f74d4160bf0 2024-03-12T17:07:39.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.895+0000 7f74d1600640 1 -- 172.21.15.52:0/2056569902 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f74cc057a00 con 0x7f74d4160bf0 2024-03-12T17:07:39.889 INFO:teuthology.orchestra.run.smithi052.stderr:listed 0 entries 2024-03-12T17:07:39.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.898+0000 7f74da600640 1 -- 172.21.15.52:0/2056569902 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f74b006ac00 msgr2=0x7f74b006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:39.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.898+0000 7f74da600640 1 --2- 172.21.15.52:0/2056569902 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f74b006ac00 0x7f74b006d0c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f74bc036000 tx=0x7f74bc03c040 comp rx=0 tx=0).stop 2024-03-12T17:07:39.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.898+0000 7f74da600640 1 -- 172.21.15.52:0/2056569902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74d4160bf0 msgr2=0x7f74d4153020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:39.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.898+0000 7f74da600640 1 --2- 172.21.15.52:0/2056569902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74d4160bf0 0x7f74d4153020 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f74cc00ca80 tx=0x7f74cc00cf40 comp rx=0 tx=0).stop 2024-03-12T17:07:39.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.898+0000 7f74da600640 1 -- 172.21.15.52:0/2056569902 shutdown_connections 2024-03-12T17:07:39.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.898+0000 7f74da600640 1 --2- 172.21.15.52:0/2056569902 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f74b006ac00 0x7f74b006d0c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:39.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.898+0000 7f74da600640 1 --2- 172.21.15.52:0/2056569902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74d4160bf0 0x7f74d4153020 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:39.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.898+0000 7f74da600640 1 --2- 172.21.15.52:0/2056569902 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74d40ca820 0x7f74d4158a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:39.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.898+0000 7f74da600640 1 --2- 172.21.15.52:0/2056569902 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74d406e0b0 0x7f74d4145450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:39.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.898+0000 7f74da600640 1 -- 172.21.15.52:0/2056569902 >> 172.21.15.52:0/2056569902 conn(0x7f74d4065df0 msgr2=0x7f74d41611a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:39.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.899+0000 7f74da600640 1 -- 172.21.15.52:0/2056569902 shutdown_connections 2024-03-12T17:07:39.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:39.899+0000 7f74da600640 1 -- 172.21.15.52:0/2056569902 wait complete. 2024-03-12T17:07:39.911 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-12T17:07:39.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:39.912 DEBUG:teuthology.orchestra.run.smithi102:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-03-12T17:07:40.020 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi102.front.sepia.ceph.com 2024-03-12T17:07:40.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:40.021 DEBUG:teuthology.orchestra.run.smithi102:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-12T17:07:40.072 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:07:40.072 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-12T17:07:40.131 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:07:40.131 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-12T17:07:40.186 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:07:40.186 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-12T17:07:40.240 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:07:40.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.467+0000 7fb437c00640 1 Processor -- start 2024-03-12T17:07:40.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.468+0000 7fb437c00640 1 -- start start 2024-03-12T17:07:40.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.469+0000 7fb437c00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb4300b0ae0 0x7fb43006afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:40.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.469+0000 7fb437c00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb4300b1200 0x7fb43006b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:40.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.469+0000 7fb437c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb43006bcf0 0x7fb430160860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:40.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.469+0000 7fb437c00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb43006c140 con 0x7fb4300b0ae0 2024-03-12T17:07:40.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.469+0000 7fb437c00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4300cf310 con 0x7fb43006bcf0 2024-03-12T17:07:40.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.469+0000 7fb437c00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb430098a40 con 0x7fb4300b1200 2024-03-12T17:07:40.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.470+0000 7fb435200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb4300b0ae0 0x7fb43006afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:40.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.470+0000 7fb435200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb4300b0ae0 0x7fb43006afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51756/0 (socket says 172.21.15.52:51756) 2024-03-12T17:07:40.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.470+0000 7fb435200640 1 -- 172.21.15.52:0/4163653636 learned_addr learned my addr 172.21.15.52:0/4163653636 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:40.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.470+0000 7fb427e00640 1 --2- 172.21.15.52:0/4163653636 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb4300b1200 0x7fb43006b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:40.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.470+0000 7fb435c00640 1 --2- 172.21.15.52:0/4163653636 >> v2:172.21.15.102:3300/0 conn(0x7fb43006bcf0 0x7fb430160860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:40.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.470+0000 7fb435200640 1 -- 172.21.15.52:0/4163653636 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb4300b1200 msgr2=0x7fb43006b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:40.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.470+0000 7fb435200640 1 --2- 172.21.15.52:0/4163653636 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb4300b1200 0x7fb43006b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:40.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.470+0000 7fb435200640 1 -- 172.21.15.52:0/4163653636 >> v2:172.21.15.102:3300/0 conn(0x7fb43006bcf0 msgr2=0x7fb430160860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:40.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.470+0000 7fb435200640 1 --2- 172.21.15.52:0/4163653636 >> v2:172.21.15.102:3300/0 conn(0x7fb43006bcf0 0x7fb430160860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:40.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.470+0000 7fb435200640 1 -- 172.21.15.52:0/4163653636 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4300b19f0 con 0x7fb4300b0ae0 2024-03-12T17:07:40.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.470+0000 7fb427e00640 1 --2- 172.21.15.52:0/4163653636 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb4300b1200 0x7fb43006b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:40.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.470+0000 7fb435c00640 1 --2- 172.21.15.52:0/4163653636 >> v2:172.21.15.102:3300/0 conn(0x7fb43006bcf0 0x7fb430160860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:40.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.471+0000 7fb435200640 1 --2- 172.21.15.52:0/4163653636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4300b0ae0 0x7fb43006afb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb4300cd0a0 tx=0x7fb41802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed22a805c8032777 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:40.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.472+0000 7fb42fe00640 1 -- 172.21.15.52:0/4163653636 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb418031860 con 0x7fb4300b0ae0 2024-03-12T17:07:40.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.472+0000 7fb42fe00640 1 -- 172.21.15.52:0/4163653636 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4180319c0 con 0x7fb4300b0ae0 2024-03-12T17:07:40.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.472+0000 7fb42fe00640 1 -- 172.21.15.52:0/4163653636 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb418031d50 con 0x7fb4300b0ae0 2024-03-12T17:07:40.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.472+0000 7fb437c00640 1 -- 172.21.15.52:0/4163653636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4300b0ae0 msgr2=0x7fb43006afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:40.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.472+0000 7fb437c00640 1 --2- 172.21.15.52:0/4163653636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4300b0ae0 0x7fb43006afb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb4300cd0a0 tx=0x7fb41802b900 comp rx=0 tx=0).stop 2024-03-12T17:07:40.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.473+0000 7fb437c00640 1 -- 172.21.15.52:0/4163653636 shutdown_connections 2024-03-12T17:07:40.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.473+0000 7fb437c00640 1 --2- 172.21.15.52:0/4163653636 >> v2:172.21.15.102:3300/0 conn(0x7fb43006bcf0 0x7fb430160860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:40.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.473+0000 7fb437c00640 1 --2- 172.21.15.52:0/4163653636 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb4300b1200 0x7fb43006b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:40.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.473+0000 7fb437c00640 1 --2- 172.21.15.52:0/4163653636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4300b0ae0 0x7fb43006afb0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:40.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.473+0000 7fb437c00640 1 -- 172.21.15.52:0/4163653636 >> 172.21.15.52:0/4163653636 conn(0x7fb4300b2160 msgr2=0x7fb4300b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:40.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.473+0000 7fb437c00640 1 -- 172.21.15.52:0/4163653636 shutdown_connections 2024-03-12T17:07:40.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.473+0000 7fb437c00640 1 -- 172.21.15.52:0/4163653636 wait complete. 2024-03-12T17:07:40.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.474+0000 7fb437c00640 1 Processor -- start 2024-03-12T17:07:40.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.475+0000 7fb437c00640 1 -- start start 2024-03-12T17:07:40.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.476+0000 7fb437c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb43006bcf0 0x7fb4301f2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:40.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.476+0000 7fb437c00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb4300b0ae0 0x7fb4301f5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:40.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.476+0000 7fb437c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4300b1200 0x7fb4301f36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:40.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.476+0000 7fb437c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4300b45a0 con 0x7fb43006bcf0 2024-03-12T17:07:40.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.476+0000 7fb437c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4301f34a0 con 0x7fb4300b1200 2024-03-12T17:07:40.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.476+0000 7fb437c00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4301f5860 con 0x7fb4300b0ae0 2024-03-12T17:07:40.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.476+0000 7fb435200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb43006bcf0 0x7fb4301f2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:40.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.476+0000 7fb435200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb43006bcf0 0x7fb4301f2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51760/0 (socket says 172.21.15.52:51760) 2024-03-12T17:07:40.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.476+0000 7fb435200640 1 -- 172.21.15.52:0/152552375 learned_addr learned my addr 172.21.15.52:0/152552375 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:40.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.476+0000 7fb427e00640 1 --2- 172.21.15.52:0/152552375 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb4300b0ae0 0x7fb4301f5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:40.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.476+0000 7fb435c00640 1 --2- 172.21.15.52:0/152552375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4300b1200 0x7fb4301f36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:40.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.476+0000 7fb435200640 1 -- 172.21.15.52:0/152552375 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb4300b0ae0 msgr2=0x7fb4301f5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:40.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.476+0000 7fb435200640 1 --2- 172.21.15.52:0/152552375 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb4300b0ae0 0x7fb4301f5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:40.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.476+0000 7fb435200640 1 -- 172.21.15.52:0/152552375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4300b1200 msgr2=0x7fb4301f36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:40.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.476+0000 7fb435200640 1 --2- 172.21.15.52:0/152552375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4300b1200 0x7fb4301f36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:40.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.476+0000 7fb435200640 1 -- 172.21.15.52:0/152552375 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4180094d0 con 0x7fb43006bcf0 2024-03-12T17:07:40.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.476+0000 7fb435c00640 1 --2- 172.21.15.52:0/152552375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4300b1200 0x7fb4301f36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:40.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.477+0000 7fb435200640 1 --2- 172.21.15.52:0/152552375 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb43006bcf0 0x7fb4301f2f60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb418000c00 tx=0x7fb4180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:40.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.477+0000 7fb42d600640 1 -- 172.21.15.52:0/152552375 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb418031850 con 0x7fb43006bcf0 2024-03-12T17:07:40.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.477+0000 7fb42d600640 1 -- 172.21.15.52:0/152552375 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb418044400 con 0x7fb43006bcf0 2024-03-12T17:07:40.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.477+0000 7fb437c00640 1 -- 172.21.15.52:0/152552375 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4300c9500 con 0x7fb43006bcf0 2024-03-12T17:07:40.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.477+0000 7fb42d600640 1 -- 172.21.15.52:0/152552375 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4180447b0 con 0x7fb43006bcf0 2024-03-12T17:07:40.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.477+0000 7fb437c00640 1 -- 172.21.15.52:0/152552375 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4300c9ac0 con 0x7fb43006bcf0 2024-03-12T17:07:40.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.478+0000 7fb437c00640 1 -- 172.21.15.52:0/152552375 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb430073390 con 0x7fb43006bcf0 2024-03-12T17:07:40.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.479+0000 7fb42d600640 1 -- 172.21.15.52:0/152552375 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb41804d430 con 0x7fb43006bcf0 2024-03-12T17:07:40.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.480+0000 7fb42d600640 1 --2- 172.21.15.52:0/152552375 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb40c06ac00 0x7fb40c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:40.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.480+0000 7fb42d600640 1 -- 172.21.15.52:0/152552375 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb4180abf30 con 0x7fb43006bcf0 2024-03-12T17:07:40.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.481+0000 7fb427e00640 1 --2- 172.21.15.52:0/152552375 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb40c06ac00 0x7fb40c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:40.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.482+0000 7fb427e00640 1 --2- 172.21.15.52:0/152552375 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb40c06ac00 0x7fb40c06d0c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb420006fd0 tx=0x7fb420008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:40.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.484+0000 7fb42d600640 1 -- 172.21.15.52:0/152552375 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fb418076df0 con 0x7fb43006bcf0 2024-03-12T17:07:40.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.677+0000 7fb437c00640 1 -- 172.21.15.52:0/152552375 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb43006f3d0 con 0x7fb43006bcf0 2024-03-12T17:07:40.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.679+0000 7fb42d600640 1 -- 172.21.15.52:0/152552375 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18207 (secure 0 0 0) 0x7fb41807a410 con 0x7fb43006bcf0 2024-03-12T17:07:40.673 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:07:40.674 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":27,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:07:12.969614+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T17:06:19.218543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T17:06:23.525474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T17:06:24.911574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-12T17:06:28.017172+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.733031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.775178+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.591522+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.820838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.829252+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.582481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.18","mappings":[{"from":3,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:07:40.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.682+0000 7fb437c00640 1 -- 172.21.15.52:0/152552375 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb40c06ac00 msgr2=0x7fb40c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:40.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.682+0000 7fb437c00640 1 --2- 172.21.15.52:0/152552375 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb40c06ac00 0x7fb40c06d0c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb420006fd0 tx=0x7fb420008040 comp rx=0 tx=0).stop 2024-03-12T17:07:40.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.682+0000 7fb437c00640 1 -- 172.21.15.52:0/152552375 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb43006bcf0 msgr2=0x7fb4301f2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:40.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.682+0000 7fb437c00640 1 --2- 172.21.15.52:0/152552375 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb43006bcf0 0x7fb4301f2f60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb418000c00 tx=0x7fb4180046c0 comp rx=0 tx=0).stop 2024-03-12T17:07:40.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.682+0000 7fb437c00640 1 -- 172.21.15.52:0/152552375 shutdown_connections 2024-03-12T17:07:40.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.682+0000 7fb437c00640 1 --2- 172.21.15.52:0/152552375 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb40c06ac00 0x7fb40c06d0c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:40.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.682+0000 7fb437c00640 1 --2- 172.21.15.52:0/152552375 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb4300b0ae0 0x7fb4301f5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:40.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.682+0000 7fb437c00640 1 --2- 172.21.15.52:0/152552375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4300b1200 0x7fb4301f36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:40.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.682+0000 7fb437c00640 1 --2- 172.21.15.52:0/152552375 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb43006bcf0 0x7fb4301f2f60 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:40.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.682+0000 7fb437c00640 1 -- 172.21.15.52:0/152552375 >> 172.21.15.52:0/152552375 conn(0x7fb4300b2160 msgr2=0x7fb43006ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:40.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.683+0000 7fb437c00640 1 -- 172.21.15.52:0/152552375 shutdown_connections 2024-03-12T17:07:40.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.683+0000 7fb437c00640 1 -- 172.21.15.52:0/152552375 wait complete. 2024-03-12T17:07:40.696 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T17:06:19.218543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-12T17:06:23.525474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-12T17:06:24.911574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-12T17:06:28.017172+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-12T17:07:40.696 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T17:07:40.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.919+0000 7f6c2ee00640 1 Processor -- start 2024-03-12T17:07:40.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.920+0000 7f6c2ee00640 1 -- start start 2024-03-12T17:07:40.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.921+0000 7f6c2ee00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6c281610a0 0x7f6c280b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:40.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.921+0000 7f6c2ee00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c28099130 con 0x7f6c280995c0 2024-03-12T17:07:40.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.921+0000 7f6c2ee00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c2806b180 con 0x7f6c280af5e0 2024-03-12T17:07:40.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.921+0000 7f6c2ee00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c28098ae0 con 0x7f6c281610a0 2024-03-12T17:07:40.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.921+0000 7f6c2e400640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f6c280995c0 0x7f6c280c34f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:44842/0 (socket says 172.21.15.52:44842) 2024-03-12T17:07:40.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.921+0000 7f6c2e400640 1 -- 172.21.15.52:0/1790293444 learned_addr learned my addr 172.21.15.52:0/1790293444 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:40.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.922+0000 7f6c2da00640 1 --2- 172.21.15.52:0/1790293444 >> v2:172.21.15.102:3300/0 conn(0x7f6c281610a0 0x7f6c280b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:40.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.922+0000 7f6c27e00640 1 -- 172.21.15.52:0/1790293444 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1692578493 0 0) 0x7f6c28099130 con 0x7f6c280995c0 2024-03-12T17:07:40.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.922+0000 7f6c27e00640 1 -- 172.21.15.52:0/1790293444 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c0c003520 con 0x7f6c280995c0 2024-03-12T17:07:40.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.922+0000 7f6c27e00640 1 -- 172.21.15.52:0/1790293444 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4032136456 0 0) 0x7f6c0c003520 con 0x7f6c280995c0 2024-03-12T17:07:40.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.922+0000 7f6c27e00640 1 -- 172.21.15.52:0/1790293444 >> v2:172.21.15.102:3300/0 conn(0x7f6c281610a0 msgr2=0x7f6c280b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:40.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.922+0000 7f6c27e00640 1 --2- 172.21.15.52:0/1790293444 >> v2:172.21.15.102:3300/0 conn(0x7f6c281610a0 0x7f6c280b2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:40.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.922+0000 7f6c27e00640 1 -- 172.21.15.52:0/1790293444 >> v1:172.21.15.102:6789/0 conn(0x7f6c280af5e0 legacy=0x7f6c280cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:40.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.922+0000 7f6c27e00640 1 -- 172.21.15.52:0/1790293444 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c28098dc0 con 0x7f6c280995c0 2024-03-12T17:07:40.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.922+0000 7f6c27e00640 1 -- 172.21.15.52:0/1790293444 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f6c200019f0 con 0x7f6c280995c0 2024-03-12T17:07:40.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.922+0000 7f6c27e00640 1 -- 172.21.15.52:0/1790293444 >> v1:172.21.15.52:6789/0 conn(0x7f6c280995c0 legacy=0x7f6c280c34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:40.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.923+0000 7f6c27e00640 1 --2- 172.21.15.52:0/1790293444 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c0c003970 0x7f6c0c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:40.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.923+0000 7f6c27e00640 1 --2- 172.21.15.52:0/1790293444 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c0c0047c0 0x7f6c0c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:40.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.923+0000 7f6c27e00640 1 --2- 172.21.15.52:0/1790293444 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c0c005180 0x7f6c0c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:40.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.923+0000 7f6c27e00640 1 -- 172.21.15.52:0/1790293444 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c28098ae0 con 0x7f6c0c0047c0 2024-03-12T17:07:40.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.923+0000 7f6c27e00640 1 -- 172.21.15.52:0/1790293444 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c00c120 con 0x7f6c0c005180 2024-03-12T17:07:40.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.923+0000 7f6c27e00640 1 -- 172.21.15.52:0/1790293444 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c00c2e0 con 0x7f6c0c003970 2024-03-12T17:07:40.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.923+0000 7f6c2d000640 1 --2- 172.21.15.52:0/1790293444 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c0c0047c0 0x7f6c0c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:40.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.923+0000 7f6c2da00640 1 --2- 172.21.15.52:0/1790293444 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c0c003970 0x7f6c0c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:40.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.923+0000 7f6c2d000640 1 -- 172.21.15.52:0/1790293444 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c0c003970 msgr2=0x7f6c0c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:40.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.923+0000 7f6c2d000640 1 --2- 172.21.15.52:0/1790293444 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c0c003970 0x7f6c0c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:40.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.923+0000 7f6c2d000640 1 -- 172.21.15.52:0/1790293444 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c0c005180 msgr2=0x7f6c0c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:40.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.923+0000 7f6c2d000640 1 --2- 172.21.15.52:0/1790293444 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c0c005180 0x7f6c0c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:40.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.923+0000 7f6c2d000640 1 -- 172.21.15.52:0/1790293444 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c0c003520 con 0x7f6c0c0047c0 2024-03-12T17:07:40.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.923+0000 7f6c2e400640 1 -- 172.21.15.52:0/1790293444 reap_dead start 2024-03-12T17:07:40.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.925+0000 7f6c2d000640 1 --2- 172.21.15.52:0/1790293444 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c0c0047c0 0x7f6c0c004c40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6c14001720 tx=0x7f6c0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8361214aae1f1157 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:40.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.925+0000 7f6c27e00640 1 -- 172.21.15.52:0/1790293444 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c140052d0 con 0x7f6c0c0047c0 2024-03-12T17:07:40.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.925+0000 7f6c27e00640 1 -- 172.21.15.52:0/1790293444 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1402e050 con 0x7f6c0c0047c0 2024-03-12T17:07:40.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.925+0000 7f6c27e00640 1 -- 172.21.15.52:0/1790293444 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c14003830 con 0x7f6c0c0047c0 2024-03-12T17:07:40.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.925+0000 7f6c2ee00640 1 -- 172.21.15.52:0/1790293444 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c0c0047c0 msgr2=0x7f6c0c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:40.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.925+0000 7f6c2ee00640 1 --2- 172.21.15.52:0/1790293444 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c0c0047c0 0x7f6c0c004c40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6c14001720 tx=0x7f6c0c0057f0 comp rx=0 tx=0).stop 2024-03-12T17:07:40.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.925+0000 7f6c2ee00640 1 -- 172.21.15.52:0/1790293444 shutdown_connections 2024-03-12T17:07:40.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.925+0000 7f6c2ee00640 1 --2- 172.21.15.52:0/1790293444 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c0c0047c0 0x7f6c0c004c40 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:40.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.925+0000 7f6c2ee00640 1 -- 172.21.15.52:0/1790293444 >> 172.21.15.52:0/1790293444 conn(0x7f6c280b2010 msgr2=0x7f6c280b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:40.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.926+0000 7f6c2ee00640 1 -- 172.21.15.52:0/1790293444 shutdown_connections 2024-03-12T17:07:40.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.926+0000 7f6c2ee00640 1 -- 172.21.15.52:0/1790293444 wait complete. 2024-03-12T17:07:40.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.927+0000 7f6c2ee00640 1 Processor -- start 2024-03-12T17:07:40.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.927+0000 7f6c2ee00640 1 -- start start 2024-03-12T17:07:40.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.929+0000 7f6c2ee00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c0c0047c0 0x7f6c281585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:40.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.929+0000 7f6c2ee00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c280b3aa0 0x7f6c280b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:40.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.929+0000 7f6c2ee00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c281f9bd0 0x7f6c281fa050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:40.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.929+0000 7f6c2ee00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c281fa590 con 0x7f6c281f9bd0 2024-03-12T17:07:40.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.929+0000 7f6c2ee00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2815f660 con 0x7f6c0c0047c0 2024-03-12T17:07:40.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.929+0000 7f6c2ee00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c280afff0 con 0x7f6c280b3aa0 2024-03-12T17:07:40.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.929+0000 7f6c2e400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c281f9bd0 0x7f6c281fa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:40.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.929+0000 7f6c2e400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c281f9bd0 0x7f6c281fa050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51772/0 (socket says 172.21.15.52:51772) 2024-03-12T17:07:40.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.929+0000 7f6c2e400640 1 -- 172.21.15.52:0/3958149012 learned_addr learned my addr 172.21.15.52:0/3958149012 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:40.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.929+0000 7f6c2d000640 1 --2- 172.21.15.52:0/3958149012 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c280b3aa0 0x7f6c280b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:40.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.929+0000 7f6c2da00640 1 --2- 172.21.15.52:0/3958149012 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c0c0047c0 0x7f6c281585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:40.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.930+0000 7f6c2e400640 1 -- 172.21.15.52:0/3958149012 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c280b3aa0 msgr2=0x7f6c280b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:40.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.930+0000 7f6c2e400640 1 --2- 172.21.15.52:0/3958149012 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c280b3aa0 0x7f6c280b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:40.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.930+0000 7f6c2e400640 1 -- 172.21.15.52:0/3958149012 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c0c0047c0 msgr2=0x7f6c281585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:40.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.930+0000 7f6c2e400640 1 --2- 172.21.15.52:0/3958149012 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c0c0047c0 0x7f6c281585f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:40.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.930+0000 7f6c2e400640 1 -- 172.21.15.52:0/3958149012 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c2815f0a0 con 0x7f6c281f9bd0 2024-03-12T17:07:40.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.930+0000 7f6c2da00640 1 --2- 172.21.15.52:0/3958149012 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c0c0047c0 0x7f6c281585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:40.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.930+0000 7f6c2e400640 1 --2- 172.21.15.52:0/3958149012 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c281f9bd0 0x7f6c281fa050 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6c280ca940 tx=0x7f6c2000cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:40.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.930+0000 7f6c25600640 1 -- 172.21.15.52:0/3958149012 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2000ce90 con 0x7f6c281f9bd0 2024-03-12T17:07:40.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.930+0000 7f6c25600640 1 -- 172.21.15.52:0/3958149012 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2000a610 con 0x7f6c281f9bd0 2024-03-12T17:07:40.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.930+0000 7f6c25600640 1 -- 172.21.15.52:0/3958149012 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2000a9c0 con 0x7f6c281f9bd0 2024-03-12T17:07:40.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.930+0000 7f6c2ee00640 1 -- 172.21.15.52:0/3958149012 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c2815f380 con 0x7f6c281f9bd0 2024-03-12T17:07:40.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.930+0000 7f6c2ee00640 1 -- 172.21.15.52:0/3958149012 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c280c9610 con 0x7f6c281f9bd0 2024-03-12T17:07:40.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.932+0000 7f6c2ee00640 1 -- 172.21.15.52:0/3958149012 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c2804e030 con 0x7f6c281f9bd0 2024-03-12T17:07:40.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.932+0000 7f6c25600640 1 -- 172.21.15.52:0/3958149012 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6c2000ab20 con 0x7f6c281f9bd0 2024-03-12T17:07:40.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.934+0000 7f6c25600640 1 --2- 172.21.15.52:0/3958149012 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f6c0806ac00 0x7f6c0806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:40.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.934+0000 7f6c25600640 1 -- 172.21.15.52:0/3958149012 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6c20088f50 con 0x7f6c281f9bd0 2024-03-12T17:07:40.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.935+0000 7f6c2da00640 1 --2- 172.21.15.52:0/3958149012 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f6c0806ac00 0x7f6c0806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:40.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.936+0000 7f6c2da00640 1 --2- 172.21.15.52:0/3958149012 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f6c0806ac00 0x7f6c0806d0c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6c18009a80 tx=0x7f6c18008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:40.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:40.939+0000 7f6c25600640 1 -- 172.21.15.52:0/3958149012 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f6c20053d60 con 0x7f6c281f9bd0 2024-03-12T17:07:41.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.138+0000 7f6c2ee00640 1 -- 172.21.15.52:0/3958149012 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6c280b0130 con 0x7f6c281f9bd0 2024-03-12T17:07:41.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.140+0000 7f6c25600640 1 -- 172.21.15.52:0/3958149012 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6c2001a070 con 0x7f6c281f9bd0 2024-03-12T17:07:41.134 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-03-12T17:07:41.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.143+0000 7f6c2ee00640 1 -- 172.21.15.52:0/3958149012 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f6c0806ac00 msgr2=0x7f6c0806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:41.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.143+0000 7f6c2ee00640 1 --2- 172.21.15.52:0/3958149012 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f6c0806ac00 0x7f6c0806d0c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6c18009a80 tx=0x7f6c18008040 comp rx=0 tx=0).stop 2024-03-12T17:07:41.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.143+0000 7f6c2ee00640 1 -- 172.21.15.52:0/3958149012 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c281f9bd0 msgr2=0x7f6c281fa050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:41.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.143+0000 7f6c2ee00640 1 --2- 172.21.15.52:0/3958149012 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c281f9bd0 0x7f6c281fa050 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6c280ca940 tx=0x7f6c2000cb80 comp rx=0 tx=0).stop 2024-03-12T17:07:41.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.143+0000 7f6c2ee00640 1 -- 172.21.15.52:0/3958149012 shutdown_connections 2024-03-12T17:07:41.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.143+0000 7f6c2ee00640 1 --2- 172.21.15.52:0/3958149012 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f6c0806ac00 0x7f6c0806d0c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:41.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.143+0000 7f6c2ee00640 1 --2- 172.21.15.52:0/3958149012 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c280b3aa0 0x7f6c280b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:41.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.143+0000 7f6c2ee00640 1 --2- 172.21.15.52:0/3958149012 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c281f9bd0 0x7f6c281fa050 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:41.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.143+0000 7f6c2ee00640 1 --2- 172.21.15.52:0/3958149012 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c0c0047c0 0x7f6c281585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:41.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.143+0000 7f6c2ee00640 1 -- 172.21.15.52:0/3958149012 >> 172.21.15.52:0/3958149012 conn(0x7f6c280b2010 msgr2=0x7f6c280cebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:41.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.144+0000 7f6c2ee00640 1 -- 172.21.15.52:0/3958149012 shutdown_connections 2024-03-12T17:07:41.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.144+0000 7f6c2ee00640 1 -- 172.21.15.52:0/3958149012 wait complete. 2024-03-12T17:07:41.157 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-12T17:07:41.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.385+0000 7f2c0b000640 1 Processor -- start 2024-03-12T17:07:41.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.386+0000 7f2c0b000640 1 -- start start 2024-03-12T17:07:41.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.387+0000 7f2c0b000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c041571d0 0x7f2c040b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:41.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.387+0000 7f2c0b000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2c040cdc00 0x7f2c0415aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:41.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.387+0000 7f2c0b000640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c041575b0 con 0x7f2c0415afe0 2024-03-12T17:07:41.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.387+0000 7f2c0b000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c04159f20 con 0x7f2c040cdc00 2024-03-12T17:07:41.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.387+0000 7f2c0b000640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0415a200 con 0x7f2c041571d0 2024-03-12T17:07:41.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.388+0000 7f2c0a600640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f2c0415afe0 0x7f2c040cae50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:44846/0 (socket says 172.21.15.52:44846) 2024-03-12T17:07:41.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.388+0000 7f2c0a600640 1 -- 172.21.15.52:0/3706011103 learned_addr learned my addr 172.21.15.52:0/3706011103 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:41.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.388+0000 7f2c09200640 1 --2- 172.21.15.52:0/3706011103 >> v2:172.21.15.102:3300/0 conn(0x7f2c040cdc00 0x7f2c0415aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:41.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.388+0000 7f2c09c00640 1 --2- 172.21.15.52:0/3706011103 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c041571d0 0x7f2c040b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:41.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.388+0000 7f2c03e00640 1 -- 172.21.15.52:0/3706011103 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 929349776 0 0) 0x7f2c041575b0 con 0x7f2c0415afe0 2024-03-12T17:07:41.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.388+0000 7f2c03e00640 1 -- 172.21.15.52:0/3706011103 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bec003520 con 0x7f2c0415afe0 2024-03-12T17:07:41.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.389+0000 7f2c03e00640 1 -- 172.21.15.52:0/3706011103 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1722926933 0 0) 0x7f2bec003520 con 0x7f2c0415afe0 2024-03-12T17:07:41.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.389+0000 7f2c03e00640 1 -- 172.21.15.52:0/3706011103 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c041571d0 msgr2=0x7f2c040b35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:41.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.389+0000 7f2c03e00640 1 --2- 172.21.15.52:0/3706011103 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c041571d0 0x7f2c040b35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:41.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.389+0000 7f2c03e00640 1 -- 172.21.15.52:0/3706011103 >> v2:172.21.15.102:3300/0 conn(0x7f2c040cdc00 msgr2=0x7f2c0415aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:41.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.389+0000 7f2c03e00640 1 --2- 172.21.15.52:0/3706011103 >> v2:172.21.15.102:3300/0 conn(0x7f2c040cdc00 0x7f2c0415aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:41.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.389+0000 7f2c03e00640 1 -- 172.21.15.52:0/3706011103 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c0415a4e0 con 0x7f2c0415afe0 2024-03-12T17:07:41.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.389+0000 7f2c03e00640 1 -- 172.21.15.52:0/3706011103 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f2bfc0019f0 con 0x7f2c0415afe0 2024-03-12T17:07:41.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.389+0000 7f2c03e00640 1 -- 172.21.15.52:0/3706011103 >> v1:172.21.15.52:6789/0 conn(0x7f2c0415afe0 legacy=0x7f2c040cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:41.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.389+0000 7f2c03e00640 1 --2- 172.21.15.52:0/3706011103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bec003970 0x7f2bec003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:41.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.389+0000 7f2c03e00640 1 --2- 172.21.15.52:0/3706011103 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c041571d0 0x7f2bec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:41.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.389+0000 7f2c03e00640 1 --2- 172.21.15.52:0/3706011103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bec004d00 0x7f2bec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:41.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.389+0000 7f2c03e00640 1 -- 172.21.15.52:0/3706011103 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c04159f20 con 0x7f2bec004d00 2024-03-12T17:07:41.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.389+0000 7f2c03e00640 1 -- 172.21.15.52:0/3706011103 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0415a200 con 0x7f2bec003970 2024-03-12T17:07:41.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.389+0000 7f2c03e00640 1 -- 172.21.15.52:0/3706011103 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec00b980 con 0x7f2c041571d0 2024-03-12T17:07:41.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.389+0000 7f2c0a600640 1 --2- 172.21.15.52:0/3706011103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bec004d00 0x7f2bec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:41.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.390+0000 7f2c09c00640 1 --2- 172.21.15.52:0/3706011103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bec003970 0x7f2bec003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:41.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.390+0000 7f2c09200640 1 --2- 172.21.15.52:0/3706011103 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c041571d0 0x7f2bec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:41.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.390+0000 7f2c0a600640 1 -- 172.21.15.52:0/3706011103 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c041571d0 msgr2=0x7f2bec0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:41.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.390+0000 7f2c0a600640 1 --2- 172.21.15.52:0/3706011103 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c041571d0 0x7f2bec0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:41.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.390+0000 7f2c0a600640 1 -- 172.21.15.52:0/3706011103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bec003970 msgr2=0x7f2bec003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:41.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.390+0000 7f2c0a600640 1 --2- 172.21.15.52:0/3706011103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bec003970 0x7f2bec003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:41.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.390+0000 7f2c0a600640 1 -- 172.21.15.52:0/3706011103 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bec003520 con 0x7f2bec004d00 2024-03-12T17:07:41.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.390+0000 7f2c09200640 1 --2- 172.21.15.52:0/3706011103 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c041571d0 0x7f2bec0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:41.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.391+0000 7f2c0a600640 1 --2- 172.21.15.52:0/3706011103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bec004d00 0x7f2bec005180 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc00a790 tx=0x7f2bec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b5ebe18a4aa7e6d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:41.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.391+0000 7f2c03e00640 1 -- 172.21.15.52:0/3706011103 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc034630 con 0x7f2bec004d00 2024-03-12T17:07:41.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.391+0000 7f2c03e00640 1 -- 172.21.15.52:0/3706011103 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bfc02f070 con 0x7f2bec004d00 2024-03-12T17:07:41.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.391+0000 7f2c03e00640 1 -- 172.21.15.52:0/3706011103 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc0349f0 con 0x7f2bec004d00 2024-03-12T17:07:41.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.391+0000 7f2c0b000640 1 -- 172.21.15.52:0/3706011103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bec004d00 msgr2=0x7f2bec005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:41.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.391+0000 7f2c0b000640 1 --2- 172.21.15.52:0/3706011103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bec004d00 0x7f2bec005180 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc00a790 tx=0x7f2bec0057f0 comp rx=0 tx=0).stop 2024-03-12T17:07:41.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.392+0000 7f2c0a600640 1 -- 172.21.15.52:0/3706011103 reap_dead start 2024-03-12T17:07:41.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.392+0000 7f2c0b000640 1 -- 172.21.15.52:0/3706011103 shutdown_connections 2024-03-12T17:07:41.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.392+0000 7f2c0b000640 1 -- 172.21.15.52:0/3706011103 >> 172.21.15.52:0/3706011103 conn(0x7f2c04065df0 msgr2=0x7f2c04069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:41.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.392+0000 7f2c0b000640 1 -- 172.21.15.52:0/3706011103 shutdown_connections 2024-03-12T17:07:41.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.392+0000 7f2c0b000640 1 -- 172.21.15.52:0/3706011103 wait complete. 2024-03-12T17:07:41.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.393+0000 7f2c0b000640 1 Processor -- start 2024-03-12T17:07:41.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.394+0000 7f2c0b000640 1 -- start start 2024-03-12T17:07:41.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.394+0000 7f2c0b000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2bec004d00 0x7f2c04150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:41.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.394+0000 7f2c0b000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c04152a30 0x7f2c04152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:41.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.394+0000 7f2c0b000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c0406c080 0x7f2c0406c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:41.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.394+0000 7f2c0b000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0406ca40 con 0x7f2c04152a30 2024-03-12T17:07:41.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.394+0000 7f2c0b000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c041522c0 con 0x7f2c0406c080 2024-03-12T17:07:41.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.394+0000 7f2c0b000640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c04151520 con 0x7f2bec004d00 2024-03-12T17:07:41.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.395+0000 7f2c09200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c04152a30 0x7f2c04152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:41.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.395+0000 7f2c09200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c04152a30 0x7f2c04152eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51788/0 (socket says 172.21.15.52:51788) 2024-03-12T17:07:41.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.395+0000 7f2c09200640 1 -- 172.21.15.52:0/443682237 learned_addr learned my addr 172.21.15.52:0/443682237 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:41.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.395+0000 7f2c09c00640 1 --2- 172.21.15.52:0/443682237 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2bec004d00 0x7f2c04150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:41.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.395+0000 7f2c0a600640 1 --2- 172.21.15.52:0/443682237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c0406c080 0x7f2c0406c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:41.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.395+0000 7f2c09200640 1 -- 172.21.15.52:0/443682237 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2bec004d00 msgr2=0x7f2c04150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:41.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.395+0000 7f2c09200640 1 --2- 172.21.15.52:0/443682237 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2bec004d00 0x7f2c04150fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:41.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.395+0000 7f2c09200640 1 -- 172.21.15.52:0/443682237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c0406c080 msgr2=0x7f2c0406c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:41.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.395+0000 7f2c09200640 1 --2- 172.21.15.52:0/443682237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c0406c080 0x7f2c0406c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:41.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.395+0000 7f2c09200640 1 -- 172.21.15.52:0/443682237 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c041517c0 con 0x7f2c04152a30 2024-03-12T17:07:41.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.395+0000 7f2c09200640 1 --2- 172.21.15.52:0/443682237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c04152a30 0x7f2c04152eb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00109d0 tx=0x7f2bf0010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:41.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.396+0000 7f2c01600640 1 -- 172.21.15.52:0/443682237 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf000ec20 con 0x7f2c04152a30 2024-03-12T17:07:41.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.396+0000 7f2c01600640 1 -- 172.21.15.52:0/443682237 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf000ed80 con 0x7f2c04152a30 2024-03-12T17:07:41.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.396+0000 7f2c01600640 1 -- 172.21.15.52:0/443682237 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf001c620 con 0x7f2c04152a30 2024-03-12T17:07:41.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.396+0000 7f2c0b000640 1 -- 172.21.15.52:0/443682237 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c04151aa0 con 0x7f2c04152a30 2024-03-12T17:07:41.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.396+0000 7f2c0b000640 1 -- 172.21.15.52:0/443682237 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c04152510 con 0x7f2c04152a30 2024-03-12T17:07:41.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.397+0000 7f2c0b000640 1 -- 172.21.15.52:0/443682237 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c040b12c0 con 0x7f2c04152a30 2024-03-12T17:07:41.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.398+0000 7f2c01600640 1 -- 172.21.15.52:0/443682237 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2bf001c780 con 0x7f2c04152a30 2024-03-12T17:07:41.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.399+0000 7f2c01600640 1 --2- 172.21.15.52:0/443682237 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2be406ac30 0x7f2be406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:41.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.399+0000 7f2c01600640 1 -- 172.21.15.52:0/443682237 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2bf0089e50 con 0x7f2c04152a30 2024-03-12T17:07:41.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.399+0000 7f2c09c00640 1 --2- 172.21.15.52:0/443682237 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2be406ac30 0x7f2be406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:41.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.400+0000 7f2c09c00640 1 --2- 172.21.15.52:0/443682237 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2be406ac30 0x7f2be406d0f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2c04073650 tx=0x7f2bf8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:41.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.403+0000 7f2c01600640 1 -- 172.21.15.52:0/443682237 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f2bf0054c80 con 0x7f2c04152a30 2024-03-12T17:07:41.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.599+0000 7f2c0b000640 1 -- 172.21.15.52:0/443682237 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2c0404e030 con 0x7f2c04152a30 2024-03-12T17:07:41.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.600+0000 7f2c01600640 1 -- 172.21.15.52:0/443682237 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2bf00582a0 con 0x7f2c04152a30 2024-03-12T17:07:41.593 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-12T17:07:41.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.602+0000 7f2c0b000640 1 -- 172.21.15.52:0/443682237 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2be406ac30 msgr2=0x7f2be406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:41.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.602+0000 7f2c0b000640 1 --2- 172.21.15.52:0/443682237 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2be406ac30 0x7f2be406d0f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2c04073650 tx=0x7f2bf8008040 comp rx=0 tx=0).stop 2024-03-12T17:07:41.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.602+0000 7f2c0b000640 1 -- 172.21.15.52:0/443682237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c04152a30 msgr2=0x7f2c04152eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:41.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.602+0000 7f2c0b000640 1 --2- 172.21.15.52:0/443682237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c04152a30 0x7f2c04152eb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00109d0 tx=0x7f2bf0010e90 comp rx=0 tx=0).stop 2024-03-12T17:07:41.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.602+0000 7f2c0b000640 1 -- 172.21.15.52:0/443682237 shutdown_connections 2024-03-12T17:07:41.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.602+0000 7f2c0b000640 1 --2- 172.21.15.52:0/443682237 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2be406ac30 0x7f2be406d0f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:41.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.602+0000 7f2c0b000640 1 --2- 172.21.15.52:0/443682237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c0406c080 0x7f2c0406c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:41.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.602+0000 7f2c0b000640 1 --2- 172.21.15.52:0/443682237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c04152a30 0x7f2c04152eb0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:41.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.602+0000 7f2c0b000640 1 --2- 172.21.15.52:0/443682237 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2bec004d00 0x7f2c04150fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:41.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.602+0000 7f2c0b000640 1 -- 172.21.15.52:0/443682237 >> 172.21.15.52:0/443682237 conn(0x7f2c04065df0 msgr2=0x7f2c0413b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:41.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.602+0000 7f2c0b000640 1 -- 172.21.15.52:0/443682237 shutdown_connections 2024-03-12T17:07:41.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.602+0000 7f2c0b000640 1 -- 172.21.15.52:0/443682237 wait complete. 2024-03-12T17:07:41.614 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-12T17:07:41.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.843+0000 7fa5e2800640 1 Processor -- start 2024-03-12T17:07:41.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.844+0000 7fa5e2800640 1 -- start start 2024-03-12T17:07:41.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.845+0000 7fa5e2800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa5dc159830 0x7fa5dc0cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:41.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.845+0000 7fa5e2800640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa5dc0ce930 0x7fa5dc0cedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:41.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.846+0000 7fa5e2800640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5dc159c10 con 0x7fa5dc0ca7b0 2024-03-12T17:07:41.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.846+0000 7fa5e2800640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5dc06de50 con 0x7fa5dc0ce930 2024-03-12T17:07:41.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.846+0000 7fa5e2800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5dc0cf300 con 0x7fa5dc159830 2024-03-12T17:07:41.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.846+0000 7fa5d3e00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fa5dc0ca7b0 0x7fa5dc0cdcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:44862/0 (socket says 172.21.15.52:44862) 2024-03-12T17:07:41.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.846+0000 7fa5d3e00640 1 -- 172.21.15.52:0/3560833921 learned_addr learned my addr 172.21.15.52:0/3560833921 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:41.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.846+0000 7fa5dbe00640 1 --2- 172.21.15.52:0/3560833921 >> v2:172.21.15.52:3300/0 conn(0x7fa5dc0ce930 0x7fa5dc0cedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:41.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.846+0000 7fa5db400640 1 --2- 172.21.15.52:0/3560833921 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa5dc159830 0x7fa5dc0cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:41.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.847+0000 7fa5dbe00640 1 -- 172.21.15.52:0/3560833921 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa5dc159830 msgr2=0x7fa5dc0cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:41.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.847+0000 7fa5dbe00640 1 --2- 172.21.15.52:0/3560833921 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa5dc159830 0x7fa5dc0cc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:41.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.847+0000 7fa5dbe00640 1 -- 172.21.15.52:0/3560833921 >> v1:172.21.15.52:6789/0 conn(0x7fa5dc0ca7b0 legacy=0x7fa5dc0cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:41.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.847+0000 7fa5dbe00640 1 -- 172.21.15.52:0/3560833921 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5dc0cd840 con 0x7fa5dc0ce930 2024-03-12T17:07:41.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.848+0000 7fa5dbe00640 1 --2- 172.21.15.52:0/3560833921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5dc0ce930 0x7fa5dc0cedc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4009d90 tx=0x7fa5d402bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c4c22f684184ce server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:41.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.848+0000 7fa5daa00640 1 -- 172.21.15.52:0/3560833921 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d402cce0 con 0x7fa5dc0ce930 2024-03-12T17:07:41.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.848+0000 7fa5daa00640 1 -- 172.21.15.52:0/3560833921 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d402ce40 con 0x7fa5dc0ce930 2024-03-12T17:07:41.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.848+0000 7fa5daa00640 1 -- 172.21.15.52:0/3560833921 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d4034660 con 0x7fa5dc0ce930 2024-03-12T17:07:41.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.849+0000 7fa5e2800640 1 -- 172.21.15.52:0/3560833921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5dc0ce930 msgr2=0x7fa5dc0cedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:41.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.849+0000 7fa5e2800640 1 --2- 172.21.15.52:0/3560833921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5dc0ce930 0x7fa5dc0cedc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4009d90 tx=0x7fa5d402bf30 comp rx=0 tx=0).stop 2024-03-12T17:07:41.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.849+0000 7fa5e2800640 1 -- 172.21.15.52:0/3560833921 shutdown_connections 2024-03-12T17:07:41.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.849+0000 7fa5e2800640 1 --2- 172.21.15.52:0/3560833921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5dc0ce930 0x7fa5dc0cedc0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:41.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.849+0000 7fa5e2800640 1 --2- 172.21.15.52:0/3560833921 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa5dc159830 0x7fa5dc0cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:41.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.849+0000 7fa5e2800640 1 -- 172.21.15.52:0/3560833921 >> 172.21.15.52:0/3560833921 conn(0x7fa5dc06a0b0 msgr2=0x7fa5dc06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:41.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.849+0000 7fa5e2800640 1 -- 172.21.15.52:0/3560833921 shutdown_connections 2024-03-12T17:07:41.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.849+0000 7fa5e2800640 1 -- 172.21.15.52:0/3560833921 wait complete. 2024-03-12T17:07:41.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.851+0000 7fa5e2800640 1 Processor -- start 2024-03-12T17:07:41.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.851+0000 7fa5e2800640 1 -- start start 2024-03-12T17:07:41.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.852+0000 7fa5e2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5dc0ca7b0 0x7fa5dc142ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:41.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.852+0000 7fa5e2800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa5dc0ce930 0x7fa5dc0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:41.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.852+0000 7fa5e2800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5dc159830 0x7fa5dc0b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:41.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.852+0000 7fa5e2800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5dc159dc0 con 0x7fa5dc159830 2024-03-12T17:07:41.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.852+0000 7fa5e2800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5dc15a010 con 0x7fa5dc0ca7b0 2024-03-12T17:07:41.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.852+0000 7fa5e2800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5dc15a2f0 con 0x7fa5dc0ce930 2024-03-12T17:07:41.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.852+0000 7fa5dbe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5dc159830 0x7fa5dc0b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:41.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.852+0000 7fa5dbe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5dc159830 0x7fa5dc0b0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51800/0 (socket says 172.21.15.52:51800) 2024-03-12T17:07:41.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.852+0000 7fa5dbe00640 1 -- 172.21.15.52:0/2347080723 learned_addr learned my addr 172.21.15.52:0/2347080723 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:41.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.853+0000 7fa5d3e00640 1 --2- 172.21.15.52:0/2347080723 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa5dc0ce930 0x7fa5dc0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:41.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.853+0000 7fa5db400640 1 --2- 172.21.15.52:0/2347080723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5dc0ca7b0 0x7fa5dc142ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:41.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.853+0000 7fa5dbe00640 1 -- 172.21.15.52:0/2347080723 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa5dc0ce930 msgr2=0x7fa5dc0b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:41.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.853+0000 7fa5dbe00640 1 --2- 172.21.15.52:0/2347080723 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa5dc0ce930 0x7fa5dc0b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:41.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.853+0000 7fa5dbe00640 1 -- 172.21.15.52:0/2347080723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5dc0ca7b0 msgr2=0x7fa5dc142ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:41.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.853+0000 7fa5dbe00640 1 --2- 172.21.15.52:0/2347080723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5dc0ca7b0 0x7fa5dc142ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:41.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.853+0000 7fa5dbe00640 1 -- 172.21.15.52:0/2347080723 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d4009a70 con 0x7fa5dc159830 2024-03-12T17:07:41.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.853+0000 7fa5d3e00640 1 --2- 172.21.15.52:0/2347080723 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa5dc0ce930 0x7fa5dc0b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:07:41.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.853+0000 7fa5dbe00640 1 --2- 172.21.15.52:0/2347080723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5dc159830 0x7fa5dc0b0a70 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4009eb0 tx=0x7fa5d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:41.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.854+0000 7fa5d3400640 1 -- 172.21.15.52:0/2347080723 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d403e610 con 0x7fa5dc159830 2024-03-12T17:07:41.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.854+0000 7fa5d3400640 1 -- 172.21.15.52:0/2347080723 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d403e770 con 0x7fa5dc159830 2024-03-12T17:07:41.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.854+0000 7fa5d3400640 1 -- 172.21.15.52:0/2347080723 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d403eb00 con 0x7fa5dc159830 2024-03-12T17:07:41.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.854+0000 7fa5e2800640 1 -- 172.21.15.52:0/2347080723 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5dc15a5d0 con 0x7fa5dc159830 2024-03-12T17:07:41.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.854+0000 7fa5e2800640 1 -- 172.21.15.52:0/2347080723 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5dc143220 con 0x7fa5dc159830 2024-03-12T17:07:41.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.855+0000 7fa5e2800640 1 -- 172.21.15.52:0/2347080723 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5dc15aaf0 con 0x7fa5dc159830 2024-03-12T17:07:41.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.856+0000 7fa5d3400640 1 -- 172.21.15.52:0/2347080723 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa5d403b050 con 0x7fa5dc159830 2024-03-12T17:07:41.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.857+0000 7fa5d3400640 1 --2- 172.21.15.52:0/2347080723 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa5b806ac00 0x7fa5b806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:41.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.857+0000 7fa5d3400640 1 -- 172.21.15.52:0/2347080723 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa5d40ac0c0 con 0x7fa5dc159830 2024-03-12T17:07:41.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.858+0000 7fa5db400640 1 --2- 172.21.15.52:0/2347080723 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa5b806ac00 0x7fa5b806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:41.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.859+0000 7fa5db400640 1 --2- 172.21.15.52:0/2347080723 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa5b806ac00 0x7fa5b806d0c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc009a30 tx=0x7fa5cc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:41.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:41.862+0000 7fa5d3400640 1 -- 172.21.15.52:0/2347080723 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fa5dc15aaf0 con 0x7fa5dc159830 2024-03-12T17:07:42.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.060+0000 7fa5e2800640 1 -- 172.21.15.52:0/2347080723 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa5dc15aaf0 con 0x7fa5dc159830 2024-03-12T17:07:42.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.061+0000 7fa5d3400640 1 -- 172.21.15.52:0/2347080723 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa5d404e020 con 0x7fa5dc159830 2024-03-12T17:07:42.054 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-03-12T17:07:42.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.063+0000 7fa5e2800640 1 -- 172.21.15.52:0/2347080723 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa5b806ac00 msgr2=0x7fa5b806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:42.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.063+0000 7fa5e2800640 1 --2- 172.21.15.52:0/2347080723 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa5b806ac00 0x7fa5b806d0c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc009a30 tx=0x7fa5cc008040 comp rx=0 tx=0).stop 2024-03-12T17:07:42.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.063+0000 7fa5e2800640 1 -- 172.21.15.52:0/2347080723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5dc159830 msgr2=0x7fa5dc0b0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:42.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.063+0000 7fa5e2800640 1 --2- 172.21.15.52:0/2347080723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5dc159830 0x7fa5dc0b0a70 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4009eb0 tx=0x7fa5d4004a40 comp rx=0 tx=0).stop 2024-03-12T17:07:42.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.063+0000 7fa5e2800640 1 -- 172.21.15.52:0/2347080723 shutdown_connections 2024-03-12T17:07:42.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.063+0000 7fa5e2800640 1 --2- 172.21.15.52:0/2347080723 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa5b806ac00 0x7fa5b806d0c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:42.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.063+0000 7fa5e2800640 1 --2- 172.21.15.52:0/2347080723 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa5dc0ce930 0x7fa5dc0b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:42.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.063+0000 7fa5e2800640 1 --2- 172.21.15.52:0/2347080723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5dc159830 0x7fa5dc0b0a70 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:42.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.063+0000 7fa5e2800640 1 --2- 172.21.15.52:0/2347080723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5dc0ca7b0 0x7fa5dc142ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:42.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.063+0000 7fa5e2800640 1 -- 172.21.15.52:0/2347080723 >> 172.21.15.52:0/2347080723 conn(0x7fa5dc06a0b0 msgr2=0x7fa5dc152b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:42.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.064+0000 7fa5e2800640 1 -- 172.21.15.52:0/2347080723 shutdown_connections 2024-03-12T17:07:42.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.064+0000 7fa5e2800640 1 -- 172.21.15.52:0/2347080723 wait complete. 2024-03-12T17:07:42.075 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-12T17:07:42.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.285+0000 7f1c1f200640 1 Processor -- start 2024-03-12T17:07:42.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.285+0000 7f1c1f200640 1 -- start start 2024-03-12T17:07:42.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.287+0000 7f1c1f200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f1c180cdc00 0x7f1c180b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:42.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.287+0000 7f1c1f200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c180cae50 0x7f1c180cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:42.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.287+0000 7f1c1f200640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c181575b0 con 0x7f1c181571d0 2024-03-12T17:07:42.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.287+0000 7f1c1f200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c180cb820 con 0x7f1c180cdc00 2024-03-12T17:07:42.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.287+0000 7f1c1f200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c18159f20 con 0x7f1c180cae50 2024-03-12T17:07:42.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.287+0000 7f1c1d400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f1c180cdc00 0x7f1c180b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:42.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.287+0000 7f1c1d400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f1c180cdc00 0x7f1c180b35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51816/0 (socket says 172.21.15.52:51816) 2024-03-12T17:07:42.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.287+0000 7f1c1d400640 1 -- 172.21.15.52:0/242091394 learned_addr learned my addr 172.21.15.52:0/242091394 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:42.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.287+0000 7f1c1e800640 1 --2- 172.21.15.52:0/242091394 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c180cae50 0x7f1c180cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:42.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.288+0000 7f1c1d400640 1 -- 172.21.15.52:0/242091394 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c180cae50 msgr2=0x7f1c180cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:42.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.288+0000 7f1c1d400640 1 --2- 172.21.15.52:0/242091394 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c180cae50 0x7f1c180cb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:42.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.288+0000 7f1c1d400640 1 -- 172.21.15.52:0/242091394 >> v1:172.21.15.52:6789/0 conn(0x7f1c181571d0 legacy=0x7f1c1815aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:42.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.288+0000 7f1c1d400640 1 -- 172.21.15.52:0/242091394 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c1815a200 con 0x7f1c180cdc00 2024-03-12T17:07:42.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.288+0000 7f1c1e800640 1 --2- 172.21.15.52:0/242091394 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c180cae50 0x7f1c180cb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:42.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.289+0000 7f1c1d400640 1 --2- 172.21.15.52:0/242091394 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c180cdc00 0x7f1c180b35d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1c0800bb10 tx=0x7f1c0802e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b361dc8ef0d979 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:42.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.289+0000 7f1c1ca00640 1 -- 172.21.15.52:0/242091394 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c08036440 con 0x7f1c180cdc00 2024-03-12T17:07:42.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.289+0000 7f1c1ca00640 1 -- 172.21.15.52:0/242091394 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c080365a0 con 0x7f1c180cdc00 2024-03-12T17:07:42.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.289+0000 7f1c1ca00640 1 -- 172.21.15.52:0/242091394 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c08036930 con 0x7f1c180cdc00 2024-03-12T17:07:42.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.290+0000 7f1c1f200640 1 -- 172.21.15.52:0/242091394 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c180cdc00 msgr2=0x7f1c180b35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:42.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.290+0000 7f1c1f200640 1 --2- 172.21.15.52:0/242091394 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c180cdc00 0x7f1c180b35d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1c0800bb10 tx=0x7f1c0802e160 comp rx=0 tx=0).stop 2024-03-12T17:07:42.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.290+0000 7f1c1f200640 1 -- 172.21.15.52:0/242091394 shutdown_connections 2024-03-12T17:07:42.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.290+0000 7f1c1f200640 1 --2- 172.21.15.52:0/242091394 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c180cdc00 0x7f1c180b35d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:42.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.290+0000 7f1c1f200640 1 --2- 172.21.15.52:0/242091394 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c180cae50 0x7f1c180cb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:42.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.290+0000 7f1c1f200640 1 -- 172.21.15.52:0/242091394 >> 172.21.15.52:0/242091394 conn(0x7f1c18065df0 msgr2=0x7f1c18069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:42.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.290+0000 7f1c1f200640 1 -- 172.21.15.52:0/242091394 shutdown_connections 2024-03-12T17:07:42.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.290+0000 7f1c1f200640 1 -- 172.21.15.52:0/242091394 wait complete. 2024-03-12T17:07:42.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.291+0000 7f1c1f200640 1 Processor -- start 2024-03-12T17:07:42.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.292+0000 7f1c1f200640 1 -- start start 2024-03-12T17:07:42.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.292+0000 7f1c1f200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c180cae50 0x7f1c180c33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:42.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.292+0000 7f1c1f200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c180cdc00 0x7f1c180c4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:42.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.292+0000 7f1c1f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c181571d0 0x7f1c181f1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:42.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.292+0000 7f1c1f200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c18142710 con 0x7f1c180cdc00 2024-03-12T17:07:42.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.292+0000 7f1c1f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c180c3d30 con 0x7f1c181571d0 2024-03-12T17:07:42.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.292+0000 7f1c1f200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c180c4010 con 0x7f1c180cae50 2024-03-12T17:07:42.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.293+0000 7f1c1d400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c180cdc00 0x7f1c180c4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:42.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.293+0000 7f1c1d400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c180cdc00 0x7f1c180c4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51822/0 (socket says 172.21.15.52:51822) 2024-03-12T17:07:42.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.293+0000 7f1c1d400640 1 -- 172.21.15.52:0/398444600 learned_addr learned my addr 172.21.15.52:0/398444600 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:42.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.293+0000 7f1c1de00640 1 --2- 172.21.15.52:0/398444600 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c180cae50 0x7f1c180c33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:42.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.293+0000 7f1c1d400640 1 -- 172.21.15.52:0/398444600 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c180cae50 msgr2=0x7f1c180c33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:42.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.293+0000 7f1c1d400640 1 --2- 172.21.15.52:0/398444600 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c180cae50 0x7f1c180c33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:42.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.293+0000 7f1c1d400640 1 -- 172.21.15.52:0/398444600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c181571d0 msgr2=0x7f1c181f1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:42.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.293+0000 7f1c1d400640 1 --2- 172.21.15.52:0/398444600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c181571d0 0x7f1c181f1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:42.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.293+0000 7f1c1d400640 1 -- 172.21.15.52:0/398444600 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c0800b770 con 0x7f1c180cdc00 2024-03-12T17:07:42.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.293+0000 7f1c1d400640 1 --2- 172.21.15.52:0/398444600 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c180cdc00 0x7f1c180c4dd0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1c0802e1a0 tx=0x7f1c080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:42.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.294+0000 7f1c12000640 1 -- 172.21.15.52:0/398444600 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c08034b80 con 0x7f1c180cdc00 2024-03-12T17:07:42.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.294+0000 7f1c12000640 1 -- 172.21.15.52:0/398444600 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c08034ce0 con 0x7f1c180cdc00 2024-03-12T17:07:42.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.294+0000 7f1c12000640 1 -- 172.21.15.52:0/398444600 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c080416a0 con 0x7f1c180cdc00 2024-03-12T17:07:42.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.294+0000 7f1c1f200640 1 -- 172.21.15.52:0/398444600 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c180c42f0 con 0x7f1c180cdc00 2024-03-12T17:07:42.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.294+0000 7f1c1f200640 1 -- 172.21.15.52:0/398444600 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c180c48b0 con 0x7f1c180cdc00 2024-03-12T17:07:42.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.295+0000 7f1c1f200640 1 -- 172.21.15.52:0/398444600 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c180c4aa0 con 0x7f1c180cdc00 2024-03-12T17:07:42.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.296+0000 7f1c12000640 1 -- 172.21.15.52:0/398444600 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1c08049360 con 0x7f1c180cdc00 2024-03-12T17:07:42.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.297+0000 7f1c12000640 1 --2- 172.21.15.52:0/398444600 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1bfc06ac00 0x7f1bfc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:42.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.297+0000 7f1c12000640 1 -- 172.21.15.52:0/398444600 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1c080abd50 con 0x7f1c180cdc00 2024-03-12T17:07:42.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.297+0000 7f1c1de00640 1 --2- 172.21.15.52:0/398444600 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1bfc06ac00 0x7f1bfc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:42.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.299+0000 7f1c1de00640 1 --2- 172.21.15.52:0/398444600 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1bfc06ac00 0x7f1bfc06d0c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1c180b0270 tx=0x7f1c0c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:42.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.301+0000 7f1c12000640 1 -- 172.21.15.52:0/398444600 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f1c08076ad0 con 0x7f1c180cdc00 2024-03-12T17:07:42.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.498+0000 7f1c1f200640 1 -- 172.21.15.52:0/398444600 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1c1804e030 con 0x7f1c180cdc00 2024-03-12T17:07:42.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.499+0000 7f1c12000640 1 -- 172.21.15.52:0/398444600 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1c080460a0 con 0x7f1c180cdc00 2024-03-12T17:07:42.493 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-12T17:07:42.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.502+0000 7f1c1f200640 1 -- 172.21.15.52:0/398444600 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1bfc06ac00 msgr2=0x7f1bfc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:42.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.502+0000 7f1c1f200640 1 --2- 172.21.15.52:0/398444600 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1bfc06ac00 0x7f1bfc06d0c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1c180b0270 tx=0x7f1c0c009040 comp rx=0 tx=0).stop 2024-03-12T17:07:42.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.502+0000 7f1c1f200640 1 -- 172.21.15.52:0/398444600 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c180cdc00 msgr2=0x7f1c180c4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:42.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.502+0000 7f1c1f200640 1 --2- 172.21.15.52:0/398444600 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c180cdc00 0x7f1c180c4dd0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1c0802e1a0 tx=0x7f1c080046c0 comp rx=0 tx=0).stop 2024-03-12T17:07:42.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.502+0000 7f1c1f200640 1 -- 172.21.15.52:0/398444600 shutdown_connections 2024-03-12T17:07:42.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.502+0000 7f1c1f200640 1 --2- 172.21.15.52:0/398444600 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1bfc06ac00 0x7f1bfc06d0c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:42.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.502+0000 7f1c1f200640 1 --2- 172.21.15.52:0/398444600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c181571d0 0x7f1c181f1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:42.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.502+0000 7f1c1f200640 1 --2- 172.21.15.52:0/398444600 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c180cdc00 0x7f1c180c4dd0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:42.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.502+0000 7f1c1f200640 1 --2- 172.21.15.52:0/398444600 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c180cae50 0x7f1c180c33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:42.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.502+0000 7f1c1f200640 1 -- 172.21.15.52:0/398444600 >> 172.21.15.52:0/398444600 conn(0x7f1c18065df0 msgr2=0x7f1c1815aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:42.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.502+0000 7f1c1f200640 1 -- 172.21.15.52:0/398444600 shutdown_connections 2024-03-12T17:07:42.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.502+0000 7f1c1f200640 1 -- 172.21.15.52:0/398444600 wait complete. 2024-03-12T17:07:42.515 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-12T17:07:42.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.740+0000 7f571f000640 1 Processor -- start 2024-03-12T17:07:42.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.740+0000 7f571f000640 1 -- start start 2024-03-12T17:07:42.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.741+0000 7f571f000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f57180b80a0 0x7f57180cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:42.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.741+0000 7f571f000640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f57180cd9b0 0x7f57180d10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:42.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.741+0000 7f571f000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5718160b90 con 0x7f571815c890 2024-03-12T17:07:42.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.741+0000 7f571f000640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57180d0ca0 con 0x7f57180cd9b0 2024-03-12T17:07:42.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.741+0000 7f571f000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57180cde60 con 0x7f57180b80a0 2024-03-12T17:07:42.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.742+0000 7f571e600640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f57180cd9b0 0x7f57180d10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:42.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.742+0000 7f571e600640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f57180cd9b0 0x7f57180d10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51828/0 (socket says 172.21.15.52:51828) 2024-03-12T17:07:42.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.742+0000 7f571e600640 1 -- 172.21.15.52:0/2766088285 learned_addr learned my addr 172.21.15.52:0/2766088285 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:42.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.742+0000 7f571d200640 1 --2- 172.21.15.52:0/2766088285 >> v2:172.21.15.102:3300/0 conn(0x7f57180b80a0 0x7f57180cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:42.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.742+0000 7f571e600640 1 -- 172.21.15.52:0/2766088285 >> v2:172.21.15.102:3300/0 conn(0x7f57180b80a0 msgr2=0x7f57180cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:42.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.742+0000 7f571e600640 1 --2- 172.21.15.52:0/2766088285 >> v2:172.21.15.102:3300/0 conn(0x7f57180b80a0 0x7f57180cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:42.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.742+0000 7f571e600640 1 -- 172.21.15.52:0/2766088285 >> v1:172.21.15.102:6789/0 conn(0x7f571815c890 legacy=0x7f5718160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:42.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.742+0000 7f571e600640 1 -- 172.21.15.52:0/2766088285 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5718161390 con 0x7f57180cd9b0 2024-03-12T17:07:42.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.743+0000 7f571e600640 1 --2- 172.21.15.52:0/2766088285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57180cd9b0 0x7f57180d10c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f571808aea0 tx=0x7f571002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b021f956cc6c5a02 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:42.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.744+0000 7f5717e00640 1 -- 172.21.15.52:0/2766088285 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571002c9f0 con 0x7f57180cd9b0 2024-03-12T17:07:42.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.744+0000 7f5717e00640 1 -- 172.21.15.52:0/2766088285 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f571002cb50 con 0x7f57180cd9b0 2024-03-12T17:07:42.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.744+0000 7f5717e00640 1 -- 172.21.15.52:0/2766088285 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57100343f0 con 0x7f57180cd9b0 2024-03-12T17:07:42.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.744+0000 7f571f000640 1 -- 172.21.15.52:0/2766088285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57180cd9b0 msgr2=0x7f57180d10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:42.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.744+0000 7f571f000640 1 --2- 172.21.15.52:0/2766088285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57180cd9b0 0x7f57180d10c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f571808aea0 tx=0x7f571002be50 comp rx=0 tx=0).stop 2024-03-12T17:07:42.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.744+0000 7f571f000640 1 -- 172.21.15.52:0/2766088285 shutdown_connections 2024-03-12T17:07:42.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.744+0000 7f571f000640 1 --2- 172.21.15.52:0/2766088285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57180cd9b0 0x7f57180d10c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:42.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.744+0000 7f571f000640 1 --2- 172.21.15.52:0/2766088285 >> v2:172.21.15.102:3300/0 conn(0x7f57180b80a0 0x7f57180cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:42.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.744+0000 7f571f000640 1 -- 172.21.15.52:0/2766088285 >> 172.21.15.52:0/2766088285 conn(0x7f571806ae90 msgr2=0x7f571806b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:42.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.744+0000 7f571f000640 1 -- 172.21.15.52:0/2766088285 shutdown_connections 2024-03-12T17:07:42.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.745+0000 7f571f000640 1 -- 172.21.15.52:0/2766088285 wait complete. 2024-03-12T17:07:42.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.746+0000 7f571f000640 1 Processor -- start 2024-03-12T17:07:42.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.746+0000 7f571f000640 1 -- start start 2024-03-12T17:07:42.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.747+0000 7f571f000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f57180b80a0 0x7f57180cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:42.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.747+0000 7f571f000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57180cd9b0 0x7f57180b4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:42.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.747+0000 7f571f000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f571815c890 0x7f57180b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:42.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.747+0000 7f571f000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57180cbfe0 con 0x7f571815c890 2024-03-12T17:07:42.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.747+0000 7f571f000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57180b2f30 con 0x7f57180cd9b0 2024-03-12T17:07:42.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.747+0000 7f571f000640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57180b3210 con 0x7f57180b80a0 2024-03-12T17:07:42.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.748+0000 7f571e600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f571815c890 0x7f57180b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:42.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.748+0000 7f571e600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f571815c890 0x7f57180b49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51834/0 (socket says 172.21.15.52:51834) 2024-03-12T17:07:42.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.748+0000 7f571e600640 1 -- 172.21.15.52:0/3090901746 learned_addr learned my addr 172.21.15.52:0/3090901746 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:42.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.748+0000 7f571d200640 1 --2- 172.21.15.52:0/3090901746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57180cd9b0 0x7f57180b4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:42.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.748+0000 7f571dc00640 1 --2- 172.21.15.52:0/3090901746 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f57180b80a0 0x7f57180cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:42.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.748+0000 7f571e600640 1 -- 172.21.15.52:0/3090901746 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f57180b80a0 msgr2=0x7f57180cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:42.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.748+0000 7f571e600640 1 --2- 172.21.15.52:0/3090901746 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f57180b80a0 0x7f57180cbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:42.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.748+0000 7f571e600640 1 -- 172.21.15.52:0/3090901746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57180cd9b0 msgr2=0x7f57180b4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:42.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.748+0000 7f571e600640 1 --2- 172.21.15.52:0/3090901746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57180cd9b0 0x7f57180b4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:42.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.748+0000 7f571e600640 1 -- 172.21.15.52:0/3090901746 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5710009b30 con 0x7f571815c890 2024-03-12T17:07:42.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.748+0000 7f571e600640 1 --2- 172.21.15.52:0/3090901746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f571815c890 0x7f57180b49c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5710036040 tx=0x7f5710004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:42.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.749+0000 7f5715600640 1 -- 172.21.15.52:0/3090901746 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5710009510 con 0x7f571815c890 2024-03-12T17:07:42.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.749+0000 7f5715600640 1 -- 172.21.15.52:0/3090901746 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f571003e450 con 0x7f571815c890 2024-03-12T17:07:42.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.749+0000 7f5715600640 1 -- 172.21.15.52:0/3090901746 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571003e7e0 con 0x7f571815c890 2024-03-12T17:07:42.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.749+0000 7f571f000640 1 -- 172.21.15.52:0/3090901746 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57180b34f0 con 0x7f571815c890 2024-03-12T17:07:42.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.749+0000 7f571f000640 1 -- 172.21.15.52:0/3090901746 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57180b3ab0 con 0x7f571815c890 2024-03-12T17:07:42.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.750+0000 7f571f000640 1 -- 172.21.15.52:0/3090901746 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f571806c420 con 0x7f571815c890 2024-03-12T17:07:42.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.751+0000 7f5715600640 1 -- 172.21.15.52:0/3090901746 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f571003e940 con 0x7f571815c890 2024-03-12T17:07:42.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.752+0000 7f5715600640 1 --2- 172.21.15.52:0/3090901746 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f56f806ac00 0x7f56f806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:42.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.752+0000 7f5715600640 1 -- 172.21.15.52:0/3090901746 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f57100ac1f0 con 0x7f571815c890 2024-03-12T17:07:42.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.752+0000 7f571dc00640 1 --2- 172.21.15.52:0/3090901746 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f56f806ac00 0x7f56f806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:42.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.753+0000 7f571dc00640 1 --2- 172.21.15.52:0/3090901746 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f56f806ac00 0x7f56f806d0c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f570c006fd0 tx=0x7f570c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:42.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.756+0000 7f5715600640 1 -- 172.21.15.52:0/3090901746 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f5710077000 con 0x7f571815c890 2024-03-12T17:07:42.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.960+0000 7f571f000640 1 -- 172.21.15.52:0/3090901746 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5718160b90 con 0x7f571815c890 2024-03-12T17:07:42.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.961+0000 7f5715600640 1 -- 172.21.15.52:0/3090901746 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5718160b90 con 0x7f571815c890 2024-03-12T17:07:42.955 INFO:teuthology.orchestra.run.smithi052.stderr:listed 0 entries 2024-03-12T17:07:42.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.963+0000 7f571f000640 1 -- 172.21.15.52:0/3090901746 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f56f806ac00 msgr2=0x7f56f806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:42.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.963+0000 7f571f000640 1 --2- 172.21.15.52:0/3090901746 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f56f806ac00 0x7f56f806d0c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f570c006fd0 tx=0x7f570c008040 comp rx=0 tx=0).stop 2024-03-12T17:07:42.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.963+0000 7f571f000640 1 -- 172.21.15.52:0/3090901746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f571815c890 msgr2=0x7f57180b49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:42.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.963+0000 7f571f000640 1 --2- 172.21.15.52:0/3090901746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f571815c890 0x7f57180b49c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5710036040 tx=0x7f5710004a40 comp rx=0 tx=0).stop 2024-03-12T17:07:42.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.964+0000 7f571f000640 1 -- 172.21.15.52:0/3090901746 shutdown_connections 2024-03-12T17:07:42.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.964+0000 7f571f000640 1 --2- 172.21.15.52:0/3090901746 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f56f806ac00 0x7f56f806d0c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:42.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.964+0000 7f571f000640 1 --2- 172.21.15.52:0/3090901746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f571815c890 0x7f57180b49c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:42.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.964+0000 7f571f000640 1 --2- 172.21.15.52:0/3090901746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57180cd9b0 0x7f57180b4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:42.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.964+0000 7f571f000640 1 --2- 172.21.15.52:0/3090901746 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f57180b80a0 0x7f57180cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:42.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.964+0000 7f571f000640 1 -- 172.21.15.52:0/3090901746 >> 172.21.15.52:0/3090901746 conn(0x7f571806ae90 msgr2=0x7f57180ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:42.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.964+0000 7f571f000640 1 -- 172.21.15.52:0/3090901746 shutdown_connections 2024-03-12T17:07:42.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:42.964+0000 7f571f000640 1 -- 172.21.15.52:0/3090901746 wait complete. 2024-03-12T17:07:42.977 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-03-12T17:07:42.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:42.977 DEBUG:teuthology.orchestra.run.smithi052:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-03-12T17:07:43.077 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi052.front.sepia.ceph.com 2024-03-12T17:07:43.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:43.078 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-12T17:07:43.133 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T17:07:43.133 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-12T17:07:43.191 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T17:07:43.192 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-12T17:07:43.240 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T17:07:43.240 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-12T17:07:43.336 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:07:43.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.573+0000 7f2de5c00640 1 Processor -- start 2024-03-12T17:07:43.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.573+0000 7f2de5c00640 1 -- start start 2024-03-12T17:07:43.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.574+0000 7f2de5c00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2de0143460 0x7f2de015aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:43.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.574+0000 7f2de5c00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2de00cde20 0x7f2de0070020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:43.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.575+0000 7f2de5c00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2de00cc6b0 con 0x7f2de0143930 2024-03-12T17:07:43.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.575+0000 7f2de5c00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2de00cc950 con 0x7f2de0143460 2024-03-12T17:07:43.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.575+0000 7f2de5c00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de00ccc30 con 0x7f2de00cde20 2024-03-12T17:07:43.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.575+0000 7f2ddfe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2de0143460 0x7f2de015aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:43.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.575+0000 7f2ddfe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2de0143460 0x7f2de015aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51836/0 (socket says 172.21.15.52:51836) 2024-03-12T17:07:43.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.575+0000 7f2ddfe00640 1 -- 172.21.15.52:0/3531981478 learned_addr learned my addr 172.21.15.52:0/3531981478 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:43.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.575+0000 7f2de5200640 1 --2- 172.21.15.52:0/3531981478 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2de00cde20 0x7f2de0070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:43.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.575+0000 7f2ddfe00640 1 -- 172.21.15.52:0/3531981478 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2de00cde20 msgr2=0x7f2de0070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:43.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.575+0000 7f2ddfe00640 1 --2- 172.21.15.52:0/3531981478 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2de00cde20 0x7f2de0070020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:43.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.575+0000 7f2ddfe00640 1 -- 172.21.15.52:0/3531981478 >> v1:172.21.15.102:6789/0 conn(0x7f2de0143930 legacy=0x7f2de00cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:43.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.575+0000 7f2ddfe00640 1 -- 172.21.15.52:0/3531981478 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de00ccf10 con 0x7f2de0143460 2024-03-12T17:07:43.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.575+0000 7f2de5200640 1 --2- 172.21.15.52:0/3531981478 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2de00cde20 0x7f2de0070020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:43.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.577+0000 7f2ddfe00640 1 --2- 172.21.15.52:0/3531981478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2de0143460 0x7f2de015aba0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0009870 tx=0x7f2dd002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d1ee9bcfcb345e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:43.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.577+0000 7f2ddea00640 1 -- 172.21.15.52:0/3531981478 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0030840 con 0x7f2de0143460 2024-03-12T17:07:43.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.577+0000 7f2ddea00640 1 -- 172.21.15.52:0/3531981478 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd00309a0 con 0x7f2de0143460 2024-03-12T17:07:43.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.577+0000 7f2ddea00640 1 -- 172.21.15.52:0/3531981478 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0030d30 con 0x7f2de0143460 2024-03-12T17:07:43.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.577+0000 7f2de5c00640 1 -- 172.21.15.52:0/3531981478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2de0143460 msgr2=0x7f2de015aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:43.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.577+0000 7f2de5c00640 1 --2- 172.21.15.52:0/3531981478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2de0143460 0x7f2de015aba0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0009870 tx=0x7f2dd002b980 comp rx=0 tx=0).stop 2024-03-12T17:07:43.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.578+0000 7f2de5c00640 1 -- 172.21.15.52:0/3531981478 shutdown_connections 2024-03-12T17:07:43.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.578+0000 7f2de5c00640 1 --2- 172.21.15.52:0/3531981478 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2de00cde20 0x7f2de0070020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:43.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.578+0000 7f2de5c00640 1 --2- 172.21.15.52:0/3531981478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2de0143460 0x7f2de015aba0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:43.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.578+0000 7f2de5c00640 1 -- 172.21.15.52:0/3531981478 >> 172.21.15.52:0/3531981478 conn(0x7f2de0065df0 msgr2=0x7f2de0069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:43.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.578+0000 7f2de5c00640 1 -- 172.21.15.52:0/3531981478 shutdown_connections 2024-03-12T17:07:43.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.578+0000 7f2de5c00640 1 -- 172.21.15.52:0/3531981478 wait complete. 2024-03-12T17:07:43.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.579+0000 7f2de5c00640 1 Processor -- start 2024-03-12T17:07:43.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.579+0000 7f2de5c00640 1 -- start start 2024-03-12T17:07:43.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.580+0000 7f2de5c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2de00cde20 0x7f2de01384b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:43.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.580+0000 7f2de5c00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2de0143460 0x7f2de0135af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:43.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.580+0000 7f2de5c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2de0143930 0x7f2de013f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:43.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.580+0000 7f2de5c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de0138d90 con 0x7f2de00cde20 2024-03-12T17:07:43.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.580+0000 7f2de5c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de0139070 con 0x7f2de0143930 2024-03-12T17:07:43.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.580+0000 7f2de5c00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de0139350 con 0x7f2de0143460 2024-03-12T17:07:43.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.581+0000 7f2ddfe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2de00cde20 0x7f2de01384b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:43.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.581+0000 7f2ddfe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2de00cde20 0x7f2de01384b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51852/0 (socket says 172.21.15.52:51852) 2024-03-12T17:07:43.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.581+0000 7f2ddfe00640 1 -- 172.21.15.52:0/2919334634 learned_addr learned my addr 172.21.15.52:0/2919334634 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:43.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.581+0000 7f2ddf400640 1 --2- 172.21.15.52:0/2919334634 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2de0143460 0x7f2de0135af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:43.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.581+0000 7f2ddfe00640 1 -- 172.21.15.52:0/2919334634 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2de0143460 msgr2=0x7f2de0135af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:43.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.581+0000 7f2de5200640 1 --2- 172.21.15.52:0/2919334634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2de0143930 0x7f2de013f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:43.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.581+0000 7f2ddfe00640 1 --2- 172.21.15.52:0/2919334634 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2de0143460 0x7f2de0135af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:43.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.581+0000 7f2ddfe00640 1 -- 172.21.15.52:0/2919334634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2de0143930 msgr2=0x7f2de013f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:43.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.581+0000 7f2ddfe00640 1 --2- 172.21.15.52:0/2919334634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2de0143930 0x7f2de013f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:43.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.581+0000 7f2ddfe00640 1 -- 172.21.15.52:0/2919334634 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd00094d0 con 0x7f2de00cde20 2024-03-12T17:07:43.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.581+0000 7f2ddfe00640 1 --2- 172.21.15.52:0/2919334634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2de00cde20 0x7f2de01384b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0009870 tx=0x7f2dd0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:43.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.582+0000 7f2dd7e00640 1 -- 172.21.15.52:0/2919334634 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd00307f0 con 0x7f2de00cde20 2024-03-12T17:07:43.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.582+0000 7f2dd7e00640 1 -- 172.21.15.52:0/2919334634 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd0043400 con 0x7f2de00cde20 2024-03-12T17:07:43.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.582+0000 7f2dd7e00640 1 -- 172.21.15.52:0/2919334634 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd00437b0 con 0x7f2de00cde20 2024-03-12T17:07:43.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.582+0000 7f2de5c00640 1 -- 172.21.15.52:0/2919334634 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2de013a7b0 con 0x7f2de00cde20 2024-03-12T17:07:43.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.582+0000 7f2de5c00640 1 -- 172.21.15.52:0/2919334634 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2de01389f0 con 0x7f2de00cde20 2024-03-12T17:07:43.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.583+0000 7f2de5c00640 1 -- 172.21.15.52:0/2919334634 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2de004e030 con 0x7f2de00cde20 2024-03-12T17:07:43.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.584+0000 7f2dd7e00640 1 -- 172.21.15.52:0/2919334634 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2dd0043910 con 0x7f2de00cde20 2024-03-12T17:07:43.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.585+0000 7f2dd7e00640 1 --2- 172.21.15.52:0/2919334634 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2dc006ac00 0x7f2dc006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:43.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.585+0000 7f2dd7e00640 1 -- 172.21.15.52:0/2919334634 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2dd003e070 con 0x7f2de00cde20 2024-03-12T17:07:43.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.586+0000 7f2ddf400640 1 --2- 172.21.15.52:0/2919334634 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2dc006ac00 0x7f2dc006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:43.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.587+0000 7f2ddf400640 1 --2- 172.21.15.52:0/2919334634 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2dc006ac00 0x7f2dc006d0c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2de0141790 tx=0x7f2dcc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:43.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.590+0000 7f2dd7e00640 1 -- 172.21.15.52:0/2919334634 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f2dd00777d0 con 0x7f2de00cde20 2024-03-12T17:07:43.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.784+0000 7f2de5c00640 1 -- 172.21.15.52:0/2919334634 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2de0144050 con 0x7f2de00cde20 2024-03-12T17:07:43.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.788+0000 7f2dd7e00640 1 -- 172.21.15.52:0/2919334634 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18207 (secure 0 0 0) 0x7f2dd007adf0 con 0x7f2de00cde20 2024-03-12T17:07:43.782 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:07:43.783 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":27,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:07:12.969614+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T17:06:19.218543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T17:06:23.525474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T17:06:24.911574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-12T17:06:28.017172+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.733031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.775178+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.591522+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.820838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.829252+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.582481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.18","mappings":[{"from":3,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:07:43.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.791+0000 7f2de5c00640 1 -- 172.21.15.52:0/2919334634 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2dc006ac00 msgr2=0x7f2dc006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:43.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.791+0000 7f2de5c00640 1 --2- 172.21.15.52:0/2919334634 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2dc006ac00 0x7f2dc006d0c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2de0141790 tx=0x7f2dcc008040 comp rx=0 tx=0).stop 2024-03-12T17:07:43.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.791+0000 7f2de5c00640 1 -- 172.21.15.52:0/2919334634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2de00cde20 msgr2=0x7f2de01384b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:43.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.791+0000 7f2de5c00640 1 --2- 172.21.15.52:0/2919334634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2de00cde20 0x7f2de01384b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0009870 tx=0x7f2dd0004730 comp rx=0 tx=0).stop 2024-03-12T17:07:43.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.791+0000 7f2de5c00640 1 -- 172.21.15.52:0/2919334634 shutdown_connections 2024-03-12T17:07:43.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.791+0000 7f2de5c00640 1 --2- 172.21.15.52:0/2919334634 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2dc006ac00 0x7f2dc006d0c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:43.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.791+0000 7f2de5c00640 1 --2- 172.21.15.52:0/2919334634 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2de0143460 0x7f2de0135af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:43.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.791+0000 7f2de5c00640 1 --2- 172.21.15.52:0/2919334634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2de0143930 0x7f2de013f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:43.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.791+0000 7f2de5c00640 1 --2- 172.21.15.52:0/2919334634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2de00cde20 0x7f2de01384b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:43.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.791+0000 7f2de5c00640 1 -- 172.21.15.52:0/2919334634 >> 172.21.15.52:0/2919334634 conn(0x7f2de0065df0 msgr2=0x7f2de015b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:43.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.791+0000 7f2de5c00640 1 -- 172.21.15.52:0/2919334634 shutdown_connections 2024-03-12T17:07:43.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:43.791+0000 7f2de5c00640 1 -- 172.21.15.52:0/2919334634 wait complete. 2024-03-12T17:07:43.805 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T17:06:19.218543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-12T17:06:23.525474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-12T17:06:24.911574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-12T17:06:28.017172+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-12T17:07:43.805 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T17:07:44.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.033+0000 7fc257200640 1 Processor -- start 2024-03-12T17:07:44.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.033+0000 7fc257200640 1 -- start start 2024-03-12T17:07:44.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.034+0000 7fc257200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc25815a6a0 0x7fc25806f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:44.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.035+0000 7fc257200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc25815aa80 0x7fc2581646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:44.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.035+0000 7fc257200640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc258070370 con 0x7fc2580afb70 2024-03-12T17:07:44.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.035+0000 7fc257200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2580ad740 con 0x7fc25815a6a0 2024-03-12T17:07:44.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.035+0000 7fc257200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc25806fdb0 con 0x7fc25815aa80 2024-03-12T17:07:44.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.035+0000 7fc255400640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fc2580afb70 0x7fc2580ce9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:44882/0 (socket says 172.21.15.52:44882) 2024-03-12T17:07:44.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.035+0000 7fc255400640 1 -- 172.21.15.52:0/2672635928 learned_addr learned my addr 172.21.15.52:0/2672635928 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:44.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.035+0000 7fc255e00640 1 --2- 172.21.15.52:0/2672635928 >> v2:172.21.15.102:3300/0 conn(0x7fc25815a6a0 0x7fc25806f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:44.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.035+0000 7fc256800640 1 --2- 172.21.15.52:0/2672635928 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc25815aa80 0x7fc2581646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:44.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.035+0000 7fc254a00640 1 -- 172.21.15.52:0/2672635928 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2418960807 0 0) 0x7fc258070370 con 0x7fc2580afb70 2024-03-12T17:07:44.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.036+0000 7fc254a00640 1 -- 172.21.15.52:0/2672635928 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc23c003520 con 0x7fc2580afb70 2024-03-12T17:07:44.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.036+0000 7fc254a00640 1 -- 172.21.15.52:0/2672635928 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2277665809 0 0) 0x7fc23c003520 con 0x7fc2580afb70 2024-03-12T17:07:44.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.036+0000 7fc255e00640 1 -- 172.21.15.52:0/2672635928 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc25815aa80 msgr2=0x7fc2581646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:44.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.036+0000 7fc255e00640 1 --2- 172.21.15.52:0/2672635928 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc25815aa80 0x7fc2581646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:44.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.036+0000 7fc255e00640 1 -- 172.21.15.52:0/2672635928 >> v1:172.21.15.52:6789/0 conn(0x7fc2580afb70 legacy=0x7fc2580ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:44.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.036+0000 7fc255e00640 1 -- 172.21.15.52:0/2672635928 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc258070090 con 0x7fc25815a6a0 2024-03-12T17:07:44.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.038+0000 7fc255e00640 1 --2- 172.21.15.52:0/2672635928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc25815a6a0 0x7fc25806f4b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc250009d90 tx=0x7fc25002bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ef5fde7140aa5e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:44.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.038+0000 7fc254a00640 1 -- 172.21.15.52:0/2672635928 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc240001670 con 0x7fc25815a6a0 2024-03-12T17:07:44.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.038+0000 7fc254a00640 1 -- 172.21.15.52:0/2672635928 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25002c810 con 0x7fc25815a6a0 2024-03-12T17:07:44.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.038+0000 7fc254a00640 1 -- 172.21.15.52:0/2672635928 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25002cba0 con 0x7fc25815a6a0 2024-03-12T17:07:44.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.039+0000 7fc257200640 1 -- 172.21.15.52:0/2672635928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc25815a6a0 msgr2=0x7fc25806f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:44.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.039+0000 7fc257200640 1 --2- 172.21.15.52:0/2672635928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc25815a6a0 0x7fc25806f4b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc250009d90 tx=0x7fc25002bce0 comp rx=0 tx=0).stop 2024-03-12T17:07:44.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.039+0000 7fc257200640 1 -- 172.21.15.52:0/2672635928 shutdown_connections 2024-03-12T17:07:44.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.039+0000 7fc257200640 1 --2- 172.21.15.52:0/2672635928 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc25815aa80 0x7fc2581646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:44.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.039+0000 7fc257200640 1 --2- 172.21.15.52:0/2672635928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc25815a6a0 0x7fc25806f4b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:44.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.039+0000 7fc257200640 1 -- 172.21.15.52:0/2672635928 >> 172.21.15.52:0/2672635928 conn(0x7fc2580d0520 msgr2=0x7fc2580d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:44.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.039+0000 7fc257200640 1 -- 172.21.15.52:0/2672635928 shutdown_connections 2024-03-12T17:07:44.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.039+0000 7fc257200640 1 -- 172.21.15.52:0/2672635928 wait complete. 2024-03-12T17:07:44.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.041+0000 7fc257200640 1 Processor -- start 2024-03-12T17:07:44.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.041+0000 7fc257200640 1 -- start start 2024-03-12T17:07:44.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.042+0000 7fc257200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2580afb70 0x7fc2581f3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:44.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.042+0000 7fc257200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc25815a6a0 0x7fc2581f4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:44.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.042+0000 7fc257200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc25815aa80 0x7fc25814e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:44.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.042+0000 7fc257200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2580cb390 con 0x7fc2580afb70 2024-03-12T17:07:44.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.042+0000 7fc257200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2580c9450 con 0x7fc25815a6a0 2024-03-12T17:07:44.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.042+0000 7fc257200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2580c9730 con 0x7fc25815aa80 2024-03-12T17:07:44.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.042+0000 7fc255e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2580afb70 0x7fc2581f3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:44.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.042+0000 7fc255e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2580afb70 0x7fc2581f3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51866/0 (socket says 172.21.15.52:51866) 2024-03-12T17:07:44.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.042+0000 7fc255e00640 1 -- 172.21.15.52:0/2662232347 learned_addr learned my addr 172.21.15.52:0/2662232347 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:44.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.042+0000 7fc255400640 1 --2- 172.21.15.52:0/2662232347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc25815a6a0 0x7fc2581f4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:44.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.042+0000 7fc256800640 1 --2- 172.21.15.52:0/2662232347 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc25815aa80 0x7fc25814e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:44.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.042+0000 7fc255e00640 1 -- 172.21.15.52:0/2662232347 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc25815aa80 msgr2=0x7fc25814e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:44.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.042+0000 7fc255e00640 1 --2- 172.21.15.52:0/2662232347 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc25815aa80 0x7fc25814e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:44.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.042+0000 7fc255e00640 1 -- 172.21.15.52:0/2662232347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc25815a6a0 msgr2=0x7fc2581f4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:44.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.042+0000 7fc255e00640 1 --2- 172.21.15.52:0/2662232347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc25815a6a0 0x7fc2581f4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:44.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.042+0000 7fc255e00640 1 -- 172.21.15.52:0/2662232347 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc250009a70 con 0x7fc2580afb70 2024-03-12T17:07:44.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.042+0000 7fc256800640 1 --2- 172.21.15.52:0/2662232347 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc25815aa80 0x7fc25814e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:44.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.042+0000 7fc255400640 1 --2- 172.21.15.52:0/2662232347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc25815a6a0 0x7fc2581f4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:44.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.043+0000 7fc255e00640 1 --2- 172.21.15.52:0/2662232347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2580afb70 0x7fc2581f3b40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc250009d90 tx=0x7fc250004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:44.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.043+0000 7fc24e000640 1 -- 172.21.15.52:0/2662232347 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc250035be0 con 0x7fc2580afb70 2024-03-12T17:07:44.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.043+0000 7fc24e000640 1 -- 172.21.15.52:0/2662232347 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc250035d40 con 0x7fc2580afb70 2024-03-12T17:07:44.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.043+0000 7fc24e000640 1 -- 172.21.15.52:0/2662232347 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25003e630 con 0x7fc2580afb70 2024-03-12T17:07:44.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.043+0000 7fc257200640 1 -- 172.21.15.52:0/2662232347 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2580c9a10 con 0x7fc2580afb70 2024-03-12T17:07:44.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.043+0000 7fc257200640 1 -- 172.21.15.52:0/2662232347 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc25815dc30 con 0x7fc2580afb70 2024-03-12T17:07:44.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.044+0000 7fc257200640 1 -- 172.21.15.52:0/2662232347 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2580cb540 con 0x7fc2580afb70 2024-03-12T17:07:44.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.045+0000 7fc24e000640 1 -- 172.21.15.52:0/2662232347 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc25003b050 con 0x7fc2580afb70 2024-03-12T17:07:44.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.046+0000 7fc24e000640 1 --2- 172.21.15.52:0/2662232347 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc23406ac30 0x7fc23406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:44.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.046+0000 7fc24e000640 1 -- 172.21.15.52:0/2662232347 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc2500abbc0 con 0x7fc2580afb70 2024-03-12T17:07:44.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.047+0000 7fc255400640 1 --2- 172.21.15.52:0/2662232347 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc23406ac30 0x7fc23406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:44.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.049+0000 7fc255400640 1 --2- 172.21.15.52:0/2662232347 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc23406ac30 0x7fc23406d0f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc2580ccf90 tx=0x7fc240009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:44.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.052+0000 7fc24e000640 1 -- 172.21.15.52:0/2662232347 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fc2580cb540 con 0x7fc2580afb70 2024-03-12T17:07:44.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.250+0000 7fc257200640 1 -- 172.21.15.52:0/2662232347 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc25804e030 con 0x7fc2580afb70 2024-03-12T17:07:44.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.251+0000 7fc24e000640 1 -- 172.21.15.52:0/2662232347 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc250039020 con 0x7fc2580afb70 2024-03-12T17:07:44.245 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-03-12T17:07:44.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.254+0000 7fc257200640 1 -- 172.21.15.52:0/2662232347 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc23406ac30 msgr2=0x7fc23406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:44.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.254+0000 7fc257200640 1 --2- 172.21.15.52:0/2662232347 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc23406ac30 0x7fc23406d0f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc2580ccf90 tx=0x7fc240009040 comp rx=0 tx=0).stop 2024-03-12T17:07:44.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.254+0000 7fc257200640 1 -- 172.21.15.52:0/2662232347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2580afb70 msgr2=0x7fc2581f3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:44.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.254+0000 7fc257200640 1 --2- 172.21.15.52:0/2662232347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2580afb70 0x7fc2581f3b40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc250009d90 tx=0x7fc250004a40 comp rx=0 tx=0).stop 2024-03-12T17:07:44.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.254+0000 7fc257200640 1 -- 172.21.15.52:0/2662232347 shutdown_connections 2024-03-12T17:07:44.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.254+0000 7fc257200640 1 --2- 172.21.15.52:0/2662232347 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc23406ac30 0x7fc23406d0f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:44.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.254+0000 7fc257200640 1 --2- 172.21.15.52:0/2662232347 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc25815aa80 0x7fc25814e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:44.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.254+0000 7fc257200640 1 --2- 172.21.15.52:0/2662232347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc25815a6a0 0x7fc2581f4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:44.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.254+0000 7fc257200640 1 --2- 172.21.15.52:0/2662232347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2580afb70 0x7fc2581f3b40 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:44.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.254+0000 7fc257200640 1 -- 172.21.15.52:0/2662232347 >> 172.21.15.52:0/2662232347 conn(0x7fc2580d0520 msgr2=0x7fc2580b0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:44.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.254+0000 7fc257200640 1 -- 172.21.15.52:0/2662232347 shutdown_connections 2024-03-12T17:07:44.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.255+0000 7fc257200640 1 -- 172.21.15.52:0/2662232347 wait complete. 2024-03-12T17:07:44.267 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-12T17:07:44.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.494+0000 7f266b200640 1 Processor -- start 2024-03-12T17:07:44.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.494+0000 7f266b200640 1 -- start start 2024-03-12T17:07:44.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.496+0000 7f266b200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f266c0cb560 0x7f266c0b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:44.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.496+0000 7f266b200640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f266c1575b0 con 0x7f266c0cdc00 2024-03-12T17:07:44.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.496+0000 7f266b200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f266c159f20 con 0x7f266c1571d0 2024-03-12T17:07:44.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.496+0000 7f266b200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f266c15a200 con 0x7f266c0cb560 2024-03-12T17:07:44.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.498+0000 7f2669400640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f266c0cdc00 0x7f266c0cae50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:44898/0 (socket says 172.21.15.52:44898) 2024-03-12T17:07:44.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.498+0000 7f2669400640 1 -- 172.21.15.52:0/1054059614 learned_addr learned my addr 172.21.15.52:0/1054059614 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:44.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.498+0000 7f266a800640 1 --2- 172.21.15.52:0/1054059614 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f266c0cb560 0x7f266c0b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:44.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.498+0000 7f2668a00640 1 -- 172.21.15.52:0/1054059614 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 215661293 0 0) 0x7f266c1575b0 con 0x7f266c0cdc00 2024-03-12T17:07:44.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.498+0000 7f2668a00640 1 -- 172.21.15.52:0/1054059614 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2650003520 con 0x7f266c0cdc00 2024-03-12T17:07:44.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.498+0000 7f2668a00640 1 -- 172.21.15.52:0/1054059614 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1225210151 0 0) 0x7f2650003520 con 0x7f266c0cdc00 2024-03-12T17:07:44.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.498+0000 7f2668a00640 1 -- 172.21.15.52:0/1054059614 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f266c0cb560 msgr2=0x7f266c0b35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:44.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.498+0000 7f2668a00640 1 --2- 172.21.15.52:0/1054059614 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f266c0cb560 0x7f266c0b35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:44.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.498+0000 7f2668a00640 1 -- 172.21.15.52:0/1054059614 >> v1:172.21.15.102:6789/0 conn(0x7f266c1571d0 legacy=0x7f266c15aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:44.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.498+0000 7f2668a00640 1 -- 172.21.15.52:0/1054059614 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266c15a4e0 con 0x7f266c0cdc00 2024-03-12T17:07:44.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.498+0000 7f2668a00640 1 -- 172.21.15.52:0/1054059614 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f2658001670 con 0x7f266c0cdc00 2024-03-12T17:07:44.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.498+0000 7f2668a00640 1 -- 172.21.15.52:0/1054059614 >> v1:172.21.15.52:6789/0 conn(0x7f266c0cdc00 legacy=0x7f266c0cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:44.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.498+0000 7f2668a00640 1 --2- 172.21.15.52:0/1054059614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26500039a0 0x7f2650003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:44.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.498+0000 7f2668a00640 1 --2- 172.21.15.52:0/1054059614 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f266c0cb560 0x7f26500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:44.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.498+0000 7f2668a00640 1 --2- 172.21.15.52:0/1054059614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2650004d00 0x7f2650005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:44.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.498+0000 7f2668a00640 1 -- 172.21.15.52:0/1054059614 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266c15a200 con 0x7f26500039a0 2024-03-12T17:07:44.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.498+0000 7f2668a00640 1 -- 172.21.15.52:0/1054059614 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265000ba60 con 0x7f2650004d00 2024-03-12T17:07:44.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.498+0000 7f2668a00640 1 -- 172.21.15.52:0/1054059614 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f265000de30 con 0x7f266c0cb560 2024-03-12T17:07:44.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.498+0000 7f2669e00640 1 --2- 172.21.15.52:0/1054059614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26500039a0 0x7f2650003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:44.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.498+0000 7f2669e00640 1 -- 172.21.15.52:0/1054059614 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f266c0cb560 msgr2=0x7f26500047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:44.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.499+0000 7f2669e00640 1 --2- 172.21.15.52:0/1054059614 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f266c0cb560 0x7f26500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:44.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.499+0000 7f2669e00640 1 -- 172.21.15.52:0/1054059614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2650004d00 msgr2=0x7f2650005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:44.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.499+0000 7f2669e00640 1 --2- 172.21.15.52:0/1054059614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2650004d00 0x7f2650005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:44.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.499+0000 7f2669e00640 1 -- 172.21.15.52:0/1054059614 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2650003520 con 0x7f26500039a0 2024-03-12T17:07:44.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.500+0000 7f2669e00640 1 --2- 172.21.15.52:0/1054059614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26500039a0 0x7f2650003dc0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f265c001e20 tx=0x7f26500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc8ab64c7b64a633 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:44.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.500+0000 7f2668a00640 1 -- 172.21.15.52:0/1054059614 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c003b70 con 0x7f26500039a0 2024-03-12T17:07:44.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.500+0000 7f2668a00640 1 -- 172.21.15.52:0/1054059614 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f265c003cd0 con 0x7f26500039a0 2024-03-12T17:07:44.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.500+0000 7f2668a00640 1 -- 172.21.15.52:0/1054059614 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c005310 con 0x7f26500039a0 2024-03-12T17:07:44.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.501+0000 7f266b200640 1 -- 172.21.15.52:0/1054059614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26500039a0 msgr2=0x7f2650003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:44.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.501+0000 7f266b200640 1 --2- 172.21.15.52:0/1054059614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26500039a0 0x7f2650003dc0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f265c001e20 tx=0x7f26500057f0 comp rx=0 tx=0).stop 2024-03-12T17:07:44.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.501+0000 7f266a800640 1 -- 172.21.15.52:0/1054059614 reap_dead start 2024-03-12T17:07:44.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.501+0000 7f266b200640 1 -- 172.21.15.52:0/1054059614 shutdown_connections 2024-03-12T17:07:44.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.501+0000 7f266b200640 1 -- 172.21.15.52:0/1054059614 >> 172.21.15.52:0/1054059614 conn(0x7f266c065df0 msgr2=0x7f266c069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:44.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.501+0000 7f266b200640 1 -- 172.21.15.52:0/1054059614 shutdown_connections 2024-03-12T17:07:44.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.501+0000 7f266b200640 1 -- 172.21.15.52:0/1054059614 wait complete. 2024-03-12T17:07:44.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.504+0000 7f266b200640 1 Processor -- start 2024-03-12T17:07:44.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.504+0000 7f266b200640 1 -- start start 2024-03-12T17:07:44.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.505+0000 7f266b200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f26500039a0 0x7f266c141230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:44.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.505+0000 7f266b200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f266c1f2010 0x7f266c0aa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:44.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.505+0000 7f266b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f266c141e00 0x7f266c142280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:44.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.505+0000 7f266b200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266c1599e0 con 0x7f266c1f2010 2024-03-12T17:07:44.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.505+0000 7f266b200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266c1427c0 con 0x7f266c141e00 2024-03-12T17:07:44.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.505+0000 7f266b200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f266c1f2490 con 0x7f26500039a0 2024-03-12T17:07:44.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.505+0000 7f2669400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f266c1f2010 0x7f266c0aa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:44.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.505+0000 7f2669400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f266c1f2010 0x7f266c0aa8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51880/0 (socket says 172.21.15.52:51880) 2024-03-12T17:07:44.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.505+0000 7f2669400640 1 -- 172.21.15.52:0/3775443411 learned_addr learned my addr 172.21.15.52:0/3775443411 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:44.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.505+0000 7f266a800640 1 --2- 172.21.15.52:0/3775443411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f266c141e00 0x7f266c142280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:44.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.505+0000 7f2669e00640 1 --2- 172.21.15.52:0/3775443411 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f26500039a0 0x7f266c141230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:44.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.505+0000 7f2669400640 1 -- 172.21.15.52:0/3775443411 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f26500039a0 msgr2=0x7f266c141230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:44.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.505+0000 7f2669400640 1 --2- 172.21.15.52:0/3775443411 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f26500039a0 0x7f266c141230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:44.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.505+0000 7f2669400640 1 -- 172.21.15.52:0/3775443411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f266c141e00 msgr2=0x7f266c142280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:44.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.505+0000 7f2669400640 1 --2- 172.21.15.52:0/3775443411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f266c141e00 0x7f266c142280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:44.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.505+0000 7f2669400640 1 -- 172.21.15.52:0/3775443411 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266c0aaff0 con 0x7f266c1f2010 2024-03-12T17:07:44.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.505+0000 7f266a800640 1 --2- 172.21.15.52:0/3775443411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f266c141e00 0x7f266c142280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:44.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.505+0000 7f2669400640 1 --2- 172.21.15.52:0/3775443411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f266c1f2010 0x7f266c0aa8f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f265800c730 tx=0x7f265800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:44.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.505+0000 7f2662000640 1 -- 172.21.15.52:0/3775443411 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26580072d0 con 0x7f266c1f2010 2024-03-12T17:07:44.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.505+0000 7f2662000640 1 -- 172.21.15.52:0/3775443411 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2658007430 con 0x7f266c1f2010 2024-03-12T17:07:44.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.505+0000 7f2662000640 1 -- 172.21.15.52:0/3775443411 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26580047e0 con 0x7f266c1f2010 2024-03-12T17:07:44.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.505+0000 7f266b200640 1 -- 172.21.15.52:0/3775443411 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f266c0ab2d0 con 0x7f266c1f2010 2024-03-12T17:07:44.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.505+0000 7f266b200640 1 -- 172.21.15.52:0/3775443411 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f266c1f1810 con 0x7f266c1f2010 2024-03-12T17:07:44.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.506+0000 7f266b200640 1 -- 172.21.15.52:0/3775443411 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f266c04e030 con 0x7f266c1f2010 2024-03-12T17:07:44.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.507+0000 7f2662000640 1 -- 172.21.15.52:0/3775443411 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2658004b50 con 0x7f266c1f2010 2024-03-12T17:07:44.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.508+0000 7f2662000640 1 --2- 172.21.15.52:0/3775443411 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f264c06ac00 0x7f264c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:44.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.508+0000 7f2662000640 1 -- 172.21.15.52:0/3775443411 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2658088cf0 con 0x7f266c1f2010 2024-03-12T17:07:44.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.510+0000 7f2669e00640 1 --2- 172.21.15.52:0/3775443411 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f264c06ac00 0x7f264c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:44.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.510+0000 7f2669e00640 1 --2- 172.21.15.52:0/3775443411 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f264c06ac00 0x7f264c06d0c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f265c034000 tx=0x7f265c009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:44.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.513+0000 7f2662000640 1 -- 172.21.15.52:0/3775443411 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f2658053a70 con 0x7f266c1f2010 2024-03-12T17:07:44.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.713+0000 7f266b200640 1 -- 172.21.15.52:0/3775443411 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f266c1587d0 con 0x7f266c1f2010 2024-03-12T17:07:44.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.713+0000 7f2662000640 1 -- 172.21.15.52:0/3775443411 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f26580180a0 con 0x7f266c1f2010 2024-03-12T17:07:44.710 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-12T17:07:44.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.716+0000 7f266b200640 1 -- 172.21.15.52:0/3775443411 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f264c06ac00 msgr2=0x7f264c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:44.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.716+0000 7f266b200640 1 --2- 172.21.15.52:0/3775443411 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f264c06ac00 0x7f264c06d0c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f265c034000 tx=0x7f265c009030 comp rx=0 tx=0).stop 2024-03-12T17:07:44.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.716+0000 7f266b200640 1 -- 172.21.15.52:0/3775443411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f266c1f2010 msgr2=0x7f266c0aa8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:44.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.716+0000 7f266b200640 1 --2- 172.21.15.52:0/3775443411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f266c1f2010 0x7f266c0aa8f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f265800c730 tx=0x7f265800cbf0 comp rx=0 tx=0).stop 2024-03-12T17:07:44.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.716+0000 7f266b200640 1 -- 172.21.15.52:0/3775443411 shutdown_connections 2024-03-12T17:07:44.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.717+0000 7f266b200640 1 --2- 172.21.15.52:0/3775443411 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f264c06ac00 0x7f264c06d0c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:44.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.717+0000 7f266b200640 1 --2- 172.21.15.52:0/3775443411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f266c141e00 0x7f266c142280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:44.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.717+0000 7f266b200640 1 --2- 172.21.15.52:0/3775443411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f266c1f2010 0x7f266c0aa8f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:44.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.717+0000 7f266b200640 1 --2- 172.21.15.52:0/3775443411 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f26500039a0 0x7f266c141230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:44.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.717+0000 7f266b200640 1 -- 172.21.15.52:0/3775443411 >> 172.21.15.52:0/3775443411 conn(0x7f266c065df0 msgr2=0x7f266c157760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:44.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.717+0000 7f266b200640 1 -- 172.21.15.52:0/3775443411 shutdown_connections 2024-03-12T17:07:44.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.717+0000 7f266b200640 1 -- 172.21.15.52:0/3775443411 wait complete. 2024-03-12T17:07:44.732 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-12T17:07:44.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.958+0000 7f32c2c00640 1 Processor -- start 2024-03-12T17:07:44.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.959+0000 7f32c2c00640 1 -- start start 2024-03-12T17:07:44.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.960+0000 7f32c2c00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32bc0cd830 0x7f32bc160e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:44.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.960+0000 7f32c2c00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32bc1613d0 con 0x7f32bc0cf7c0 2024-03-12T17:07:44.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.960+0000 7f32c2c00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32bc0ccd90 con 0x7f32bc15c820 2024-03-12T17:07:44.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.960+0000 7f32c2c00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc0cdcc0 con 0x7f32bc0cd830 2024-03-12T17:07:44.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.961+0000 7f32c0e00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f32bc0cf7c0 0x7f32bc0d0560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:46366/0 (socket says 172.21.15.52:46366) 2024-03-12T17:07:44.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.961+0000 7f32c0e00640 1 -- 172.21.15.52:0/1898318027 learned_addr learned my addr 172.21.15.52:0/1898318027 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:44.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.961+0000 7f32c2200640 1 --2- 172.21.15.52:0/1898318027 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32bc0cd830 0x7f32bc160e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:44.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.961+0000 7f32bbe00640 1 -- 172.21.15.52:0/1898318027 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 427180336 0 0) 0x7f32a8001110 con 0x7f32bc0cf7c0 2024-03-12T17:07:44.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.961+0000 7f32bbe00640 1 -- 172.21.15.52:0/1898318027 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32a4003520 con 0x7f32bc0cf7c0 2024-03-12T17:07:44.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.962+0000 7f32bbe00640 1 -- 172.21.15.52:0/1898318027 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1232301415 0 0) 0x7f32a4003520 con 0x7f32bc0cf7c0 2024-03-12T17:07:44.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.962+0000 7f32bbe00640 1 -- 172.21.15.52:0/1898318027 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32bc0cd830 msgr2=0x7f32bc160e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:44.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.962+0000 7f32bbe00640 1 --2- 172.21.15.52:0/1898318027 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32bc0cd830 0x7f32bc160e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:44.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.962+0000 7f32bbe00640 1 -- 172.21.15.52:0/1898318027 >> v1:172.21.15.102:6789/0 conn(0x7f32bc15c820 legacy=0x7f32bc0cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:44.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.962+0000 7f32bbe00640 1 -- 172.21.15.52:0/1898318027 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32bc0b7fc0 con 0x7f32bc0cf7c0 2024-03-12T17:07:44.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.962+0000 7f32bbe00640 1 -- 172.21.15.52:0/1898318027 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f32a8001800 con 0x7f32bc0cf7c0 2024-03-12T17:07:44.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.962+0000 7f32bbe00640 1 -- 172.21.15.52:0/1898318027 >> v1:172.21.15.52:6789/0 conn(0x7f32bc0cf7c0 legacy=0x7f32bc0d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:44.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.962+0000 7f32bbe00640 1 --2- 172.21.15.52:0/1898318027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32a40039a0 0x7f32a4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:44.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.962+0000 7f32bbe00640 1 --2- 172.21.15.52:0/1898318027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32a40047c0 0x7f32a4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:44.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.962+0000 7f32bbe00640 1 --2- 172.21.15.52:0/1898318027 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32bc0cd830 0x7f32a4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:44.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.962+0000 7f32bbe00640 1 -- 172.21.15.52:0/1898318027 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc0cdcc0 con 0x7f32a40047c0 2024-03-12T17:07:44.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.962+0000 7f32bbe00640 1 -- 172.21.15.52:0/1898318027 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a400bac0 con 0x7f32a40039a0 2024-03-12T17:07:44.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.962+0000 7f32bbe00640 1 -- 172.21.15.52:0/1898318027 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a400bc80 con 0x7f32bc0cd830 2024-03-12T17:07:44.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.962+0000 7f32c0e00640 1 --2- 172.21.15.52:0/1898318027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32a40047c0 0x7f32a4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:44.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.962+0000 7f32c1800640 1 --2- 172.21.15.52:0/1898318027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32a40039a0 0x7f32a4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:44.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.963+0000 7f32c2200640 1 --2- 172.21.15.52:0/1898318027 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32bc0cd830 0x7f32a4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:44.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.963+0000 7f32c0e00640 1 -- 172.21.15.52:0/1898318027 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32bc0cd830 msgr2=0x7f32a4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:44.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.963+0000 7f32c0e00640 1 --2- 172.21.15.52:0/1898318027 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32bc0cd830 0x7f32a4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:44.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.963+0000 7f32c0e00640 1 -- 172.21.15.52:0/1898318027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32a40039a0 msgr2=0x7f32a4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:44.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.963+0000 7f32c0e00640 1 --2- 172.21.15.52:0/1898318027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32a40039a0 0x7f32a4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:44.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.963+0000 7f32c0e00640 1 -- 172.21.15.52:0/1898318027 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a4003520 con 0x7f32a40047c0 2024-03-12T17:07:44.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.963+0000 7f32c2200640 1 --2- 172.21.15.52:0/1898318027 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32bc0cd830 0x7f32a4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:44.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.964+0000 7f32c0e00640 1 --2- 172.21.15.52:0/1898318027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32a40047c0 0x7f32a4004c40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f32a800c210 tx=0x7f32a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4052d77ffc39d1ee server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:44.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.964+0000 7f32bbe00640 1 -- 172.21.15.52:0/1898318027 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a8005ad0 con 0x7f32a40047c0 2024-03-12T17:07:44.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.964+0000 7f32bbe00640 1 -- 172.21.15.52:0/1898318027 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a8005c50 con 0x7f32a40047c0 2024-03-12T17:07:44.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.964+0000 7f32bbe00640 1 -- 172.21.15.52:0/1898318027 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a802c870 con 0x7f32a40047c0 2024-03-12T17:07:44.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.965+0000 7f32c2c00640 1 -- 172.21.15.52:0/1898318027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32a40047c0 msgr2=0x7f32a4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:44.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.965+0000 7f32c2c00640 1 --2- 172.21.15.52:0/1898318027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32a40047c0 0x7f32a4004c40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f32a800c210 tx=0x7f32a40057f0 comp rx=0 tx=0).stop 2024-03-12T17:07:44.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.965+0000 7f32c2200640 1 -- 172.21.15.52:0/1898318027 reap_dead start 2024-03-12T17:07:44.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.965+0000 7f32c2c00640 1 -- 172.21.15.52:0/1898318027 shutdown_connections 2024-03-12T17:07:44.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.965+0000 7f32c2c00640 1 -- 172.21.15.52:0/1898318027 >> 172.21.15.52:0/1898318027 conn(0x7f32bc06b050 msgr2=0x7f32bc06b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:44.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.965+0000 7f32c2c00640 1 -- 172.21.15.52:0/1898318027 shutdown_connections 2024-03-12T17:07:44.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.965+0000 7f32c2c00640 1 -- 172.21.15.52:0/1898318027 wait complete. 2024-03-12T17:07:44.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.967+0000 7f32c2c00640 1 Processor -- start 2024-03-12T17:07:44.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.967+0000 7f32c2c00640 1 -- start start 2024-03-12T17:07:44.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.968+0000 7f32c2c00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32a40047c0 0x7f32bc150c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:44.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.968+0000 7f32c2c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32bc1f5900 0x7f32bc1f5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:44.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.968+0000 7f32c2c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32bc1566d0 0x7f32bc156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:44.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.968+0000 7f32c2c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc1f62a0 con 0x7f32bc1f5900 2024-03-12T17:07:44.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.968+0000 7f32c2c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc1580a0 con 0x7f32bc1566d0 2024-03-12T17:07:44.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.968+0000 7f32c2c00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc157490 con 0x7f32a40047c0 2024-03-12T17:07:44.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.968+0000 7f32c0e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32bc1f5900 0x7f32bc1f5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:44.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.968+0000 7f32c0e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32bc1f5900 0x7f32bc1f5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55618/0 (socket says 172.21.15.52:55618) 2024-03-12T17:07:44.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.968+0000 7f32c0e00640 1 -- 172.21.15.52:0/1952009352 learned_addr learned my addr 172.21.15.52:0/1952009352 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:44.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.968+0000 7f32c1800640 1 --2- 172.21.15.52:0/1952009352 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32a40047c0 0x7f32bc150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:44.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.969+0000 7f32c0e00640 1 -- 172.21.15.52:0/1952009352 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32a40047c0 msgr2=0x7f32bc150c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:44.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.969+0000 7f32c0e00640 1 --2- 172.21.15.52:0/1952009352 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32a40047c0 0x7f32bc150c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:44.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.969+0000 7f32c0e00640 1 -- 172.21.15.52:0/1952009352 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32bc1566d0 msgr2=0x7f32bc156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:44.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.969+0000 7f32c0e00640 1 --2- 172.21.15.52:0/1952009352 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32bc1566d0 0x7f32bc156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:44.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.969+0000 7f32c0e00640 1 -- 172.21.15.52:0/1952009352 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32bc157770 con 0x7f32bc1f5900 2024-03-12T17:07:44.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.969+0000 7f32c0e00640 1 --2- 172.21.15.52:0/1952009352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32bc1f5900 0x7f32bc1f5d60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f32a802c410 tx=0x7f32a8001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:44.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.969+0000 7f32b9600640 1 -- 172.21.15.52:0/1952009352 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a803c040 con 0x7f32bc1f5900 2024-03-12T17:07:44.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.969+0000 7f32b9600640 1 -- 172.21.15.52:0/1952009352 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a802cc80 con 0x7f32bc1f5900 2024-03-12T17:07:44.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.969+0000 7f32b9600640 1 -- 172.21.15.52:0/1952009352 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a8005e20 con 0x7f32bc1f5900 2024-03-12T17:07:44.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.969+0000 7f32c2c00640 1 -- 172.21.15.52:0/1952009352 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32bc157a50 con 0x7f32bc1f5900 2024-03-12T17:07:44.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.969+0000 7f32c2c00640 1 -- 172.21.15.52:0/1952009352 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32bc151160 con 0x7f32bc1f5900 2024-03-12T17:07:44.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.971+0000 7f32c2c00640 1 -- 172.21.15.52:0/1952009352 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32bc158250 con 0x7f32bc1f5900 2024-03-12T17:07:44.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.971+0000 7f32b9600640 1 -- 172.21.15.52:0/1952009352 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f32a8032070 con 0x7f32bc1f5900 2024-03-12T17:07:44.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.972+0000 7f32b9600640 1 --2- 172.21.15.52:0/1952009352 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f329c06ac30 0x7f329c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:44.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.972+0000 7f32b9600640 1 -- 172.21.15.52:0/1952009352 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f32a80aa660 con 0x7f32bc1f5900 2024-03-12T17:07:44.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.973+0000 7f32c1800640 1 --2- 172.21.15.52:0/1952009352 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f329c06ac30 0x7f329c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:44.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.974+0000 7f32c1800640 1 --2- 172.21.15.52:0/1952009352 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f329c06ac30 0x7f329c06d0f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f32b0007fd0 tx=0x7f32b0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:44.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:44.977+0000 7f32b9600640 1 -- 172.21.15.52:0/1952009352 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f32bc158250 con 0x7f32bc1f5900 2024-03-12T17:07:45.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.180+0000 7f32c2c00640 1 -- 172.21.15.52:0/1952009352 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f32bc1613d0 con 0x7f32bc1f5900 2024-03-12T17:07:45.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.181+0000 7f32b9600640 1 -- 172.21.15.52:0/1952009352 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f32bc1613d0 con 0x7f32bc1f5900 2024-03-12T17:07:45.177 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-03-12T17:07:45.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.183+0000 7f32c2c00640 1 -- 172.21.15.52:0/1952009352 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f329c06ac30 msgr2=0x7f329c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:45.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.183+0000 7f32c2c00640 1 --2- 172.21.15.52:0/1952009352 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f329c06ac30 0x7f329c06d0f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f32b0007fd0 tx=0x7f32b0009040 comp rx=0 tx=0).stop 2024-03-12T17:07:45.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.183+0000 7f32c2c00640 1 -- 172.21.15.52:0/1952009352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32bc1f5900 msgr2=0x7f32bc1f5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:45.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.183+0000 7f32c2c00640 1 --2- 172.21.15.52:0/1952009352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32bc1f5900 0x7f32bc1f5d60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f32a802c410 tx=0x7f32a8001b90 comp rx=0 tx=0).stop 2024-03-12T17:07:45.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.183+0000 7f32c2c00640 1 -- 172.21.15.52:0/1952009352 shutdown_connections 2024-03-12T17:07:45.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.183+0000 7f32c2c00640 1 --2- 172.21.15.52:0/1952009352 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f329c06ac30 0x7f329c06d0f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:45.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.183+0000 7f32c2c00640 1 --2- 172.21.15.52:0/1952009352 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32bc1566d0 0x7f32bc156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:45.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.183+0000 7f32c2c00640 1 --2- 172.21.15.52:0/1952009352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32bc1f5900 0x7f32bc1f5d60 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:45.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.183+0000 7f32c2c00640 1 --2- 172.21.15.52:0/1952009352 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32a40047c0 0x7f32bc150c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:45.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.183+0000 7f32c2c00640 1 -- 172.21.15.52:0/1952009352 >> 172.21.15.52:0/1952009352 conn(0x7f32bc06b050 msgr2=0x7f32bc0caf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:45.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.183+0000 7f32c2c00640 1 -- 172.21.15.52:0/1952009352 shutdown_connections 2024-03-12T17:07:45.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.183+0000 7f32c2c00640 1 -- 172.21.15.52:0/1952009352 wait complete. 2024-03-12T17:07:45.196 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-12T17:07:45.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.432+0000 7f09dd000640 1 Processor -- start 2024-03-12T17:07:45.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.432+0000 7f09dd000640 1 -- start start 2024-03-12T17:07:45.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.434+0000 7f09dd000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f09d80cafc0 0x7f09d806a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:45.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.434+0000 7f09dd000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f09d815e860 0x7f09d815ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:45.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.434+0000 7f09dd000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09d815e3d0 con 0x7f09d815ac10 2024-03-12T17:07:45.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.434+0000 7f09dd000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09d8143d70 con 0x7f09d80cafc0 2024-03-12T17:07:45.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.434+0000 7f09dd000640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d815b3d0 con 0x7f09d815e860 2024-03-12T17:07:45.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.435+0000 7f09d7400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f09d80cafc0 0x7f09d806a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:45.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.435+0000 7f09d7e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f09d815e860 0x7f09d815ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:45.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.435+0000 7f09d7400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f09d80cafc0 0x7f09d806a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:51256/0 (socket says 172.21.15.52:51256) 2024-03-12T17:07:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.435+0000 7f09d7400640 1 -- 172.21.15.52:0/4045183859 learned_addr learned my addr 172.21.15.52:0/4045183859 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.435+0000 7f09d7400640 1 -- 172.21.15.52:0/4045183859 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f09d815e860 msgr2=0x7f09d815ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.435+0000 7f09d7400640 1 --2- 172.21.15.52:0/4045183859 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f09d815e860 0x7f09d815ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.435+0000 7f09d7400640 1 -- 172.21.15.52:0/4045183859 >> v1:172.21.15.102:6789/0 conn(0x7f09d815ac10 legacy=0x7f09d80cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.435+0000 7f09d7400640 1 -- 172.21.15.52:0/4045183859 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d815b6b0 con 0x7f09d80cafc0 2024-03-12T17:07:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.437+0000 7f09d7400640 1 --2- 172.21.15.52:0/4045183859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09d80cafc0 0x7f09d806a6a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f09c8009870 tx=0x7f09c802b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=450331db95710956 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:45.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.437+0000 7f09d6000640 1 -- 172.21.15.52:0/4045183859 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c8004230 con 0x7f09d80cafc0 2024-03-12T17:07:45.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.437+0000 7f09d6000640 1 -- 172.21.15.52:0/4045183859 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c802e050 con 0x7f09d80cafc0 2024-03-12T17:07:45.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.437+0000 7f09d6000640 1 -- 172.21.15.52:0/4045183859 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c8003610 con 0x7f09d80cafc0 2024-03-12T17:07:45.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.438+0000 7f09dd000640 1 -- 172.21.15.52:0/4045183859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09d80cafc0 msgr2=0x7f09d806a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:45.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.438+0000 7f09dd000640 1 --2- 172.21.15.52:0/4045183859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09d80cafc0 0x7f09d806a6a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f09c8009870 tx=0x7f09c802b7e0 comp rx=0 tx=0).stop 2024-03-12T17:07:45.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.438+0000 7f09dd000640 1 -- 172.21.15.52:0/4045183859 shutdown_connections 2024-03-12T17:07:45.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.438+0000 7f09dd000640 1 --2- 172.21.15.52:0/4045183859 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f09d815e860 0x7f09d815ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:45.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.438+0000 7f09dd000640 1 --2- 172.21.15.52:0/4045183859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09d80cafc0 0x7f09d806a6a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:45.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.438+0000 7f09dd000640 1 -- 172.21.15.52:0/4045183859 >> 172.21.15.52:0/4045183859 conn(0x7f09d8098a40 msgr2=0x7f09d80b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:45.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.438+0000 7f09dd000640 1 -- 172.21.15.52:0/4045183859 shutdown_connections 2024-03-12T17:07:45.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.439+0000 7f09dd000640 1 -- 172.21.15.52:0/4045183859 wait complete. 2024-03-12T17:07:45.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.440+0000 7f09dd000640 1 Processor -- start 2024-03-12T17:07:45.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.440+0000 7f09dd000640 1 -- start start 2024-03-12T17:07:45.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.441+0000 7f09dd000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f09d80cafc0 0x7f09d8159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:45.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.441+0000 7f09dd000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09d815ac10 0x7f09d8142820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:45.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.441+0000 7f09dd000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09d815e860 0x7f09d8142d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:45.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.441+0000 7f09dd000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d8142170 con 0x7f09d815ac10 2024-03-12T17:07:45.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.441+0000 7f09dd000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d8132780 con 0x7f09d815e860 2024-03-12T17:07:45.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.441+0000 7f09dd000640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d812ff10 con 0x7f09d80cafc0 2024-03-12T17:07:45.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.442+0000 7f09d6a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09d815ac10 0x7f09d8142820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:45.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.442+0000 7f09d6a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09d815ac10 0x7f09d8142820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55622/0 (socket says 172.21.15.52:55622) 2024-03-12T17:07:45.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.442+0000 7f09d6a00640 1 -- 172.21.15.52:0/1779416989 learned_addr learned my addr 172.21.15.52:0/1779416989 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:45.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.442+0000 7f09d7400640 1 --2- 172.21.15.52:0/1779416989 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f09d80cafc0 0x7f09d8159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:45.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.442+0000 7f09d7e00640 1 --2- 172.21.15.52:0/1779416989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09d815e860 0x7f09d8142d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:45.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.442+0000 7f09d6a00640 1 -- 172.21.15.52:0/1779416989 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f09d80cafc0 msgr2=0x7f09d8159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:45.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.442+0000 7f09d6a00640 1 --2- 172.21.15.52:0/1779416989 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f09d80cafc0 0x7f09d8159e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:45.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.442+0000 7f09d6a00640 1 -- 172.21.15.52:0/1779416989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09d815e860 msgr2=0x7f09d8142d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:45.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.442+0000 7f09d6a00640 1 --2- 172.21.15.52:0/1779416989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09d815e860 0x7f09d8142d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:45.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.442+0000 7f09d6a00640 1 -- 172.21.15.52:0/1779416989 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c80094d0 con 0x7f09d815ac10 2024-03-12T17:07:45.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.443+0000 7f09d6a00640 1 --2- 172.21.15.52:0/1779416989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09d815ac10 0x7f09d8142820 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f09c400c7b0 tx=0x7f09c400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:45.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.443+0000 7f09cf400640 1 -- 172.21.15.52:0/1779416989 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c4005270 con 0x7f09d815ac10 2024-03-12T17:07:45.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.443+0000 7f09cf400640 1 -- 172.21.15.52:0/1779416989 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c40053d0 con 0x7f09d815ac10 2024-03-12T17:07:45.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.443+0000 7f09cf400640 1 -- 172.21.15.52:0/1779416989 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c400a860 con 0x7f09d815ac10 2024-03-12T17:07:45.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.443+0000 7f09dd000640 1 -- 172.21.15.52:0/1779416989 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d81324a0 con 0x7f09d815ac10 2024-03-12T17:07:45.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.443+0000 7f09dd000640 1 -- 172.21.15.52:0/1779416989 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d812fba0 con 0x7f09d815ac10 2024-03-12T17:07:45.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.444+0000 7f09dd000640 1 -- 172.21.15.52:0/1779416989 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09d80ca2b0 con 0x7f09d815ac10 2024-03-12T17:07:45.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.445+0000 7f09cf400640 1 -- 172.21.15.52:0/1779416989 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f09c400a9c0 con 0x7f09d815ac10 2024-03-12T17:07:45.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.447+0000 7f09cf400640 1 --2- 172.21.15.52:0/1779416989 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f09b806ac30 0x7f09b806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:45.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.447+0000 7f09cf400640 1 -- 172.21.15.52:0/1779416989 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f09c4088e30 con 0x7f09d815ac10 2024-03-12T17:07:45.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.447+0000 7f09d7400640 1 --2- 172.21.15.52:0/1779416989 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f09b806ac30 0x7f09b806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:45.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.448+0000 7f09d7400640 1 --2- 172.21.15.52:0/1779416989 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f09b806ac30 0x7f09b806d0f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f09c8034000 tx=0x7f09c8008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:45.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.451+0000 7f09cf400640 1 -- 172.21.15.52:0/1779416989 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f09c4053c40 con 0x7f09d815ac10 2024-03-12T17:07:45.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.648+0000 7f09dd000640 1 -- 172.21.15.52:0/1779416989 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f09d804e030 con 0x7f09d815ac10 2024-03-12T17:07:45.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.649+0000 7f09cf400640 1 -- 172.21.15.52:0/1779416989 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f09c4017070 con 0x7f09d815ac10 2024-03-12T17:07:45.642 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-12T17:07:45.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.651+0000 7f09dd000640 1 -- 172.21.15.52:0/1779416989 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f09b806ac30 msgr2=0x7f09b806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:45.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.651+0000 7f09dd000640 1 --2- 172.21.15.52:0/1779416989 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f09b806ac30 0x7f09b806d0f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f09c8034000 tx=0x7f09c8008030 comp rx=0 tx=0).stop 2024-03-12T17:07:45.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.651+0000 7f09dd000640 1 -- 172.21.15.52:0/1779416989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09d815ac10 msgr2=0x7f09d8142820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:45.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.651+0000 7f09dd000640 1 --2- 172.21.15.52:0/1779416989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09d815ac10 0x7f09d8142820 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f09c400c7b0 tx=0x7f09c400cc70 comp rx=0 tx=0).stop 2024-03-12T17:07:45.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.652+0000 7f09dd000640 1 -- 172.21.15.52:0/1779416989 shutdown_connections 2024-03-12T17:07:45.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.652+0000 7f09dd000640 1 --2- 172.21.15.52:0/1779416989 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f09b806ac30 0x7f09b806d0f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:45.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.652+0000 7f09dd000640 1 --2- 172.21.15.52:0/1779416989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09d815e860 0x7f09d8142d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:45.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.652+0000 7f09dd000640 1 --2- 172.21.15.52:0/1779416989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09d815ac10 0x7f09d8142820 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:45.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.652+0000 7f09dd000640 1 --2- 172.21.15.52:0/1779416989 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f09d80cafc0 0x7f09d8159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:45.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.652+0000 7f09dd000640 1 -- 172.21.15.52:0/1779416989 >> 172.21.15.52:0/1779416989 conn(0x7f09d8098a40 msgr2=0x7f09d815dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:45.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.652+0000 7f09dd000640 1 -- 172.21.15.52:0/1779416989 shutdown_connections 2024-03-12T17:07:45.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.652+0000 7f09dd000640 1 -- 172.21.15.52:0/1779416989 wait complete. 2024-03-12T17:07:45.662 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-12T17:07:45.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.890+0000 7f540d600640 1 Processor -- start 2024-03-12T17:07:45.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.891+0000 7f540d600640 1 -- start start 2024-03-12T17:07:45.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.892+0000 7f540d600640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f54080b5a30 0x7f54080b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:45.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.892+0000 7f540d600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f54080b4350 0x7f54080b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:45.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.892+0000 7f540d600640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54080d1830 con 0x7f540815cda0 2024-03-12T17:07:45.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.892+0000 7f540d600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54080b4d20 con 0x7f54080b4350 2024-03-12T17:07:45.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.892+0000 7f540d600640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54080b1f20 con 0x7f54080b5a30 2024-03-12T17:07:45.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.892+0000 7f5406a00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f540815cda0 0x7f54080cdb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:46376/0 (socket says 172.21.15.52:46376) 2024-03-12T17:07:45.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.892+0000 7f5406a00640 1 -- 172.21.15.52:0/1512037948 learned_addr learned my addr 172.21.15.52:0/1512037948 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:45.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.893+0000 7f5405600640 1 -- 172.21.15.52:0/1512037948 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 951263104 0 0) 0x7f54080d1830 con 0x7f540815cda0 2024-03-12T17:07:45.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.893+0000 7f5406000640 1 --2- 172.21.15.52:0/1512037948 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f54080b5a30 0x7f54080b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:45.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.893+0000 7f5405600640 1 -- 172.21.15.52:0/1512037948 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53ec003520 con 0x7f540815cda0 2024-03-12T17:07:45.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.893+0000 7f5407400640 1 --2- 172.21.15.52:0/1512037948 >> v2:172.21.15.102:3300/0 conn(0x7f54080b4350 0x7f54080b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:45.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.893+0000 7f5405600640 1 -- 172.21.15.52:0/1512037948 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2656736087 0 0) 0x7f53ec003520 con 0x7f540815cda0 2024-03-12T17:07:45.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.893+0000 7f5405600640 1 -- 172.21.15.52:0/1512037948 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f54080b5a30 msgr2=0x7f54080b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:45.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.893+0000 7f5405600640 1 --2- 172.21.15.52:0/1512037948 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f54080b5a30 0x7f54080b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:45.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.893+0000 7f5405600640 1 -- 172.21.15.52:0/1512037948 >> v2:172.21.15.102:3300/0 conn(0x7f54080b4350 msgr2=0x7f54080b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:45.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.893+0000 7f5405600640 1 --2- 172.21.15.52:0/1512037948 >> v2:172.21.15.102:3300/0 conn(0x7f54080b4350 0x7f54080b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:45.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.893+0000 7f5405600640 1 -- 172.21.15.52:0/1512037948 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54080d0c20 con 0x7f540815cda0 2024-03-12T17:07:45.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.893+0000 7f5405600640 1 -- 172.21.15.52:0/1512037948 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f53f0001670 con 0x7f540815cda0 2024-03-12T17:07:45.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.893+0000 7f5405600640 1 -- 172.21.15.52:0/1512037948 >> v1:172.21.15.52:6789/0 conn(0x7f540815cda0 legacy=0x7f54080cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:45.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.893+0000 7f5405600640 1 --2- 172.21.15.52:0/1512037948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53ec003970 0x7f53ec003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:45.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.894+0000 7f5405600640 1 --2- 172.21.15.52:0/1512037948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ec0047c0 0x7f53ec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:45.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.894+0000 7f5405600640 1 --2- 172.21.15.52:0/1512037948 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f54080b5a30 0x7f53ec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:45.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.894+0000 7f5405600640 1 -- 172.21.15.52:0/1512037948 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54080b4d20 con 0x7f53ec003970 2024-03-12T17:07:45.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.894+0000 7f5405600640 1 -- 172.21.15.52:0/1512037948 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54080b1f20 con 0x7f53ec0047c0 2024-03-12T17:07:45.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.894+0000 7f5405600640 1 -- 172.21.15.52:0/1512037948 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec00ba30 con 0x7f54080b5a30 2024-03-12T17:07:45.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.894+0000 7f5407400640 1 --2- 172.21.15.52:0/1512037948 >> v2:172.21.15.102:3300/0 conn(0x7f54080b4350 0x7f54080b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:07:45.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.894+0000 7f5406a00640 1 --2- 172.21.15.52:0/1512037948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53ec003970 0x7f53ec003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:45.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.894+0000 7f5406000640 1 --2- 172.21.15.52:0/1512037948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ec0047c0 0x7f53ec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:45.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.894+0000 7f5407400640 1 --2- 172.21.15.52:0/1512037948 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f54080b5a30 0x7f53ec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:45.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.894+0000 7f5406a00640 1 -- 172.21.15.52:0/1512037948 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f54080b5a30 msgr2=0x7f53ec005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:45.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.894+0000 7f5406a00640 1 --2- 172.21.15.52:0/1512037948 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f54080b5a30 0x7f53ec005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:45.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.894+0000 7f5406a00640 1 -- 172.21.15.52:0/1512037948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ec0047c0 msgr2=0x7f53ec004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:45.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.894+0000 7f5406a00640 1 --2- 172.21.15.52:0/1512037948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ec0047c0 0x7f53ec004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:45.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.894+0000 7f5406a00640 1 -- 172.21.15.52:0/1512037948 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ec003520 con 0x7f53ec003970 2024-03-12T17:07:45.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.894+0000 7f5406000640 1 --2- 172.21.15.52:0/1512037948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ec0047c0 0x7f53ec004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:45.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.895+0000 7f5406a00640 1 --2- 172.21.15.52:0/1512037948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53ec003970 0x7f53ec003d90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f53f000c210 tx=0x7f53ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b956d0e1017747 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:45.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.896+0000 7f5405600640 1 -- 172.21.15.52:0/1512037948 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f0005ad0 con 0x7f53ec003970 2024-03-12T17:07:45.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.896+0000 7f5405600640 1 -- 172.21.15.52:0/1512037948 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f0005c30 con 0x7f53ec003970 2024-03-12T17:07:45.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.896+0000 7f5405600640 1 -- 172.21.15.52:0/1512037948 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f002c8a0 con 0x7f53ec003970 2024-03-12T17:07:45.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.896+0000 7f540d600640 1 -- 172.21.15.52:0/1512037948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53ec003970 msgr2=0x7f53ec003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:45.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.896+0000 7f540d600640 1 --2- 172.21.15.52:0/1512037948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53ec003970 0x7f53ec003d90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f53f000c210 tx=0x7f53ec0057f0 comp rx=0 tx=0).stop 2024-03-12T17:07:45.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.896+0000 7f5407400640 1 -- 172.21.15.52:0/1512037948 reap_dead start 2024-03-12T17:07:45.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.896+0000 7f540d600640 1 -- 172.21.15.52:0/1512037948 shutdown_connections 2024-03-12T17:07:45.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.896+0000 7f540d600640 1 -- 172.21.15.52:0/1512037948 >> 172.21.15.52:0/1512037948 conn(0x7f54080cf350 msgr2=0x7f54080cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:45.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.897+0000 7f540d600640 1 -- 172.21.15.52:0/1512037948 shutdown_connections 2024-03-12T17:07:45.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.897+0000 7f540d600640 1 -- 172.21.15.52:0/1512037948 wait complete. 2024-03-12T17:07:45.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.898+0000 7f540d600640 1 Processor -- start 2024-03-12T17:07:45.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.898+0000 7f540d600640 1 -- start start 2024-03-12T17:07:45.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.899+0000 7f540d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ec003970 0x7f5408141950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:45.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.899+0000 7f540d600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54080b3480 0x7f54080b3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:45.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.899+0000 7f540d600640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5408146e60 0x7f54081472e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:45.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.899+0000 7f540d600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5408144980 con 0x7f54080b3480 2024-03-12T17:07:45.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.899+0000 7f540d600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5408154800 con 0x7f53ec003970 2024-03-12T17:07:45.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.899+0000 7f540d600640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54080b3e40 con 0x7f5408146e60 2024-03-12T17:07:45.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.900+0000 7f5406000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54080b3480 0x7f54080b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:45.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.900+0000 7f5406000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54080b3480 0x7f54080b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55644/0 (socket says 172.21.15.52:55644) 2024-03-12T17:07:45.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.900+0000 7f5406000640 1 -- 172.21.15.52:0/721608119 learned_addr learned my addr 172.21.15.52:0/721608119 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:45.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.900+0000 7f5406000640 1 -- 172.21.15.52:0/721608119 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5408146e60 msgr2=0x7f54081472e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:45.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.900+0000 7f5406000640 1 --2- 172.21.15.52:0/721608119 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5408146e60 0x7f54081472e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:45.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.900+0000 7f5406000640 1 -- 172.21.15.52:0/721608119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ec003970 msgr2=0x7f5408141950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:45.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.900+0000 7f5406000640 1 --2- 172.21.15.52:0/721608119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ec003970 0x7f5408141950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:45.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.900+0000 7f5406000640 1 -- 172.21.15.52:0/721608119 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5408147820 con 0x7f54080b3480 2024-03-12T17:07:45.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.900+0000 7f5406000640 1 --2- 172.21.15.52:0/721608119 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54080b3480 0x7f54080b3900 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f53f8009fd0 tx=0x7f53f800f5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:45.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.901+0000 7f53fea00640 1 -- 172.21.15.52:0/721608119 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f80145f0 con 0x7f54080b3480 2024-03-12T17:07:45.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.901+0000 7f53fea00640 1 -- 172.21.15.52:0/721608119 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f8014770 con 0x7f54080b3480 2024-03-12T17:07:45.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.901+0000 7f53fea00640 1 -- 172.21.15.52:0/721608119 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f8014b20 con 0x7f54080b3480 2024-03-12T17:07:45.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.901+0000 7f540d600640 1 -- 172.21.15.52:0/721608119 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5408153ea0 con 0x7f54080b3480 2024-03-12T17:07:45.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.901+0000 7f540d600640 1 -- 172.21.15.52:0/721608119 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5408154140 con 0x7f54080b3480 2024-03-12T17:07:45.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.902+0000 7f540d600640 1 -- 172.21.15.52:0/721608119 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5408154a70 con 0x7f54080b3480 2024-03-12T17:07:45.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.903+0000 7f53fea00640 1 -- 172.21.15.52:0/721608119 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f53f8014c80 con 0x7f54080b3480 2024-03-12T17:07:45.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.904+0000 7f53fea00640 1 --2- 172.21.15.52:0/721608119 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f53e406ac00 0x7f53e406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:45.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.904+0000 7f53fea00640 1 -- 172.21.15.52:0/721608119 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f53f808a4d0 con 0x7f54080b3480 2024-03-12T17:07:45.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.905+0000 7f5406a00640 1 --2- 172.21.15.52:0/721608119 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f53e406ac00 0x7f53e406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:45.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.906+0000 7f5406a00640 1 --2- 172.21.15.52:0/721608119 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f53e406ac00 0x7f53e406d0c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f53f0034000 tx=0x7f53f002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:45.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:45.910+0000 7f53fea00640 1 -- 172.21.15.52:0/721608119 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f53f80552e0 con 0x7f54080b3480 2024-03-12T17:07:46.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.130+0000 7f540d600640 1 -- 172.21.15.52:0/721608119 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5408073390 con 0x7f54080b3480 2024-03-12T17:07:46.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.131+0000 7f53fea00640 1 -- 172.21.15.52:0/721608119 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f53f8058900 con 0x7f54080b3480 2024-03-12T17:07:46.124 INFO:teuthology.orchestra.run.smithi052.stderr:listed 0 entries 2024-03-12T17:07:46.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.134+0000 7f540d600640 1 -- 172.21.15.52:0/721608119 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f53e406ac00 msgr2=0x7f53e406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:46.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.134+0000 7f540d600640 1 --2- 172.21.15.52:0/721608119 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f53e406ac00 0x7f53e406d0c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f53f0034000 tx=0x7f53f002e000 comp rx=0 tx=0).stop 2024-03-12T17:07:46.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.134+0000 7f540d600640 1 -- 172.21.15.52:0/721608119 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54080b3480 msgr2=0x7f54080b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:46.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.134+0000 7f540d600640 1 --2- 172.21.15.52:0/721608119 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54080b3480 0x7f54080b3900 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f53f8009fd0 tx=0x7f53f800f5e0 comp rx=0 tx=0).stop 2024-03-12T17:07:46.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.134+0000 7f540d600640 1 -- 172.21.15.52:0/721608119 shutdown_connections 2024-03-12T17:07:46.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.134+0000 7f540d600640 1 --2- 172.21.15.52:0/721608119 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f53e406ac00 0x7f53e406d0c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:46.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.134+0000 7f540d600640 1 --2- 172.21.15.52:0/721608119 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5408146e60 0x7f54081472e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:46.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.134+0000 7f540d600640 1 --2- 172.21.15.52:0/721608119 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54080b3480 0x7f54080b3900 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:46.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.134+0000 7f540d600640 1 --2- 172.21.15.52:0/721608119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ec003970 0x7f5408141950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:46.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.135+0000 7f540d600640 1 -- 172.21.15.52:0/721608119 >> 172.21.15.52:0/721608119 conn(0x7f54080cf350 msgr2=0x7f54080cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:46.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.135+0000 7f540d600640 1 -- 172.21.15.52:0/721608119 shutdown_connections 2024-03-12T17:07:46.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.135+0000 7f540d600640 1 -- 172.21.15.52:0/721608119 wait complete. 2024-03-12T17:07:46.149 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-03-12T17:07:46.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:46.149 DEBUG:teuthology.orchestra.run.smithi102:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-03-12T17:07:46.250 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi102.front.sepia.ceph.com 2024-03-12T17:07:46.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:07:46.251 DEBUG:teuthology.orchestra.run.smithi102:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-12T17:07:46.299 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-12T17:07:46.299 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T17:07:46.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.530+0000 7fdb4a800640 1 Processor -- start 2024-03-12T17:07:46.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.531+0000 7fdb4a800640 1 -- start start 2024-03-12T17:07:46.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.532+0000 7fdb4a800640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdb4415cda0 0x7fdb440cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:46.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.532+0000 7fdb4a800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdb440b4a60 0x7fdb440b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:46.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.532+0000 7fdb4a800640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb440d1830 con 0x7fdb440b5a30 2024-03-12T17:07:46.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.532+0000 7fdb4a800640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb440b1f20 con 0x7fdb4415cda0 2024-03-12T17:07:46.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.532+0000 7fdb4a800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb440d0c20 con 0x7fdb440b4a60 2024-03-12T17:07:46.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.532+0000 7fdb43400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdb4415cda0 0x7fdb440cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:46.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.533+0000 7fdb43400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdb4415cda0 0x7fdb440cdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55654/0 (socket says 172.21.15.52:55654) 2024-03-12T17:07:46.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.533+0000 7fdb43400640 1 -- 172.21.15.52:0/996416759 learned_addr learned my addr 172.21.15.52:0/996416759 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:46.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.533+0000 7fdb42a00640 1 --1- 172.21.15.52:0/996416759 >> v1:172.21.15.52:6789/0 conn(0x7fdb440b5a30 0x7fdb440b4350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:46390/0 (socket says 172.21.15.52:46390) 2024-03-12T17:07:46.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.533+0000 7fdb43e00640 1 --2- 172.21.15.52:0/996416759 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdb440b4a60 0x7fdb440b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:46.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.533+0000 7fdb43400640 1 -- 172.21.15.52:0/996416759 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdb440b4a60 msgr2=0x7fdb440b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:46.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.533+0000 7fdb43400640 1 --2- 172.21.15.52:0/996416759 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdb440b4a60 0x7fdb440b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:46.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.533+0000 7fdb43400640 1 -- 172.21.15.52:0/996416759 >> v1:172.21.15.52:6789/0 conn(0x7fdb440b5a30 legacy=0x7fdb440b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:46.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.533+0000 7fdb43400640 1 -- 172.21.15.52:0/996416759 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb440d0f00 con 0x7fdb4415cda0 2024-03-12T17:07:46.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.533+0000 7fdb43e00640 1 --2- 172.21.15.52:0/996416759 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdb440b4a60 0x7fdb440b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:46.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.534+0000 7fdb43400640 1 --2- 172.21.15.52:0/996416759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb4415cda0 0x7fdb440cdb40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdb34009e50 tx=0x7fdb3402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd8fb6d5d637120 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:46.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.534+0000 7fdb42000640 1 -- 172.21.15.52:0/996416759 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3402cb00 con 0x7fdb4415cda0 2024-03-12T17:07:46.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.534+0000 7fdb42000640 1 -- 172.21.15.52:0/996416759 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb3402cc60 con 0x7fdb4415cda0 2024-03-12T17:07:46.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.534+0000 7fdb42000640 1 -- 172.21.15.52:0/996416759 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb340343f0 con 0x7fdb4415cda0 2024-03-12T17:07:46.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.535+0000 7fdb4a800640 1 -- 172.21.15.52:0/996416759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb4415cda0 msgr2=0x7fdb440cdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:46.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.535+0000 7fdb4a800640 1 --2- 172.21.15.52:0/996416759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb4415cda0 0x7fdb440cdb40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdb34009e50 tx=0x7fdb3402bf60 comp rx=0 tx=0).stop 2024-03-12T17:07:46.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.535+0000 7fdb4a800640 1 -- 172.21.15.52:0/996416759 shutdown_connections 2024-03-12T17:07:46.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.535+0000 7fdb4a800640 1 --2- 172.21.15.52:0/996416759 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdb440b4a60 0x7fdb440b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:46.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.535+0000 7fdb4a800640 1 --2- 172.21.15.52:0/996416759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb4415cda0 0x7fdb440cdb40 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:46.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.535+0000 7fdb4a800640 1 -- 172.21.15.52:0/996416759 >> 172.21.15.52:0/996416759 conn(0x7fdb440cf350 msgr2=0x7fdb440cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:46.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.535+0000 7fdb4a800640 1 -- 172.21.15.52:0/996416759 shutdown_connections 2024-03-12T17:07:46.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.535+0000 7fdb4a800640 1 -- 172.21.15.52:0/996416759 wait complete. 2024-03-12T17:07:46.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.537+0000 7fdb4a800640 1 Processor -- start 2024-03-12T17:07:46.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.537+0000 7fdb4a800640 1 -- start start 2024-03-12T17:07:46.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.538+0000 7fdb4a800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdb440b4a60 0x7fdb441f2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:46.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.538+0000 7fdb4a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb440b5a30 0x7fdb440c8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:46.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.538+0000 7fdb4a800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb4415cda0 0x7fdb440b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:46.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.538+0000 7fdb4a800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4415df10 con 0x7fdb4415cda0 2024-03-12T17:07:46.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.538+0000 7fdb4a800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb440c9730 con 0x7fdb440b5a30 2024-03-12T17:07:46.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.538+0000 7fdb4a800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb440c8bd0 con 0x7fdb440b4a60 2024-03-12T17:07:46.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.538+0000 7fdb43e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb4415cda0 0x7fdb440b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:46.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.538+0000 7fdb43e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb4415cda0 0x7fdb440b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55662/0 (socket says 172.21.15.52:55662) 2024-03-12T17:07:46.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.538+0000 7fdb43e00640 1 -- 172.21.15.52:0/3154912157 learned_addr learned my addr 172.21.15.52:0/3154912157 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:46.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.539+0000 7fdb43e00640 1 -- 172.21.15.52:0/3154912157 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdb440b4a60 msgr2=0x7fdb441f2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:46.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.539+0000 7fdb43e00640 1 --2- 172.21.15.52:0/3154912157 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdb440b4a60 0x7fdb441f2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:46.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.539+0000 7fdb43e00640 1 -- 172.21.15.52:0/3154912157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb440b5a30 msgr2=0x7fdb440c8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:46.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.539+0000 7fdb43e00640 1 --2- 172.21.15.52:0/3154912157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb440b5a30 0x7fdb440c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:46.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.539+0000 7fdb43e00640 1 -- 172.21.15.52:0/3154912157 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb34009b30 con 0x7fdb4415cda0 2024-03-12T17:07:46.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.539+0000 7fdb43e00640 1 --2- 172.21.15.52:0/3154912157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb4415cda0 0x7fdb440b0ab0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00c7f0 tx=0x7fdb3c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:46.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.539+0000 7fdb3b400640 1 -- 172.21.15.52:0/3154912157 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c007820 con 0x7fdb4415cda0 2024-03-12T17:07:46.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.539+0000 7fdb3b400640 1 -- 172.21.15.52:0/3154912157 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb3c007980 con 0x7fdb4415cda0 2024-03-12T17:07:46.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.539+0000 7fdb3b400640 1 -- 172.21.15.52:0/3154912157 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c007d70 con 0x7fdb4415cda0 2024-03-12T17:07:46.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.539+0000 7fdb4a800640 1 -- 172.21.15.52:0/3154912157 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb440c8e90 con 0x7fdb4415cda0 2024-03-12T17:07:46.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.539+0000 7fdb4a800640 1 -- 172.21.15.52:0/3154912157 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb440c9450 con 0x7fdb4415cda0 2024-03-12T17:07:46.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.541+0000 7fdb4a800640 1 -- 172.21.15.52:0/3154912157 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb440ce080 con 0x7fdb4415cda0 2024-03-12T17:07:46.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.541+0000 7fdb3b400640 1 -- 172.21.15.52:0/3154912157 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fdb3c0113f0 con 0x7fdb4415cda0 2024-03-12T17:07:46.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.542+0000 7fdb3b400640 1 --2- 172.21.15.52:0/3154912157 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fdb2406ac30 0x7fdb2406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:46.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.543+0000 7fdb3b400640 1 -- 172.21.15.52:0/3154912157 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fdb3c089a20 con 0x7fdb4415cda0 2024-03-12T17:07:46.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.543+0000 7fdb43400640 1 --2- 172.21.15.52:0/3154912157 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fdb2406ac30 0x7fdb2406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:46.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.544+0000 7fdb43400640 1 --2- 172.21.15.52:0/3154912157 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fdb2406ac30 0x7fdb2406d0f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdb34035000 tx=0x7fdb3402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:46.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.547+0000 7fdb3b400640 1 -- 172.21.15.52:0/3154912157 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fdb440ce080 con 0x7fdb4415cda0 2024-03-12T17:07:46.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.787+0000 7fdb4a800640 1 -- 172.21.15.52:0/3154912157 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb440ce080 con 0x7fdb4415cda0 2024-03-12T17:07:46.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.789+0000 7fdb3b400640 1 -- 172.21.15.52:0/3154912157 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4733 (secure 0 0 0) 0x7fdb440ce080 con 0x7fdb4415cda0 2024-03-12T17:07:46.782 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:07:46.782 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2950099092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2950099092},{"type":"v1","addr":"172.21.15.52:6834","nonce":2950099092}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/846108993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":846108993},{"type":"v1","addr":"172.21.15.52:6835","nonce":846108993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6836/3002500679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3002500679},{"type":"v1","addr":"172.21.15.102:6836","nonce":3002500679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-12T17:06:25.905186+0000","modified":"2024-03-12T17:06:54.075752+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6837/3333451504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6835","nonce":3333451504},{"type":"v1","addr":"172.21.15.102:6837","nonce":3333451504}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T17:07:46.782 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 9 2024-03-12T17:07:46.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.791+0000 7fdb4a800640 1 -- 172.21.15.52:0/3154912157 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fdb2406ac30 msgr2=0x7fdb2406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:46.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.791+0000 7fdb4a800640 1 --2- 172.21.15.52:0/3154912157 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fdb2406ac30 0x7fdb2406d0f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdb34035000 tx=0x7fdb3402e000 comp rx=0 tx=0).stop 2024-03-12T17:07:46.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.791+0000 7fdb4a800640 1 -- 172.21.15.52:0/3154912157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb4415cda0 msgr2=0x7fdb440b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:46.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.791+0000 7fdb4a800640 1 --2- 172.21.15.52:0/3154912157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb4415cda0 0x7fdb440b0ab0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00c7f0 tx=0x7fdb3c00ccb0 comp rx=0 tx=0).stop 2024-03-12T17:07:46.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.792+0000 7fdb4a800640 1 -- 172.21.15.52:0/3154912157 shutdown_connections 2024-03-12T17:07:46.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.792+0000 7fdb4a800640 1 --2- 172.21.15.52:0/3154912157 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fdb2406ac30 0x7fdb2406d0f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:46.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.792+0000 7fdb4a800640 1 --2- 172.21.15.52:0/3154912157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb4415cda0 0x7fdb440b0ab0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:46.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.792+0000 7fdb4a800640 1 --2- 172.21.15.52:0/3154912157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb440b5a30 0x7fdb440c8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:46.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.792+0000 7fdb4a800640 1 --2- 172.21.15.52:0/3154912157 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdb440b4a60 0x7fdb441f2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:46.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.792+0000 7fdb4a800640 1 -- 172.21.15.52:0/3154912157 >> 172.21.15.52:0/3154912157 conn(0x7fdb440cf350 msgr2=0x7fdb440b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:46.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.792+0000 7fdb4a800640 1 -- 172.21.15.52:0/3154912157 shutdown_connections 2024-03-12T17:07:46.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:46.792+0000 7fdb4a800640 1 -- 172.21.15.52:0/3154912157 wait complete. 2024-03-12T17:07:46.804 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:07:47.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.033+0000 7f540d000640 1 Processor -- start 2024-03-12T17:07:47.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.033+0000 7f540d000640 1 -- start start 2024-03-12T17:07:47.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.035+0000 7f540d000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5408143930 0x7f54080cd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:47.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.035+0000 7f540d000640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f540815b2b0 con 0x7f54080cdc50 2024-03-12T17:07:47.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.035+0000 7f540d000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54080cc6b0 con 0x7f5408143460 2024-03-12T17:07:47.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.035+0000 7f540d000640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54080cc950 con 0x7f5408143930 2024-03-12T17:07:47.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.036+0000 7f5407e00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f54080cdc50 0x7f5408070020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:46402/0 (socket says 172.21.15.52:46402) 2024-03-12T17:07:47.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.036+0000 7f5407e00640 1 -- 172.21.15.52:0/2360494646 learned_addr learned my addr 172.21.15.52:0/2360494646 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:47.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.036+0000 7f5406a00640 1 --2- 172.21.15.52:0/2360494646 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5408143930 0x7f54080cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:47.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.036+0000 7f5406000640 1 -- 172.21.15.52:0/2360494646 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2268805279 0 0) 0x7f540815b2b0 con 0x7f54080cdc50 2024-03-12T17:07:47.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.036+0000 7f5406000640 1 -- 172.21.15.52:0/2360494646 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53ec003520 con 0x7f54080cdc50 2024-03-12T17:07:47.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.037+0000 7f5406000640 1 -- 172.21.15.52:0/2360494646 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1668321072 0 0) 0x7f53ec003520 con 0x7f54080cdc50 2024-03-12T17:07:47.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.037+0000 7f5406000640 1 -- 172.21.15.52:0/2360494646 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5408143930 msgr2=0x7f54080cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:47.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.037+0000 7f5406000640 1 --2- 172.21.15.52:0/2360494646 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5408143930 0x7f54080cd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:47.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.037+0000 7f5406000640 1 -- 172.21.15.52:0/2360494646 >> v1:172.21.15.102:6789/0 conn(0x7f5408143460 legacy=0x7f540815aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:47.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.037+0000 7f5406000640 1 -- 172.21.15.52:0/2360494646 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54080ccc30 con 0x7f54080cdc50 2024-03-12T17:07:47.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.037+0000 7f5406000640 1 -- 172.21.15.52:0/2360494646 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f54000019f0 con 0x7f54080cdc50 2024-03-12T17:07:47.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.037+0000 7f5406a00640 1 --2- 172.21.15.52:0/2360494646 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5408143930 0x7f54080cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T17:07:47.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.037+0000 7f5406000640 1 -- 172.21.15.52:0/2360494646 >> v1:172.21.15.52:6789/0 conn(0x7f54080cdc50 legacy=0x7f5408070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:47.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.037+0000 7f5406000640 1 --2- 172.21.15.52:0/2360494646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ec0039a0 0x7f53ec003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:47.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.037+0000 7f5406000640 1 --2- 172.21.15.52:0/2360494646 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5408143930 0x7f53ec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:47.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.037+0000 7f5406000640 1 --2- 172.21.15.52:0/2360494646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53ec004d00 0x7f53ec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:47.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.037+0000 7f5406000640 1 -- 172.21.15.52:0/2360494646 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54080cc950 con 0x7f53ec004d00 2024-03-12T17:07:47.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.037+0000 7f5406000640 1 -- 172.21.15.52:0/2360494646 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec00b980 con 0x7f53ec0039a0 2024-03-12T17:07:47.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.037+0000 7f5406000640 1 -- 172.21.15.52:0/2360494646 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec00baf0 con 0x7f5408143930 2024-03-12T17:07:47.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.037+0000 7f5407e00640 1 --2- 172.21.15.52:0/2360494646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53ec004d00 0x7f53ec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:47.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.038+0000 7f5407400640 1 --2- 172.21.15.52:0/2360494646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ec0039a0 0x7f53ec003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:47.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.038+0000 7f5406a00640 1 --2- 172.21.15.52:0/2360494646 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5408143930 0x7f53ec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:47.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.038+0000 7f5407e00640 1 -- 172.21.15.52:0/2360494646 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5408143930 msgr2=0x7f53ec0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:47.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.038+0000 7f5407e00640 1 --2- 172.21.15.52:0/2360494646 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5408143930 0x7f53ec0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:47.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.038+0000 7f5407e00640 1 -- 172.21.15.52:0/2360494646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ec0039a0 msgr2=0x7f53ec003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:47.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.038+0000 7f5407e00640 1 --2- 172.21.15.52:0/2360494646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ec0039a0 0x7f53ec003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:47.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.038+0000 7f5407e00640 1 -- 172.21.15.52:0/2360494646 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f4009400 con 0x7f53ec004d00 2024-03-12T17:07:47.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.039+0000 7f5407e00640 1 --2- 172.21.15.52:0/2360494646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53ec004d00 0x7f53ec005180 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f540000a7a0 tx=0x7f53ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9cb2a9a0bb6fc0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:47.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.039+0000 7f5406000640 1 -- 172.21.15.52:0/2360494646 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5400034630 con 0x7f53ec004d00 2024-03-12T17:07:47.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.039+0000 7f5406000640 1 -- 172.21.15.52:0/2360494646 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f540002f070 con 0x7f53ec004d00 2024-03-12T17:07:47.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.039+0000 7f5406000640 1 -- 172.21.15.52:0/2360494646 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54000349f0 con 0x7f53ec004d00 2024-03-12T17:07:47.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.040+0000 7f540d000640 1 -- 172.21.15.52:0/2360494646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53ec004d00 msgr2=0x7f53ec005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:47.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.040+0000 7f540d000640 1 --2- 172.21.15.52:0/2360494646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53ec004d00 0x7f53ec005180 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f540000a7a0 tx=0x7f53ec0057f0 comp rx=0 tx=0).stop 2024-03-12T17:07:47.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.040+0000 7f5407e00640 1 -- 172.21.15.52:0/2360494646 reap_dead start 2024-03-12T17:07:47.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.040+0000 7f540d000640 1 -- 172.21.15.52:0/2360494646 shutdown_connections 2024-03-12T17:07:47.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.040+0000 7f540d000640 1 -- 172.21.15.52:0/2360494646 >> 172.21.15.52:0/2360494646 conn(0x7f5408065df0 msgr2=0x7f5408069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:47.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.040+0000 7f540d000640 1 -- 172.21.15.52:0/2360494646 shutdown_connections 2024-03-12T17:07:47.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.040+0000 7f540d000640 1 -- 172.21.15.52:0/2360494646 wait complete. 2024-03-12T17:07:47.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.042+0000 7f540d000640 1 Processor -- start 2024-03-12T17:07:47.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.042+0000 7f540d000640 1 -- start start 2024-03-12T17:07:47.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.043+0000 7f540d000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ec004d00 0x7f540806c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:47.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.043+0000 7f540d000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f540815b720 0x7f54081f53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:47.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.043+0000 7f540d000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54081f5920 0x7f540813bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:47.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.043+0000 7f540d000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54081f6340 con 0x7f54081f5920 2024-03-12T17:07:47.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.043+0000 7f540d000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540815b300 con 0x7f53ec004d00 2024-03-12T17:07:47.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.043+0000 7f540d000640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f540815c8b0 con 0x7f540815b720 2024-03-12T17:07:47.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.043+0000 7f5407e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54081f5920 0x7f540813bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:47.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.043+0000 7f5407e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54081f5920 0x7f540813bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55678/0 (socket says 172.21.15.52:55678) 2024-03-12T17:07:47.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.043+0000 7f5407e00640 1 -- 172.21.15.52:0/4240051155 learned_addr learned my addr 172.21.15.52:0/4240051155 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:47.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.044+0000 7f5407400640 1 --2- 172.21.15.52:0/4240051155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ec004d00 0x7f540806c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:47.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.044+0000 7f5406a00640 1 --2- 172.21.15.52:0/4240051155 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f540815b720 0x7f54081f53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:47.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.044+0000 7f5407e00640 1 -- 172.21.15.52:0/4240051155 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f540815b720 msgr2=0x7f54081f53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:47.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.044+0000 7f5407e00640 1 --2- 172.21.15.52:0/4240051155 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f540815b720 0x7f54081f53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:47.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.044+0000 7f5407e00640 1 -- 172.21.15.52:0/4240051155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ec004d00 msgr2=0x7f540806c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:47.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.044+0000 7f5407e00640 1 --2- 172.21.15.52:0/4240051155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ec004d00 0x7f540806c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:47.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.044+0000 7f5407e00640 1 -- 172.21.15.52:0/4240051155 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54081f5dd0 con 0x7f54081f5920 2024-03-12T17:07:47.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.044+0000 7f5407400640 1 --2- 172.21.15.52:0/4240051155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ec004d00 0x7f540806c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:47.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.044+0000 7f5407e00640 1 --2- 172.21.15.52:0/4240051155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54081f5920 0x7f540813bba0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5400001c30 tx=0x7f54080a74a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:47.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.045+0000 7f53ff400640 1 -- 172.21.15.52:0/4240051155 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5400034840 con 0x7f54081f5920 2024-03-12T17:07:47.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.045+0000 7f53ff400640 1 -- 172.21.15.52:0/4240051155 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54000019e0 con 0x7f54081f5920 2024-03-12T17:07:47.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.045+0000 7f53ff400640 1 -- 172.21.15.52:0/4240051155 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5400034ba0 con 0x7f54081f5920 2024-03-12T17:07:47.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.045+0000 7f540d000640 1 -- 172.21.15.52:0/4240051155 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54081f6060 con 0x7f54081f5920 2024-03-12T17:07:47.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.045+0000 7f540d000640 1 -- 172.21.15.52:0/4240051155 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f540815af20 con 0x7f54081f5920 2024-03-12T17:07:47.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.046+0000 7f540d000640 1 -- 172.21.15.52:0/4240051155 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f540815aaa0 con 0x7f54081f5920 2024-03-12T17:07:47.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.047+0000 7f53ff400640 1 -- 172.21.15.52:0/4240051155 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5400031030 con 0x7f54081f5920 2024-03-12T17:07:47.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.048+0000 7f53ff400640 1 --2- 172.21.15.52:0/4240051155 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f53e806ac30 0x7f53e806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:47.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.048+0000 7f53ff400640 1 -- 172.21.15.52:0/4240051155 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f54000abab0 con 0x7f54081f5920 2024-03-12T17:07:47.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.049+0000 7f5407400640 1 --2- 172.21.15.52:0/4240051155 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f53e806ac30 0x7f53e806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:47.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.050+0000 7f5407400640 1 --2- 172.21.15.52:0/4240051155 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f53e806ac30 0x7f53e806d0f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f53f8001790 tx=0x7f53f8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:47.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.052+0000 7f53ff400640 1 -- 172.21.15.52:0/4240051155 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f540815aaa0 con 0x7f54081f5920 2024-03-12T17:07:47.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.252+0000 7f540d000640 1 -- 172.21.15.52:0/4240051155 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f540815aaa0 con 0x7f54081f5920 2024-03-12T17:07:47.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.255+0000 7f53ff400640 1 -- 172.21.15.52:0/4240051155 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18207 (secure 0 0 0) 0x7f540815aaa0 con 0x7f54081f5920 2024-03-12T17:07:47.249 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:07:47.249 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":27,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:07:12.969614+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T17:06:19.218543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T17:06:23.525474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T17:06:24.911574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-12T17:06:28.017172+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.733031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.775178+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.591522+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.820838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.829252+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.582481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.18","mappings":[{"from":3,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:07:47.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.258+0000 7f540d000640 1 -- 172.21.15.52:0/4240051155 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f53e806ac30 msgr2=0x7f53e806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:47.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.258+0000 7f540d000640 1 --2- 172.21.15.52:0/4240051155 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f53e806ac30 0x7f53e806d0f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f53f8001790 tx=0x7f53f8009040 comp rx=0 tx=0).stop 2024-03-12T17:07:47.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.258+0000 7f540d000640 1 -- 172.21.15.52:0/4240051155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54081f5920 msgr2=0x7f540813bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:47.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.258+0000 7f540d000640 1 --2- 172.21.15.52:0/4240051155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54081f5920 0x7f540813bba0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5400001c30 tx=0x7f54080a74a0 comp rx=0 tx=0).stop 2024-03-12T17:07:47.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.258+0000 7f540d000640 1 -- 172.21.15.52:0/4240051155 shutdown_connections 2024-03-12T17:07:47.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.258+0000 7f540d000640 1 --2- 172.21.15.52:0/4240051155 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f53e806ac30 0x7f53e806d0f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:47.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.258+0000 7f540d000640 1 --2- 172.21.15.52:0/4240051155 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f540815b720 0x7f54081f53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:47.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.258+0000 7f540d000640 1 --2- 172.21.15.52:0/4240051155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54081f5920 0x7f540813bba0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:47.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.258+0000 7f540d000640 1 --2- 172.21.15.52:0/4240051155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ec004d00 0x7f540806c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:47.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.258+0000 7f540d000640 1 -- 172.21.15.52:0/4240051155 >> 172.21.15.52:0/4240051155 conn(0x7f5408065df0 msgr2=0x7f540813e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:47.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.258+0000 7f540d000640 1 -- 172.21.15.52:0/4240051155 shutdown_connections 2024-03-12T17:07:47.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.258+0000 7f540d000640 1 -- 172.21.15.52:0/4240051155 wait complete. 2024-03-12T17:07:47.268 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T17:06:19.218543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-12T17:06:23.525474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-12T17:06:24.911574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-12T17:06:28.017172+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-12T17:07:47.268 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T17:07:47.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.485+0000 7efcf3200640 1 Processor -- start 2024-03-12T17:07:47.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.485+0000 7efcf3200640 1 -- start start 2024-03-12T17:07:47.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.487+0000 7efcf3200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7efcf40b8280 0x7efcf40cd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:47.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.487+0000 7efcf3200640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf40d0b00 con 0x7efcf40cd9d0 2024-03-12T17:07:47.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.487+0000 7efcf3200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf40b86a0 con 0x7efcf415c8d0 2024-03-12T17:07:47.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.487+0000 7efcf3200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcf40cdde0 con 0x7efcf40b8280 2024-03-12T17:07:47.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.487+0000 7efcf2800640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7efcf40cd9d0 0x7efcf40d1270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:46406/0 (socket says 172.21.15.52:46406) 2024-03-12T17:07:47.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.487+0000 7efcf2800640 1 -- 172.21.15.52:0/4126484025 learned_addr learned my addr 172.21.15.52:0/4126484025 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:47.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.487+0000 7efcf1400640 1 --2- 172.21.15.52:0/4126484025 >> v2:172.21.15.102:3300/0 conn(0x7efcf40b8280 0x7efcf40cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:47.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.487+0000 7efcf0a00640 1 -- 172.21.15.52:0/4126484025 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1385646731 0 0) 0x7efcf40d0b00 con 0x7efcf40cd9d0 2024-03-12T17:07:47.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.488+0000 7efcf0a00640 1 -- 172.21.15.52:0/4126484025 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcd8003520 con 0x7efcf40cd9d0 2024-03-12T17:07:47.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.488+0000 7efcf0a00640 1 -- 172.21.15.52:0/4126484025 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2804837057 0 0) 0x7efcd8003520 con 0x7efcf40cd9d0 2024-03-12T17:07:47.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.488+0000 7efcf0a00640 1 -- 172.21.15.52:0/4126484025 >> v2:172.21.15.102:3300/0 conn(0x7efcf40b8280 msgr2=0x7efcf40cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:47.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.488+0000 7efcf0a00640 1 --2- 172.21.15.52:0/4126484025 >> v2:172.21.15.102:3300/0 conn(0x7efcf40b8280 0x7efcf40cd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:47.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.488+0000 7efcf0a00640 1 -- 172.21.15.52:0/4126484025 >> v1:172.21.15.102:6789/0 conn(0x7efcf415c8d0 legacy=0x7efcf4160530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:47.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.488+0000 7efcf0a00640 1 -- 172.21.15.52:0/4126484025 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf4160cf0 con 0x7efcf40cd9d0 2024-03-12T17:07:47.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.488+0000 7efcf0a00640 1 -- 172.21.15.52:0/4126484025 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7efcec0019f0 con 0x7efcf40cd9d0 2024-03-12T17:07:47.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.488+0000 7efcf0a00640 1 -- 172.21.15.52:0/4126484025 >> v1:172.21.15.52:6789/0 conn(0x7efcf40cd9d0 legacy=0x7efcf40d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:47.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.488+0000 7efcf0a00640 1 --2- 172.21.15.52:0/4126484025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efcd8003970 0x7efcd8003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:47.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.488+0000 7efcf0a00640 1 --2- 172.21.15.52:0/4126484025 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efcd8004370 0x7efcd8004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:47.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.488+0000 7efcf0a00640 1 --2- 172.21.15.52:0/4126484025 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efcd8005040 0x7efcd800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:47.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.488+0000 7efcf0a00640 1 -- 172.21.15.52:0/4126484025 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf40cdde0 con 0x7efcd8003970 2024-03-12T17:07:47.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.488+0000 7efcf0a00640 1 -- 172.21.15.52:0/4126484025 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd800bfa0 con 0x7efcd8004370 2024-03-12T17:07:47.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.488+0000 7efcf0a00640 1 -- 172.21.15.52:0/4126484025 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd800c160 con 0x7efcd8005040 2024-03-12T17:07:47.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.488+0000 7efcf1e00640 1 --2- 172.21.15.52:0/4126484025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efcd8003970 0x7efcd8003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:47.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.489+0000 7efcf1400640 1 --2- 172.21.15.52:0/4126484025 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efcd8004370 0x7efcd8004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:47.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.489+0000 7efcf1e00640 1 -- 172.21.15.52:0/4126484025 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efcd8005040 msgr2=0x7efcd800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:47.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.489+0000 7efcf1e00640 1 --2- 172.21.15.52:0/4126484025 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efcd8005040 0x7efcd800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:47.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.489+0000 7efcf1e00640 1 -- 172.21.15.52:0/4126484025 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efcd8004370 msgr2=0x7efcd8004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:47.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.489+0000 7efcf1e00640 1 --2- 172.21.15.52:0/4126484025 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efcd8004370 0x7efcd8004840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:47.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.489+0000 7efcf1e00640 1 -- 172.21.15.52:0/4126484025 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd8003520 con 0x7efcd8003970 2024-03-12T17:07:47.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.489+0000 7efcf1400640 1 --2- 172.21.15.52:0/4126484025 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efcd8004370 0x7efcd8004840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:47.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.489+0000 7efcf2800640 1 -- 172.21.15.52:0/4126484025 reap_dead start 2024-03-12T17:07:47.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.490+0000 7efcf1e00640 1 --2- 172.21.15.52:0/4126484025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efcd8003970 0x7efcd8003d50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7efce4001860 tx=0x7efcd80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ecee24025957def server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:47.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.490+0000 7efcf0a00640 1 -- 172.21.15.52:0/4126484025 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce4033630 con 0x7efcd8003970 2024-03-12T17:07:47.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.490+0000 7efcf0a00640 1 -- 172.21.15.52:0/4126484025 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce40337b0 con 0x7efcd8003970 2024-03-12T17:07:47.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.490+0000 7efcf0a00640 1 -- 172.21.15.52:0/4126484025 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce4033b60 con 0x7efcd8003970 2024-03-12T17:07:47.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.491+0000 7efcf3200640 1 -- 172.21.15.52:0/4126484025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efcd8003970 msgr2=0x7efcd8003d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:47.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.491+0000 7efcf3200640 1 --2- 172.21.15.52:0/4126484025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efcd8003970 0x7efcd8003d50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7efce4001860 tx=0x7efcd80057f0 comp rx=0 tx=0).stop 2024-03-12T17:07:47.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.491+0000 7efcf3200640 1 -- 172.21.15.52:0/4126484025 shutdown_connections 2024-03-12T17:07:47.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.491+0000 7efcf3200640 1 --2- 172.21.15.52:0/4126484025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efcd8003970 0x7efcd8003d50 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:47.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.491+0000 7efcf3200640 1 -- 172.21.15.52:0/4126484025 >> 172.21.15.52:0/4126484025 conn(0x7efcf406ae90 msgr2=0x7efcf406b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:47.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.491+0000 7efcf3200640 1 -- 172.21.15.52:0/4126484025 shutdown_connections 2024-03-12T17:07:47.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.491+0000 7efcf3200640 1 -- 172.21.15.52:0/4126484025 wait complete. 2024-03-12T17:07:47.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.493+0000 7efcf3200640 1 Processor -- start 2024-03-12T17:07:47.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.493+0000 7efcf3200640 1 -- start start 2024-03-12T17:07:47.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.494+0000 7efcf3200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efcd8003970 0x7efcf414cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:47.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.494+0000 7efcf3200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efcf41f5aa0 0x7efcf41f5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:47.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.494+0000 7efcf3200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efcf4158630 0x7efcf4158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:47.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.494+0000 7efcf3200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf4159020 con 0x7efcf4158630 2024-03-12T17:07:47.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.494+0000 7efcf3200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf41f3670 con 0x7efcf41f5aa0 2024-03-12T17:07:47.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.494+0000 7efcf3200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf414d0b0 con 0x7efcd8003970 2024-03-12T17:07:47.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.494+0000 7efcf2800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efcf4158630 0x7efcf4158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:47.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.494+0000 7efcf2800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efcf4158630 0x7efcf4158ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55702/0 (socket says 172.21.15.52:55702) 2024-03-12T17:07:47.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.494+0000 7efcf2800640 1 -- 172.21.15.52:0/1951424223 learned_addr learned my addr 172.21.15.52:0/1951424223 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:47.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.495+0000 7efcf1e00640 1 --2- 172.21.15.52:0/1951424223 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efcd8003970 0x7efcf414cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:47.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.495+0000 7efcf1400640 1 --2- 172.21.15.52:0/1951424223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efcf41f5aa0 0x7efcf41f5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:47.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.495+0000 7efcf2800640 1 -- 172.21.15.52:0/1951424223 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efcd8003970 msgr2=0x7efcf414cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:47.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.495+0000 7efcf2800640 1 --2- 172.21.15.52:0/1951424223 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efcd8003970 0x7efcf414cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:47.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.495+0000 7efcf2800640 1 -- 172.21.15.52:0/1951424223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efcf41f5aa0 msgr2=0x7efcf41f5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:47.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.495+0000 7efcf2800640 1 --2- 172.21.15.52:0/1951424223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efcf41f5aa0 0x7efcf41f5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:47.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.495+0000 7efcf2800640 1 -- 172.21.15.52:0/1951424223 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf41f6540 con 0x7efcf4158630 2024-03-12T17:07:47.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.495+0000 7efcf2800640 1 --2- 172.21.15.52:0/1951424223 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efcf4158630 0x7efcf4158ae0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7efcf40d2670 tx=0x7efcec00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:47.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.496+0000 7efcea000640 1 -- 172.21.15.52:0/1951424223 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec004d60 con 0x7efcf4158630 2024-03-12T17:07:47.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.496+0000 7efcea000640 1 -- 172.21.15.52:0/1951424223 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcec00a610 con 0x7efcf4158630 2024-03-12T17:07:47.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.496+0000 7efcea000640 1 -- 172.21.15.52:0/1951424223 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec00a9c0 con 0x7efcf4158630 2024-03-12T17:07:47.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.496+0000 7efcf3200640 1 -- 172.21.15.52:0/1951424223 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcf41f6820 con 0x7efcf4158630 2024-03-12T17:07:47.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.496+0000 7efcf3200640 1 -- 172.21.15.52:0/1951424223 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcf41f3b60 con 0x7efcf4158630 2024-03-12T17:07:47.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.497+0000 7efcf3200640 1 -- 172.21.15.52:0/1951424223 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcf41f3d80 con 0x7efcf4158630 2024-03-12T17:07:47.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.498+0000 7efcea000640 1 -- 172.21.15.52:0/1951424223 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7efcec00ab20 con 0x7efcf4158630 2024-03-12T17:07:47.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.499+0000 7efcea000640 1 --2- 172.21.15.52:0/1951424223 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7efcd406ac00 0x7efcd406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:47.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.499+0000 7efcea000640 1 -- 172.21.15.52:0/1951424223 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7efcec088e70 con 0x7efcf4158630 2024-03-12T17:07:47.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.499+0000 7efcf1e00640 1 --2- 172.21.15.52:0/1951424223 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7efcd406ac00 0x7efcd406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:47.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.500+0000 7efcf1e00640 1 --2- 172.21.15.52:0/1951424223 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7efcd406ac00 0x7efcd406d0c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efce4036000 tx=0x7efce4031000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:47.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.503+0000 7efcea000640 1 -- 172.21.15.52:0/1951424223 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7efcec053bf0 con 0x7efcf4158630 2024-03-12T17:07:47.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.705+0000 7efcf3200640 1 -- 172.21.15.52:0/1951424223 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efcf404e030 con 0x7efcf4158630 2024-03-12T17:07:47.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.706+0000 7efcea000640 1 -- 172.21.15.52:0/1951424223 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7efcec01a020 con 0x7efcf4158630 2024-03-12T17:07:47.699 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-03-12T17:07:47.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.708+0000 7efcf3200640 1 -- 172.21.15.52:0/1951424223 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7efcd406ac00 msgr2=0x7efcd406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:47.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.708+0000 7efcf3200640 1 --2- 172.21.15.52:0/1951424223 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7efcd406ac00 0x7efcd406d0c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efce4036000 tx=0x7efce4031000 comp rx=0 tx=0).stop 2024-03-12T17:07:47.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.708+0000 7efcf3200640 1 -- 172.21.15.52:0/1951424223 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efcf4158630 msgr2=0x7efcf4158ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:47.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.708+0000 7efcf3200640 1 --2- 172.21.15.52:0/1951424223 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efcf4158630 0x7efcf4158ae0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7efcf40d2670 tx=0x7efcec00cb10 comp rx=0 tx=0).stop 2024-03-12T17:07:47.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.709+0000 7efcf3200640 1 -- 172.21.15.52:0/1951424223 shutdown_connections 2024-03-12T17:07:47.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.709+0000 7efcf3200640 1 --2- 172.21.15.52:0/1951424223 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7efcd406ac00 0x7efcd406d0c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:47.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.709+0000 7efcf3200640 1 --2- 172.21.15.52:0/1951424223 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efcf4158630 0x7efcf4158ae0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:47.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.709+0000 7efcf3200640 1 --2- 172.21.15.52:0/1951424223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efcf41f5aa0 0x7efcf41f5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:47.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.709+0000 7efcf3200640 1 --2- 172.21.15.52:0/1951424223 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efcd8003970 0x7efcf414cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:47.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.709+0000 7efcf3200640 1 -- 172.21.15.52:0/1951424223 >> 172.21.15.52:0/1951424223 conn(0x7efcf406ae90 msgr2=0x7efcf40cf070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:47.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.709+0000 7efcf3200640 1 -- 172.21.15.52:0/1951424223 shutdown_connections 2024-03-12T17:07:47.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.709+0000 7efcf3200640 1 -- 172.21.15.52:0/1951424223 wait complete. 2024-03-12T17:07:47.721 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-12T17:07:47.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.949+0000 7f5a75800640 1 Processor -- start 2024-03-12T17:07:47.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.949+0000 7f5a75800640 1 -- start start 2024-03-12T17:07:47.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.950+0000 7f5a75800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5a700add30 0x7f5a700ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:47.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.950+0000 7f5a75800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5a7006ce90 0x7f5a7006d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:47.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.951+0000 7f5a75800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a700cf250 con 0x7f5a700ce930 2024-03-12T17:07:47.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.951+0000 7f5a75800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a7006c760 con 0x7f5a7006ce90 2024-03-12T17:07:47.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.951+0000 7f5a75800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a700b42d0 con 0x7f5a700add30 2024-03-12T17:07:47.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.951+0000 7f5a6ea00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f5a700ce930 0x7f5a700cd3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.52:52634/0 (socket says 172.21.15.52:52634) 2024-03-12T17:07:47.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.951+0000 7f5a6ea00640 1 -- 172.21.15.52:0/3850111672 learned_addr learned my addr 172.21.15.52:0/3850111672 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:47.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.951+0000 7f5a6f400640 1 --2- 172.21.15.52:0/3850111672 >> v2:172.21.15.102:3300/0 conn(0x7f5a7006ce90 0x7f5a7006d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:47.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.951+0000 7f5a67e00640 1 --2- 172.21.15.52:0/3850111672 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5a700add30 0x7f5a700ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:47.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.952+0000 7f5a6e000640 1 -- 172.21.15.52:0/3850111672 <== mon.1 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1224298709 0 0) 0x7f5a700cf250 con 0x7f5a700ce930 2024-03-12T17:07:47.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.952+0000 7f5a6e000640 1 -- 172.21.15.52:0/3850111672 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a54003520 con 0x7f5a700ce930 2024-03-12T17:07:47.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.952+0000 7f5a6f400640 1 -- 172.21.15.52:0/3850111672 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5a700add30 msgr2=0x7f5a700ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:47.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.952+0000 7f5a6f400640 1 --2- 172.21.15.52:0/3850111672 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5a700add30 0x7f5a700ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:47.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.952+0000 7f5a6f400640 1 -- 172.21.15.52:0/3850111672 >> v1:172.21.15.102:6789/0 conn(0x7f5a700ce930 legacy=0x7f5a700cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:47.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.952+0000 7f5a6f400640 1 -- 172.21.15.52:0/3850111672 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a700cdba0 con 0x7f5a7006ce90 2024-03-12T17:07:47.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.953+0000 7f5a6f400640 1 --2- 172.21.15.52:0/3850111672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a7006ce90 0x7f5a7006d320 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009d90 tx=0x7f5a6802bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33c2a93992de0f83 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:47.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.954+0000 7f5a6e000640 1 -- 172.21.15.52:0/3850111672 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6802cc50 con 0x7f5a7006ce90 2024-03-12T17:07:47.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.954+0000 7f5a6e000640 1 -- 172.21.15.52:0/3850111672 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6802cdb0 con 0x7f5a7006ce90 2024-03-12T17:07:47.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.954+0000 7f5a6e000640 1 -- 172.21.15.52:0/3850111672 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a68034660 con 0x7f5a7006ce90 2024-03-12T17:07:47.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.954+0000 7f5a75800640 1 -- 172.21.15.52:0/3850111672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a7006ce90 msgr2=0x7f5a7006d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:47.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.954+0000 7f5a75800640 1 --2- 172.21.15.52:0/3850111672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a7006ce90 0x7f5a7006d320 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009d90 tx=0x7f5a6802bea0 comp rx=0 tx=0).stop 2024-03-12T17:07:47.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.955+0000 7f5a75800640 1 -- 172.21.15.52:0/3850111672 shutdown_connections 2024-03-12T17:07:47.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.955+0000 7f5a75800640 1 --2- 172.21.15.52:0/3850111672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a7006ce90 0x7f5a7006d320 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:47.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.955+0000 7f5a75800640 1 --2- 172.21.15.52:0/3850111672 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5a700add30 0x7f5a700ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:47.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.955+0000 7f5a75800640 1 -- 172.21.15.52:0/3850111672 >> 172.21.15.52:0/3850111672 conn(0x7f5a7006a0b0 msgr2=0x7f5a7006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:47.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.955+0000 7f5a75800640 1 -- 172.21.15.52:0/3850111672 shutdown_connections 2024-03-12T17:07:47.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.955+0000 7f5a75800640 1 -- 172.21.15.52:0/3850111672 wait complete. 2024-03-12T17:07:47.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.957+0000 7f5a75800640 1 Processor -- start 2024-03-12T17:07:47.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.957+0000 7f5a75800640 1 -- start start 2024-03-12T17:07:47.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.958+0000 7f5a75800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a7006ce90 0x7f5a700b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:47.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.958+0000 7f5a75800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a700add30 0x7f5a7015af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:47.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.958+0000 7f5a75800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5a700ce930 0x7f5a7015a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:47.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.958+0000 7f5a75800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a700b2f90 con 0x7f5a700add30 2024-03-12T17:07:47.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.958+0000 7f5a75800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a700b2380 con 0x7f5a7006ce90 2024-03-12T17:07:47.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.958+0000 7f5a75800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a700b2660 con 0x7f5a700ce930 2024-03-12T17:07:47.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.958+0000 7f5a67e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a700add30 0x7f5a7015af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:47.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.958+0000 7f5a67e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a700add30 0x7f5a7015af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55718/0 (socket says 172.21.15.52:55718) 2024-03-12T17:07:47.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.958+0000 7f5a67e00640 1 -- 172.21.15.52:0/2928256122 learned_addr learned my addr 172.21.15.52:0/2928256122 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:47.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.958+0000 7f5a6ea00640 1 --2- 172.21.15.52:0/2928256122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a7006ce90 0x7f5a700b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:47.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.958+0000 7f5a6f400640 1 --2- 172.21.15.52:0/2928256122 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5a700ce930 0x7f5a7015a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:47.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.959+0000 7f5a67e00640 1 -- 172.21.15.52:0/2928256122 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5a700ce930 msgr2=0x7f5a7015a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:47.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.959+0000 7f5a67e00640 1 --2- 172.21.15.52:0/2928256122 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5a700ce930 0x7f5a7015a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:47.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.959+0000 7f5a67e00640 1 -- 172.21.15.52:0/2928256122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a7006ce90 msgr2=0x7f5a700b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:47.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.959+0000 7f5a67e00640 1 --2- 172.21.15.52:0/2928256122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a7006ce90 0x7f5a700b1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:47.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.959+0000 7f5a67e00640 1 -- 172.21.15.52:0/2928256122 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a68009a70 con 0x7f5a700add30 2024-03-12T17:07:47.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.959+0000 7f5a67e00640 1 --2- 172.21.15.52:0/2928256122 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a700add30 0x7f5a7015af30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5a5800e9b0 tx=0x7f5a5800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:47.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.959+0000 7f5a66a00640 1 -- 172.21.15.52:0/2928256122 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a58009670 con 0x7f5a700add30 2024-03-12T17:07:47.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.959+0000 7f5a66a00640 1 -- 172.21.15.52:0/2928256122 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5800c9a0 con 0x7f5a700add30 2024-03-12T17:07:47.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.959+0000 7f5a75800640 1 -- 172.21.15.52:0/2928256122 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a700b2940 con 0x7f5a700add30 2024-03-12T17:07:47.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.959+0000 7f5a66a00640 1 -- 172.21.15.52:0/2928256122 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5800cd50 con 0x7f5a700add30 2024-03-12T17:07:47.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.959+0000 7f5a75800640 1 -- 172.21.15.52:0/2928256122 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a7015a570 con 0x7f5a700add30 2024-03-12T17:07:47.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.961+0000 7f5a75800640 1 -- 172.21.15.52:0/2928256122 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a700cf600 con 0x7f5a700add30 2024-03-12T17:07:47.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.961+0000 7f5a66a00640 1 -- 172.21.15.52:0/2928256122 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5a58013470 con 0x7f5a700add30 2024-03-12T17:07:47.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.962+0000 7f5a66a00640 1 --2- 172.21.15.52:0/2928256122 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f5a4c06ac00 0x7f5a4c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:47.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.962+0000 7f5a66a00640 1 -- 172.21.15.52:0/2928256122 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5a580899c0 con 0x7f5a700add30 2024-03-12T17:07:47.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.963+0000 7f5a6ea00640 1 --2- 172.21.15.52:0/2928256122 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f5a4c06ac00 0x7f5a4c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:47.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.964+0000 7f5a6ea00640 1 --2- 172.21.15.52:0/2928256122 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f5a4c06ac00 0x7f5a4c06d0c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5a600011e0 tx=0x7f5a60009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:47.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:47.969+0000 7f5a66a00640 1 -- 172.21.15.52:0/2928256122 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f5a580547d0 con 0x7f5a700add30 2024-03-12T17:07:48.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.159+0000 7f5a75800640 1 -- 172.21.15.52:0/2928256122 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5a70073390 con 0x7f5a700add30 2024-03-12T17:07:48.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.160+0000 7f5a66a00640 1 -- 172.21.15.52:0/2928256122 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5a58057df0 con 0x7f5a700add30 2024-03-12T17:07:48.153 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-12T17:07:48.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.163+0000 7f5a75800640 1 -- 172.21.15.52:0/2928256122 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f5a4c06ac00 msgr2=0x7f5a4c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:48.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.163+0000 7f5a75800640 1 --2- 172.21.15.52:0/2928256122 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f5a4c06ac00 0x7f5a4c06d0c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5a600011e0 tx=0x7f5a60009040 comp rx=0 tx=0).stop 2024-03-12T17:07:48.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.163+0000 7f5a75800640 1 -- 172.21.15.52:0/2928256122 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a700add30 msgr2=0x7f5a7015af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:48.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.163+0000 7f5a75800640 1 --2- 172.21.15.52:0/2928256122 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a700add30 0x7f5a7015af30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5a5800e9b0 tx=0x7f5a5800ee70 comp rx=0 tx=0).stop 2024-03-12T17:07:48.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.163+0000 7f5a75800640 1 -- 172.21.15.52:0/2928256122 shutdown_connections 2024-03-12T17:07:48.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.163+0000 7f5a75800640 1 --2- 172.21.15.52:0/2928256122 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f5a4c06ac00 0x7f5a4c06d0c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:48.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.163+0000 7f5a75800640 1 --2- 172.21.15.52:0/2928256122 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5a700ce930 0x7f5a7015a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:48.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.163+0000 7f5a75800640 1 --2- 172.21.15.52:0/2928256122 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a700add30 0x7f5a7015af30 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:48.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.163+0000 7f5a75800640 1 --2- 172.21.15.52:0/2928256122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a7006ce90 0x7f5a700b1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:48.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.163+0000 7f5a75800640 1 -- 172.21.15.52:0/2928256122 >> 172.21.15.52:0/2928256122 conn(0x7f5a7006a0b0 msgr2=0x7f5a70151070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:48.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.163+0000 7f5a75800640 1 -- 172.21.15.52:0/2928256122 shutdown_connections 2024-03-12T17:07:48.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.163+0000 7f5a75800640 1 -- 172.21.15.52:0/2928256122 wait complete. 2024-03-12T17:07:48.173 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-12T17:07:48.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.385+0000 7fba1c400640 1 Processor -- start 2024-03-12T17:07:48.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.386+0000 7fba1c400640 1 -- start start 2024-03-12T17:07:48.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.387+0000 7fba1c400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fba140cd740 0x7fba140b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:48.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.387+0000 7fba1c400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fba140d0d40 0x7fba140cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:48.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.387+0000 7fba1c400640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba140d20c0 con 0x7fba1406b0a0 2024-03-12T17:07:48.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.387+0000 7fba1c400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba140ce060 con 0x7fba140cd740 2024-03-12T17:07:48.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.387+0000 7fba1c400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba140d14b0 con 0x7fba140d0d40 2024-03-12T17:07:48.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.388+0000 7fba1a400640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fba1406b0a0 0x7fba1406b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:46412/0 (socket says 172.21.15.52:46412) 2024-03-12T17:07:48.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.388+0000 7fba1a400640 1 -- 172.21.15.52:0/777300152 learned_addr learned my addr 172.21.15.52:0/777300152 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:48.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.388+0000 7fba19a00640 1 --2- 172.21.15.52:0/777300152 >> v2:172.21.15.102:3300/0 conn(0x7fba140cd740 0x7fba140b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:48.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.388+0000 7fba19000640 1 --2- 172.21.15.52:0/777300152 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fba140d0d40 0x7fba140cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:48.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.388+0000 7fba13e00640 1 -- 172.21.15.52:0/777300152 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1227836627 0 0) 0x7fba140d20c0 con 0x7fba1406b0a0 2024-03-12T17:07:48.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.388+0000 7fba13e00640 1 -- 172.21.15.52:0/777300152 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9f8003520 con 0x7fba1406b0a0 2024-03-12T17:07:48.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.389+0000 7fba13e00640 1 -- 172.21.15.52:0/777300152 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 867857993 0 0) 0x7fb9f8003520 con 0x7fba1406b0a0 2024-03-12T17:07:48.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.389+0000 7fba19000640 1 -- 172.21.15.52:0/777300152 >> v2:172.21.15.102:3300/0 conn(0x7fba140cd740 msgr2=0x7fba140b84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:48.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.389+0000 7fba19000640 1 --2- 172.21.15.52:0/777300152 >> v2:172.21.15.102:3300/0 conn(0x7fba140cd740 0x7fba140b84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:48.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.389+0000 7fba19000640 1 -- 172.21.15.52:0/777300152 >> v1:172.21.15.52:6789/0 conn(0x7fba1406b0a0 legacy=0x7fba1406b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:48.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.389+0000 7fba19000640 1 -- 172.21.15.52:0/777300152 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba140d1790 con 0x7fba140d0d40 2024-03-12T17:07:48.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.389+0000 7fba19a00640 1 --2- 172.21.15.52:0/777300152 >> v2:172.21.15.102:3300/0 conn(0x7fba140cd740 0x7fba140b84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T17:07:48.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.390+0000 7fba19000640 1 --2- 172.21.15.52:0/777300152 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fba140d0d40 0x7fba140cdb20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fba000097a0 tx=0x7fba0002b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d466e5540d1bb0a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:48.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.391+0000 7fba13e00640 1 -- 172.21.15.52:0/777300152 <== mon.2 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0c001670 con 0x7fba140d0d40 2024-03-12T17:07:48.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.391+0000 7fba13e00640 1 -- 172.21.15.52:0/777300152 <== mon.2 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0002e050 con 0x7fba140d0d40 2024-03-12T17:07:48.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.391+0000 7fba13e00640 1 -- 172.21.15.52:0/777300152 <== mon.2 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba00030730 con 0x7fba140d0d40 2024-03-12T17:07:48.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.391+0000 7fba1c400640 1 -- 172.21.15.52:0/777300152 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fba140d0d40 msgr2=0x7fba140cdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:48.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.391+0000 7fba1c400640 1 --2- 172.21.15.52:0/777300152 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fba140d0d40 0x7fba140cdb20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fba000097a0 tx=0x7fba0002b790 comp rx=0 tx=0).stop 2024-03-12T17:07:48.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.391+0000 7fba1c400640 1 -- 172.21.15.52:0/777300152 shutdown_connections 2024-03-12T17:07:48.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.391+0000 7fba1c400640 1 --2- 172.21.15.52:0/777300152 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fba140d0d40 0x7fba140cdb20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:48.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.391+0000 7fba1c400640 1 --2- 172.21.15.52:0/777300152 >> v2:172.21.15.102:3300/0 conn(0x7fba140cd740 0x7fba140b84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:48.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.391+0000 7fba1c400640 1 -- 172.21.15.52:0/777300152 >> 172.21.15.52:0/777300152 conn(0x7fba140cf5d0 msgr2=0x7fba140cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:48.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.391+0000 7fba1c400640 1 -- 172.21.15.52:0/777300152 shutdown_connections 2024-03-12T17:07:48.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.391+0000 7fba1c400640 1 -- 172.21.15.52:0/777300152 wait complete. 2024-03-12T17:07:48.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.393+0000 7fba1c400640 1 Processor -- start 2024-03-12T17:07:48.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.393+0000 7fba1c400640 1 -- start start 2024-03-12T17:07:48.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.394+0000 7fba1c400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fba1406b0a0 0x7fba1415a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:48.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.394+0000 7fba1c400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba140cd740 0x7fba140ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:48.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.394+0000 7fba1c400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba140d0d40 0x7fba1414ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:48.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.394+0000 7fba1c400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1415ad90 con 0x7fba140d0d40 2024-03-12T17:07:48.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.394+0000 7fba1c400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba140ca990 con 0x7fba140cd740 2024-03-12T17:07:48.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.394+0000 7fba1c400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba140cb4b0 con 0x7fba1406b0a0 2024-03-12T17:07:48.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.395+0000 7fba1a400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba140d0d40 0x7fba1414ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:48.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.395+0000 7fba1a400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba140d0d40 0x7fba1414ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55722/0 (socket says 172.21.15.52:55722) 2024-03-12T17:07:48.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.395+0000 7fba1a400640 1 -- 172.21.15.52:0/3340549867 learned_addr learned my addr 172.21.15.52:0/3340549867 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:48.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.395+0000 7fba19000640 1 --2- 172.21.15.52:0/3340549867 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba140cd740 0x7fba140ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:48.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.395+0000 7fba19a00640 1 --2- 172.21.15.52:0/3340549867 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fba1406b0a0 0x7fba1415a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:48.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.395+0000 7fba1a400640 1 -- 172.21.15.52:0/3340549867 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fba1406b0a0 msgr2=0x7fba1415a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:48.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.395+0000 7fba1a400640 1 --2- 172.21.15.52:0/3340549867 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fba1406b0a0 0x7fba1415a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:48.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.395+0000 7fba1a400640 1 -- 172.21.15.52:0/3340549867 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba140cd740 msgr2=0x7fba140ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:48.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.395+0000 7fba1a400640 1 --2- 172.21.15.52:0/3340549867 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba140cd740 0x7fba140ca450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:48.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.395+0000 7fba1a400640 1 -- 172.21.15.52:0/3340549867 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba04009a70 con 0x7fba140d0d40 2024-03-12T17:07:48.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.395+0000 7fba1a400640 1 --2- 172.21.15.52:0/3340549867 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba140d0d40 0x7fba1414ab60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fba0c00c700 tx=0x7fba0c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:48.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.396+0000 7fba11600640 1 -- 172.21.15.52:0/3340549867 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0c005ad0 con 0x7fba140d0d40 2024-03-12T17:07:48.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.396+0000 7fba11600640 1 -- 172.21.15.52:0/3340549867 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0c005c30 con 0x7fba140d0d40 2024-03-12T17:07:48.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.396+0000 7fba11600640 1 -- 172.21.15.52:0/3340549867 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0c007070 con 0x7fba140d0d40 2024-03-12T17:07:48.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.396+0000 7fba1c400640 1 -- 172.21.15.52:0/3340549867 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba00009400 con 0x7fba140d0d40 2024-03-12T17:07:48.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.396+0000 7fba1c400640 1 -- 172.21.15.52:0/3340549867 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba140caef0 con 0x7fba140d0d40 2024-03-12T17:07:48.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.397+0000 7fba1c400640 1 -- 172.21.15.52:0/3340549867 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba14073390 con 0x7fba140d0d40 2024-03-12T17:07:48.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.398+0000 7fba11600640 1 -- 172.21.15.52:0/3340549867 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fba0c0071d0 con 0x7fba140d0d40 2024-03-12T17:07:48.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.399+0000 7fba11600640 1 --2- 172.21.15.52:0/3340549867 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb9f406ac00 0x7fb9f406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:48.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.399+0000 7fba11600640 1 -- 172.21.15.52:0/3340549867 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fba0c089580 con 0x7fba140d0d40 2024-03-12T17:07:48.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.400+0000 7fba19a00640 1 --2- 172.21.15.52:0/3340549867 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb9f406ac00 0x7fb9f406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:48.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.401+0000 7fba19a00640 1 --2- 172.21.15.52:0/3340549867 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb9f406ac00 0x7fb9f406d0c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fba04007600 tx=0x7fba0400b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:48.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.403+0000 7fba11600640 1 -- 172.21.15.52:0/3340549867 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fba0c054310 con 0x7fba140d0d40 2024-03-12T17:07:48.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.602+0000 7fba1c400640 1 -- 172.21.15.52:0/3340549867 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fba140b20b0 con 0x7fba140d0d40 2024-03-12T17:07:48.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.603+0000 7fba11600640 1 -- 172.21.15.52:0/3340549867 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fba0c057930 con 0x7fba140d0d40 2024-03-12T17:07:48.596 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-03-12T17:07:48.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.605+0000 7fba1c400640 1 -- 172.21.15.52:0/3340549867 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb9f406ac00 msgr2=0x7fb9f406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:48.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.605+0000 7fba1c400640 1 --2- 172.21.15.52:0/3340549867 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb9f406ac00 0x7fb9f406d0c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fba04007600 tx=0x7fba0400b040 comp rx=0 tx=0).stop 2024-03-12T17:07:48.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.605+0000 7fba1c400640 1 -- 172.21.15.52:0/3340549867 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba140d0d40 msgr2=0x7fba1414ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:48.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.605+0000 7fba1c400640 1 --2- 172.21.15.52:0/3340549867 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba140d0d40 0x7fba1414ab60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fba0c00c700 tx=0x7fba0c00cbc0 comp rx=0 tx=0).stop 2024-03-12T17:07:48.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.606+0000 7fba1c400640 1 -- 172.21.15.52:0/3340549867 shutdown_connections 2024-03-12T17:07:48.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.606+0000 7fba1c400640 1 --2- 172.21.15.52:0/3340549867 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb9f406ac00 0x7fb9f406d0c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:48.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.606+0000 7fba1c400640 1 --2- 172.21.15.52:0/3340549867 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba140d0d40 0x7fba1414ab60 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:48.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.606+0000 7fba1c400640 1 --2- 172.21.15.52:0/3340549867 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba140cd740 0x7fba140ca450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:48.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.606+0000 7fba1c400640 1 --2- 172.21.15.52:0/3340549867 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fba1406b0a0 0x7fba1415a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:48.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.606+0000 7fba1c400640 1 -- 172.21.15.52:0/3340549867 >> 172.21.15.52:0/3340549867 conn(0x7fba140cf5d0 msgr2=0x7fba140d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:48.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.606+0000 7fba1c400640 1 -- 172.21.15.52:0/3340549867 shutdown_connections 2024-03-12T17:07:48.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.606+0000 7fba1c400640 1 -- 172.21.15.52:0/3340549867 wait complete. 2024-03-12T17:07:48.618 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-12T17:07:48.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.841+0000 7f4a7ec00640 1 Processor -- start 2024-03-12T17:07:48.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.841+0000 7f4a7ec00640 1 -- start start 2024-03-12T17:07:48.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.842+0000 7f4a7ec00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4a780cd740 0x7f4a780b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:48.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.842+0000 7f4a7ec00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a780d0d40 0x7f4a780cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:48.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.842+0000 7f4a7ec00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4a7806b0a0 0x7f4a7806b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:48.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.842+0000 7f4a7ec00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a780d20c0 con 0x7f4a7806b0a0 2024-03-12T17:07:48.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.842+0000 7f4a7ec00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a780ce060 con 0x7f4a780cd740 2024-03-12T17:07:48.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.842+0000 7f4a7ec00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a780d14b0 con 0x7f4a780d0d40 2024-03-12T17:07:48.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.842+0000 7f4a7cc00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4a7806b0a0 0x7f4a7806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:48.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.842+0000 7f4a7cc00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4a7806b0a0 0x7f4a7806b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55730/0 (socket says 172.21.15.52:55730) 2024-03-12T17:07:48.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.842+0000 7f4a7cc00640 1 -- 172.21.15.52:0/2223233035 learned_addr learned my addr 172.21.15.52:0/2223233035 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:48.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.843+0000 7f4a77400640 1 --2- 172.21.15.52:0/2223233035 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a780d0d40 0x7f4a780cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:48.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.843+0000 7f4a77e00640 1 --2- 172.21.15.52:0/2223233035 >> v2:172.21.15.102:3300/0 conn(0x7f4a780cd740 0x7f4a780b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:48.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.843+0000 7f4a7cc00640 1 -- 172.21.15.52:0/2223233035 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a780d0d40 msgr2=0x7f4a780cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:48.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.843+0000 7f4a7cc00640 1 --2- 172.21.15.52:0/2223233035 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a780d0d40 0x7f4a780cdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:48.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.843+0000 7f4a7cc00640 1 -- 172.21.15.52:0/2223233035 >> v2:172.21.15.102:3300/0 conn(0x7f4a780cd740 msgr2=0x7f4a780b84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:48.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.843+0000 7f4a7cc00640 1 --2- 172.21.15.52:0/2223233035 >> v2:172.21.15.102:3300/0 conn(0x7f4a780cd740 0x7f4a780b84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:48.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.843+0000 7f4a7cc00640 1 -- 172.21.15.52:0/2223233035 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a780d1790 con 0x7f4a7806b0a0 2024-03-12T17:07:48.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.843+0000 7f4a77e00640 1 --2- 172.21.15.52:0/2223233035 >> v2:172.21.15.102:3300/0 conn(0x7f4a780cd740 0x7f4a780b84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:48.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.844+0000 7f4a7cc00640 1 --2- 172.21.15.52:0/2223233035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a7806b0a0 0x7f4a7806b530 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4a7808a100 tx=0x7f4a7002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a40396913fc43f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:48.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.845+0000 7f4a76a00640 1 -- 172.21.15.52:0/2223233035 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7002ca70 con 0x7f4a7806b0a0 2024-03-12T17:07:48.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.845+0000 7f4a76a00640 1 -- 172.21.15.52:0/2223233035 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a7002cbd0 con 0x7f4a7806b0a0 2024-03-12T17:07:48.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.845+0000 7f4a76a00640 1 -- 172.21.15.52:0/2223233035 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a700343f0 con 0x7f4a7806b0a0 2024-03-12T17:07:48.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.845+0000 7f4a7ec00640 1 -- 172.21.15.52:0/2223233035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a7806b0a0 msgr2=0x7f4a7806b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:48.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.845+0000 7f4a7ec00640 1 --2- 172.21.15.52:0/2223233035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a7806b0a0 0x7f4a7806b530 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4a7808a100 tx=0x7f4a7002bed0 comp rx=0 tx=0).stop 2024-03-12T17:07:48.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.845+0000 7f4a7ec00640 1 -- 172.21.15.52:0/2223233035 shutdown_connections 2024-03-12T17:07:48.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.845+0000 7f4a7ec00640 1 --2- 172.21.15.52:0/2223233035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a7806b0a0 0x7f4a7806b530 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:48.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.845+0000 7f4a7ec00640 1 --2- 172.21.15.52:0/2223233035 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a780d0d40 0x7f4a780cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:48.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.845+0000 7f4a7ec00640 1 --2- 172.21.15.52:0/2223233035 >> v2:172.21.15.102:3300/0 conn(0x7f4a780cd740 0x7f4a780b84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:48.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.845+0000 7f4a7ec00640 1 -- 172.21.15.52:0/2223233035 >> 172.21.15.52:0/2223233035 conn(0x7f4a780cf5d0 msgr2=0x7f4a780cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:48.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.846+0000 7f4a7ec00640 1 -- 172.21.15.52:0/2223233035 shutdown_connections 2024-03-12T17:07:48.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.846+0000 7f4a7ec00640 1 -- 172.21.15.52:0/2223233035 wait complete. 2024-03-12T17:07:48.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.847+0000 7f4a7ec00640 1 Processor -- start 2024-03-12T17:07:48.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.847+0000 7f4a7ec00640 1 -- start start 2024-03-12T17:07:48.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.848+0000 7f4a7ec00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a7806b0a0 0x7f4a78149f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:48.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.848+0000 7f4a7ec00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a780cd740 0x7f4a7814a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:48.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.848+0000 7f4a7ec00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a780d0d40 0x7f4a780aa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:48.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.848+0000 7f4a7ec00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7814b060 con 0x7f4a780cd740 2024-03-12T17:07:48.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.848+0000 7f4a7ec00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7814b340 con 0x7f4a780d0d40 2024-03-12T17:07:48.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.848+0000 7f4a7ec00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7814b620 con 0x7f4a7806b0a0 2024-03-12T17:07:48.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.849+0000 7f4a77400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a780cd740 0x7f4a7814a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:48.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.849+0000 7f4a77400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a780cd740 0x7f4a7814a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55742/0 (socket says 172.21.15.52:55742) 2024-03-12T17:07:48.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.849+0000 7f4a77400640 1 -- 172.21.15.52:0/1298528622 learned_addr learned my addr 172.21.15.52:0/1298528622 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:48.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.849+0000 7f4a77e00640 1 --2- 172.21.15.52:0/1298528622 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a7806b0a0 0x7f4a78149f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:48.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.849+0000 7f4a7cc00640 1 --2- 172.21.15.52:0/1298528622 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a780d0d40 0x7f4a780aa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:48.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.849+0000 7f4a77400640 1 -- 172.21.15.52:0/1298528622 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a7806b0a0 msgr2=0x7f4a78149f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:48.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.849+0000 7f4a77400640 1 --2- 172.21.15.52:0/1298528622 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a7806b0a0 0x7f4a78149f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:48.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.849+0000 7f4a77400640 1 -- 172.21.15.52:0/1298528622 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a780d0d40 msgr2=0x7f4a780aa2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:48.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.849+0000 7f4a77400640 1 --2- 172.21.15.52:0/1298528622 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a780d0d40 0x7f4a780aa2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:48.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.849+0000 7f4a77400640 1 -- 172.21.15.52:0/1298528622 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a70009b30 con 0x7f4a780cd740 2024-03-12T17:07:48.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.849+0000 7f4a7cc00640 1 --2- 172.21.15.52:0/1298528622 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a780d0d40 0x7f4a780aa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:48.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.849+0000 7f4a77400640 1 --2- 172.21.15.52:0/1298528622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a780cd740 0x7f4a7814a5a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000d5f0 tx=0x7f4a6000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:48.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.850+0000 7f4a6fe00640 1 -- 172.21.15.52:0/1298528622 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a600125f0 con 0x7f4a780cd740 2024-03-12T17:07:48.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.850+0000 7f4a6fe00640 1 -- 172.21.15.52:0/1298528622 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a60012750 con 0x7f4a780cd740 2024-03-12T17:07:48.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.850+0000 7f4a6fe00640 1 -- 172.21.15.52:0/1298528622 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a60012b00 con 0x7f4a780cd740 2024-03-12T17:07:48.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.850+0000 7f4a7ec00640 1 -- 172.21.15.52:0/1298528622 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a7814b900 con 0x7f4a780cd740 2024-03-12T17:07:48.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.850+0000 7f4a7ec00640 1 -- 172.21.15.52:0/1298528622 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a7815f9b0 con 0x7f4a780cd740 2024-03-12T17:07:48.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.851+0000 7f4a7ec00640 1 -- 172.21.15.52:0/1298528622 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a78073390 con 0x7f4a780cd740 2024-03-12T17:07:48.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.852+0000 7f4a6fe00640 1 -- 172.21.15.52:0/1298528622 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4a60012c60 con 0x7f4a780cd740 2024-03-12T17:07:48.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.853+0000 7f4a6fe00640 1 --2- 172.21.15.52:0/1298528622 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4a5406ac50 0x7f4a5406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:48.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.853+0000 7f4a6fe00640 1 -- 172.21.15.52:0/1298528622 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4a60089bf0 con 0x7f4a780cd740 2024-03-12T17:07:48.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.854+0000 7f4a77e00640 1 --2- 172.21.15.52:0/1298528622 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4a5406ac50 0x7f4a5406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:48.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.855+0000 7f4a77e00640 1 --2- 172.21.15.52:0/1298528622 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4a5406ac50 0x7f4a5406d110 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4a68006b10 tx=0x7f4a68008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:48.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:48.858+0000 7f4a6fe00640 1 -- 172.21.15.52:0/1298528622 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f4a600549f0 con 0x7f4a780cd740 2024-03-12T17:07:49.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.054+0000 7f4a7ec00640 1 -- 172.21.15.52:0/1298528622 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4a780b20b0 con 0x7f4a780cd740 2024-03-12T17:07:49.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.055+0000 7f4a6fe00640 1 -- 172.21.15.52:0/1298528622 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4a60058010 con 0x7f4a780cd740 2024-03-12T17:07:49.049 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-12T17:07:49.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.058+0000 7f4a7ec00640 1 -- 172.21.15.52:0/1298528622 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4a5406ac50 msgr2=0x7f4a5406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:49.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.058+0000 7f4a7ec00640 1 --2- 172.21.15.52:0/1298528622 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4a5406ac50 0x7f4a5406d110 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4a68006b10 tx=0x7f4a68008040 comp rx=0 tx=0).stop 2024-03-12T17:07:49.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.058+0000 7f4a7ec00640 1 -- 172.21.15.52:0/1298528622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a780cd740 msgr2=0x7f4a7814a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:49.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.058+0000 7f4a7ec00640 1 --2- 172.21.15.52:0/1298528622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a780cd740 0x7f4a7814a5a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000d5f0 tx=0x7f4a6000dab0 comp rx=0 tx=0).stop 2024-03-12T17:07:49.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.059+0000 7f4a7ec00640 1 -- 172.21.15.52:0/1298528622 shutdown_connections 2024-03-12T17:07:49.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.059+0000 7f4a7ec00640 1 --2- 172.21.15.52:0/1298528622 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4a5406ac50 0x7f4a5406d110 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:49.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.059+0000 7f4a7ec00640 1 --2- 172.21.15.52:0/1298528622 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a780d0d40 0x7f4a780aa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:49.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.059+0000 7f4a7ec00640 1 --2- 172.21.15.52:0/1298528622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a780cd740 0x7f4a7814a5a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:49.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.059+0000 7f4a7ec00640 1 --2- 172.21.15.52:0/1298528622 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a7806b0a0 0x7f4a78149f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:49.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.059+0000 7f4a7ec00640 1 -- 172.21.15.52:0/1298528622 >> 172.21.15.52:0/1298528622 conn(0x7f4a780cf5d0 msgr2=0x7f4a7815c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:49.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.059+0000 7f4a7ec00640 1 -- 172.21.15.52:0/1298528622 shutdown_connections 2024-03-12T17:07:49.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.059+0000 7f4a7ec00640 1 -- 172.21.15.52:0/1298528622 wait complete. 2024-03-12T17:07:49.070 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T17:07:49.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.303+0000 7f04a3e00640 1 Processor -- start 2024-03-12T17:07:49.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.303+0000 7f04a3e00640 1 -- start start 2024-03-12T17:07:49.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.305+0000 7f04a3e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f04a4156e30 0x7f04a406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:49.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.305+0000 7f04a3e00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04a40cc8c0 con 0x7f04a40cb540 2024-03-12T17:07:49.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.305+0000 7f04a3e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04a406d630 con 0x7f04a40b3310 2024-03-12T17:07:49.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.305+0000 7f04a3e00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04a415ce60 con 0x7f04a4156e30 2024-03-12T17:07:49.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.305+0000 7f04a3400640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f04a40cb540 0x7f04a40b1d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:46420/0 (socket says 172.21.15.52:46420) 2024-03-12T17:07:49.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.305+0000 7f04a3400640 1 -- 172.21.15.52:0/3887808148 learned_addr learned my addr 172.21.15.52:0/3887808148 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:49.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.305+0000 7f04a2a00640 1 --2- 172.21.15.52:0/3887808148 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f04a4156e30 0x7f04a406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:49.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.306+0000 7f04a1600640 1 -- 172.21.15.52:0/3887808148 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2300504192 0 0) 0x7f04a40cc8c0 con 0x7f04a40cb540 2024-03-12T17:07:49.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.306+0000 7f04a1600640 1 -- 172.21.15.52:0/3887808148 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0488003520 con 0x7f04a40cb540 2024-03-12T17:07:49.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.306+0000 7f04a1600640 1 -- 172.21.15.52:0/3887808148 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2543106682 0 0) 0x7f0488003520 con 0x7f04a40cb540 2024-03-12T17:07:49.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.306+0000 7f04a1600640 1 -- 172.21.15.52:0/3887808148 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f04a4156e30 msgr2=0x7f04a406d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:49.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.306+0000 7f04a1600640 1 --2- 172.21.15.52:0/3887808148 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f04a4156e30 0x7f04a406d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:49.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.306+0000 7f04a1600640 1 -- 172.21.15.52:0/3887808148 >> v1:172.21.15.102:6789/0 conn(0x7f04a40b3310 legacy=0x7f04a40cae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:49.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.306+0000 7f04a1600640 1 -- 172.21.15.52:0/3887808148 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a415d120 con 0x7f04a40cb540 2024-03-12T17:07:49.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.306+0000 7f04a1600640 1 -- 172.21.15.52:0/3887808148 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f049c0019f0 con 0x7f04a40cb540 2024-03-12T17:07:49.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.306+0000 7f04a1600640 1 -- 172.21.15.52:0/3887808148 >> v1:172.21.15.52:6789/0 conn(0x7f04a40cb540 legacy=0x7f04a40b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:49.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.306+0000 7f04a1600640 1 --2- 172.21.15.52:0/3887808148 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04880039a0 0x7f0488003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:49.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.306+0000 7f04a1600640 1 --2- 172.21.15.52:0/3887808148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04880047c0 0x7f0488004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:49.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.306+0000 7f04a1600640 1 --2- 172.21.15.52:0/3887808148 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f04a4156e30 0x7f0488005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:49.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.306+0000 7f04a1600640 1 -- 172.21.15.52:0/3887808148 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a415ce60 con 0x7f04880039a0 2024-03-12T17:07:49.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.306+0000 7f04a1600640 1 -- 172.21.15.52:0/3887808148 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048800ba30 con 0x7f04880047c0 2024-03-12T17:07:49.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.306+0000 7f04a1600640 1 -- 172.21.15.52:0/3887808148 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f048800bbf0 con 0x7f04a4156e30 2024-03-12T17:07:49.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.307+0000 7f04a2a00640 1 --2- 172.21.15.52:0/3887808148 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04880039a0 0x7f0488003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:49.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.307+0000 7f04a2a00640 1 -- 172.21.15.52:0/3887808148 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f04a4156e30 msgr2=0x7f0488005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:49.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.307+0000 7f04a2a00640 1 --2- 172.21.15.52:0/3887808148 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f04a4156e30 0x7f0488005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:49.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.307+0000 7f04a2a00640 1 -- 172.21.15.52:0/3887808148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04880047c0 msgr2=0x7f0488004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:49.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.307+0000 7f04a2a00640 1 --2- 172.21.15.52:0/3887808148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04880047c0 0x7f0488004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:49.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.307+0000 7f04a2a00640 1 -- 172.21.15.52:0/3887808148 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0488003520 con 0x7f04880039a0 2024-03-12T17:07:49.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.308+0000 7f04a2a00640 1 --2- 172.21.15.52:0/3887808148 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04880039a0 0x7f0488003dc0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0494009a90 tx=0x7f04880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa17f5968db571f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:49.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.309+0000 7f04a1600640 1 -- 172.21.15.52:0/3887808148 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049402c690 con 0x7f04880039a0 2024-03-12T17:07:49.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.309+0000 7f04a1600640 1 -- 172.21.15.52:0/3887808148 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f049402c7f0 con 0x7f04880039a0 2024-03-12T17:07:49.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.309+0000 7f04a1600640 1 -- 172.21.15.52:0/3887808148 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049402cb80 con 0x7f04880039a0 2024-03-12T17:07:49.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.309+0000 7f04a3e00640 1 -- 172.21.15.52:0/3887808148 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04880039a0 msgr2=0x7f0488003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:49.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.309+0000 7f04a3e00640 1 --2- 172.21.15.52:0/3887808148 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04880039a0 0x7f0488003dc0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0494009a90 tx=0x7f04880057f0 comp rx=0 tx=0).stop 2024-03-12T17:07:49.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.309+0000 7f04a3400640 1 -- 172.21.15.52:0/3887808148 reap_dead start 2024-03-12T17:07:49.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.309+0000 7f04a3e00640 1 -- 172.21.15.52:0/3887808148 shutdown_connections 2024-03-12T17:07:49.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.309+0000 7f04a3e00640 1 -- 172.21.15.52:0/3887808148 >> 172.21.15.52:0/3887808148 conn(0x7f04a4065df0 msgr2=0x7f04a4069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:49.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.309+0000 7f04a3e00640 1 -- 172.21.15.52:0/3887808148 shutdown_connections 2024-03-12T17:07:49.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.309+0000 7f04a3e00640 1 -- 172.21.15.52:0/3887808148 wait complete. 2024-03-12T17:07:49.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.311+0000 7f04a3e00640 1 Processor -- start 2024-03-12T17:07:49.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.311+0000 7f04a3e00640 1 -- start start 2024-03-12T17:07:49.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.312+0000 7f04a3e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04880039a0 0x7f04a414ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:49.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.312+0000 7f04a3e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f04a4150f20 0x7f04a4144bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:49.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.312+0000 7f04a3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04a415f050 0x7f04a415f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:49.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.312+0000 7f04a3e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a4151d30 con 0x7f04880039a0 2024-03-12T17:07:49.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.312+0000 7f04a3e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a415fa10 con 0x7f04a415f050 2024-03-12T17:07:49.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.312+0000 7f04a3e00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04a4151770 con 0x7f04a4150f20 2024-03-12T17:07:49.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.312+0000 7f04a2a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04880039a0 0x7f04a414ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:49.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.312+0000 7f04a2a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04880039a0 0x7f04a414ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55772/0 (socket says 172.21.15.52:55772) 2024-03-12T17:07:49.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.312+0000 7f04a2a00640 1 -- 172.21.15.52:0/4274258866 learned_addr learned my addr 172.21.15.52:0/4274258866 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:49.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.313+0000 7f04a2000640 1 --2- 172.21.15.52:0/4274258866 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f04a4150f20 0x7f04a4144bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:49.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.313+0000 7f04a3400640 1 --2- 172.21.15.52:0/4274258866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04a415f050 0x7f04a415f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:49.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.313+0000 7f04a2a00640 1 -- 172.21.15.52:0/4274258866 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f04a4150f20 msgr2=0x7f04a4144bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:49.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.313+0000 7f04a2a00640 1 --2- 172.21.15.52:0/4274258866 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f04a4150f20 0x7f04a4144bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:49.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.313+0000 7f04a2a00640 1 -- 172.21.15.52:0/4274258866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04a415f050 msgr2=0x7f04a415f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:49.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.313+0000 7f04a2a00640 1 --2- 172.21.15.52:0/4274258866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04a415f050 0x7f04a415f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:49.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.313+0000 7f04a2a00640 1 -- 172.21.15.52:0/4274258866 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a4151a50 con 0x7f04880039a0 2024-03-12T17:07:49.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.313+0000 7f04a2000640 1 --2- 172.21.15.52:0/4274258866 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f04a4150f20 0x7f04a4144bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:49.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.313+0000 7f04a3400640 1 --2- 172.21.15.52:0/4274258866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04a415f050 0x7f04a415f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:49.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.313+0000 7f04a2a00640 1 --2- 172.21.15.52:0/4274258866 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04880039a0 0x7f04a414ddb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f049402c230 tx=0x7f0494004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:49.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.313+0000 7f049b400640 1 -- 172.21.15.52:0/4274258866 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0494003810 con 0x7f04880039a0 2024-03-12T17:07:49.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.314+0000 7f049b400640 1 -- 172.21.15.52:0/4274258866 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0494003970 con 0x7f04880039a0 2024-03-12T17:07:49.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.314+0000 7f049b400640 1 -- 172.21.15.52:0/4274258866 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0494003d20 con 0x7f04880039a0 2024-03-12T17:07:49.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.314+0000 7f04a3e00640 1 -- 172.21.15.52:0/4274258866 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04a40af340 con 0x7f04880039a0 2024-03-12T17:07:49.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.314+0000 7f04a3e00640 1 -- 172.21.15.52:0/4274258866 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a41513a0 con 0x7f04880039a0 2024-03-12T17:07:49.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.315+0000 7f04a3e00640 1 -- 172.21.15.52:0/4274258866 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04a40af1a0 con 0x7f04880039a0 2024-03-12T17:07:49.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.316+0000 7f049b400640 1 -- 172.21.15.52:0/4274258866 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0494040450 con 0x7f04880039a0 2024-03-12T17:07:49.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.317+0000 7f049b400640 1 --2- 172.21.15.52:0/4274258866 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f048006ac00 0x7f048006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:49.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.317+0000 7f049b400640 1 -- 172.21.15.52:0/4274258866 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f04940ab5e0 con 0x7f04880039a0 2024-03-12T17:07:49.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.317+0000 7f04a2000640 1 --2- 172.21.15.52:0/4274258866 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f048006ac00 0x7f048006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:49.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.318+0000 7f04a2000640 1 --2- 172.21.15.52:0/4274258866 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f048006ac00 0x7f048006d0c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f04a40ca9b0 tx=0x7f048c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:49.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.321+0000 7f049b400640 1 -- 172.21.15.52:0/4274258866 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f04a40af1a0 con 0x7f04880039a0 2024-03-12T17:07:49.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.557+0000 7f04a3e00640 1 -- 172.21.15.52:0/4274258866 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04a40af1a0 con 0x7f04880039a0 2024-03-12T17:07:49.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.558+0000 7f049b400640 1 -- 172.21.15.52:0/4274258866 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4733 (secure 0 0 0) 0x7f04a40af1a0 con 0x7f04880039a0 2024-03-12T17:07:49.551 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:07:49.551 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2950099092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2950099092},{"type":"v1","addr":"172.21.15.52:6834","nonce":2950099092}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/846108993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":846108993},{"type":"v1","addr":"172.21.15.52:6835","nonce":846108993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6836/3002500679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3002500679},{"type":"v1","addr":"172.21.15.102:6836","nonce":3002500679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-12T17:06:25.905186+0000","modified":"2024-03-12T17:06:54.075752+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6837/3333451504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6835","nonce":3333451504},{"type":"v1","addr":"172.21.15.102:6837","nonce":3333451504}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T17:07:49.552 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 9 2024-03-12T17:07:49.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.561+0000 7f04a3e00640 1 -- 172.21.15.52:0/4274258866 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f048006ac00 msgr2=0x7f048006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:49.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.561+0000 7f04a3e00640 1 --2- 172.21.15.52:0/4274258866 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f048006ac00 0x7f048006d0c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f04a40ca9b0 tx=0x7f048c009040 comp rx=0 tx=0).stop 2024-03-12T17:07:49.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.561+0000 7f04a3e00640 1 -- 172.21.15.52:0/4274258866 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04880039a0 msgr2=0x7f04a414ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:49.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.561+0000 7f04a3e00640 1 --2- 172.21.15.52:0/4274258866 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04880039a0 0x7f04a414ddb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f049402c230 tx=0x7f0494004730 comp rx=0 tx=0).stop 2024-03-12T17:07:49.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.561+0000 7f04a3e00640 1 -- 172.21.15.52:0/4274258866 shutdown_connections 2024-03-12T17:07:49.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.561+0000 7f04a3e00640 1 --2- 172.21.15.52:0/4274258866 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f048006ac00 0x7f048006d0c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:49.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.561+0000 7f04a3e00640 1 --2- 172.21.15.52:0/4274258866 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f04a4150f20 0x7f04a4144bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:49.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.561+0000 7f04a3e00640 1 --2- 172.21.15.52:0/4274258866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04a415f050 0x7f04a415f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:49.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.561+0000 7f04a3e00640 1 --2- 172.21.15.52:0/4274258866 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04880039a0 0x7f04a414ddb0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:49.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.561+0000 7f04a3e00640 1 -- 172.21.15.52:0/4274258866 >> 172.21.15.52:0/4274258866 conn(0x7f04a4065df0 msgr2=0x7f04a406c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:49.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.561+0000 7f04a3e00640 1 -- 172.21.15.52:0/4274258866 shutdown_connections 2024-03-12T17:07:49.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.562+0000 7f04a3e00640 1 -- 172.21.15.52:0/4274258866 wait complete. 2024-03-12T17:07:49.576 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:07:49.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.806+0000 7f2105a00640 1 Processor -- start 2024-03-12T17:07:49.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.806+0000 7f2105a00640 1 -- start start 2024-03-12T17:07:49.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.808+0000 7f2105a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f21000cf5c0 0x7f21000b8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:49.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.808+0000 7f2105a00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f210015ce90 con 0x7f21000d0c20 2024-03-12T17:07:49.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.808+0000 7f2105a00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21000cddf0 con 0x7f210015cab0 2024-03-12T17:07:49.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.808+0000 7f2105a00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21000cd570 con 0x7f21000cf5c0 2024-03-12T17:07:49.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.808+0000 7f2105000640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f21000d0c20 0x7f21000d10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:46426/0 (socket says 172.21.15.52:46426) 2024-03-12T17:07:49.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.808+0000 7f2105000640 1 -- 172.21.15.52:0/1237890217 learned_addr learned my addr 172.21.15.52:0/1237890217 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:49.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.808+0000 7f20ff400640 1 --2- 172.21.15.52:0/1237890217 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f21000cf5c0 0x7f21000b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:49.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.809+0000 7f20fea00640 1 -- 172.21.15.52:0/1237890217 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3787995390 0 0) 0x7f210015ce90 con 0x7f21000d0c20 2024-03-12T17:07:49.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.809+0000 7f20fea00640 1 -- 172.21.15.52:0/1237890217 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20e4003520 con 0x7f21000d0c20 2024-03-12T17:07:49.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.809+0000 7f20fea00640 1 -- 172.21.15.52:0/1237890217 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1425051541 0 0) 0x7f20e4003520 con 0x7f21000d0c20 2024-03-12T17:07:49.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.809+0000 7f20fea00640 1 -- 172.21.15.52:0/1237890217 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f21000cf5c0 msgr2=0x7f21000b8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:49.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.809+0000 7f20fea00640 1 --2- 172.21.15.52:0/1237890217 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f21000cf5c0 0x7f21000b8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:49.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.809+0000 7f20fea00640 1 -- 172.21.15.52:0/1237890217 >> v1:172.21.15.102:6789/0 conn(0x7f210015cab0 legacy=0x7f21000ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:49.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.809+0000 7f20fea00640 1 -- 172.21.15.52:0/1237890217 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2100161370 con 0x7f21000d0c20 2024-03-12T17:07:49.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.809+0000 7f20fea00640 1 -- 172.21.15.52:0/1237890217 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f20f8001670 con 0x7f21000d0c20 2024-03-12T17:07:49.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.809+0000 7f20fea00640 1 -- 172.21.15.52:0/1237890217 >> v1:172.21.15.52:6789/0 conn(0x7f21000d0c20 legacy=0x7f21000d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:49.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.809+0000 7f20fea00640 1 --2- 172.21.15.52:0/1237890217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20e40039a0 0x7f20e4003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:49.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.809+0000 7f20fea00640 1 --2- 172.21.15.52:0/1237890217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f20e40047c0 0x7f20e4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:49.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.809+0000 7f20fea00640 1 --2- 172.21.15.52:0/1237890217 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f21000cf5c0 0x7f20e4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:49.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.809+0000 7f20fea00640 1 -- 172.21.15.52:0/1237890217 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21000cd570 con 0x7f20e40047c0 2024-03-12T17:07:49.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.809+0000 7f20fea00640 1 -- 172.21.15.52:0/1237890217 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e400ba30 con 0x7f20e40039a0 2024-03-12T17:07:49.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.809+0000 7f20fea00640 1 -- 172.21.15.52:0/1237890217 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e400bba0 con 0x7f21000cf5c0 2024-03-12T17:07:49.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.810+0000 7f20ff400640 1 --2- 172.21.15.52:0/1237890217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f20e40047c0 0x7f20e4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:49.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.810+0000 7f20ffe00640 1 --2- 172.21.15.52:0/1237890217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20e40039a0 0x7f20e4003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:49.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.810+0000 7f2105000640 1 --2- 172.21.15.52:0/1237890217 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f21000cf5c0 0x7f20e4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:49.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.810+0000 7f20ff400640 1 -- 172.21.15.52:0/1237890217 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f21000cf5c0 msgr2=0x7f20e4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:49.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.810+0000 7f20ff400640 1 --2- 172.21.15.52:0/1237890217 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f21000cf5c0 0x7f20e4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:49.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.810+0000 7f20ff400640 1 -- 172.21.15.52:0/1237890217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20e40039a0 msgr2=0x7f20e4003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:49.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.810+0000 7f20ff400640 1 --2- 172.21.15.52:0/1237890217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20e40039a0 0x7f20e4003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:49.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.810+0000 7f20ff400640 1 -- 172.21.15.52:0/1237890217 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e4003520 con 0x7f20e40047c0 2024-03-12T17:07:49.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.811+0000 7f20ff400640 1 --2- 172.21.15.52:0/1237890217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f20e40047c0 0x7f20e4004c40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f20e8009a90 tx=0x7f20e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354b18a4d5534f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:49.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.812+0000 7f20fea00640 1 -- 172.21.15.52:0/1237890217 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e802c650 con 0x7f20e40047c0 2024-03-12T17:07:49.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.812+0000 7f20fea00640 1 -- 172.21.15.52:0/1237890217 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e802c7b0 con 0x7f20e40047c0 2024-03-12T17:07:49.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.812+0000 7f20fea00640 1 -- 172.21.15.52:0/1237890217 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e802cb40 con 0x7f20e40047c0 2024-03-12T17:07:49.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.812+0000 7f2105a00640 1 -- 172.21.15.52:0/1237890217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f20e40047c0 msgr2=0x7f20e4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:49.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.812+0000 7f2105a00640 1 --2- 172.21.15.52:0/1237890217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f20e40047c0 0x7f20e4004c40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f20e8009a90 tx=0x7f20e40057f0 comp rx=0 tx=0).stop 2024-03-12T17:07:49.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.812+0000 7f2105000640 1 -- 172.21.15.52:0/1237890217 reap_dead start 2024-03-12T17:07:49.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.812+0000 7f2105a00640 1 -- 172.21.15.52:0/1237890217 shutdown_connections 2024-03-12T17:07:49.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.812+0000 7f2105a00640 1 -- 172.21.15.52:0/1237890217 >> 172.21.15.52:0/1237890217 conn(0x7f210006ae90 msgr2=0x7f210006b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:49.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.812+0000 7f2105a00640 1 -- 172.21.15.52:0/1237890217 shutdown_connections 2024-03-12T17:07:49.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.813+0000 7f2105a00640 1 -- 172.21.15.52:0/1237890217 wait complete. 2024-03-12T17:07:49.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.814+0000 7f2105a00640 1 Processor -- start 2024-03-12T17:07:49.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.814+0000 7f2105a00640 1 -- start start 2024-03-12T17:07:49.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.815+0000 7f2105a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20e40047c0 0x7f21000c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:49.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.815+0000 7f2105a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f21000b7290 0x7f21000b7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:49.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.815+0000 7f2105a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21000b2c80 0x7f21000b3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:49.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.815+0000 7f2105a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21000c91b0 con 0x7f21000b2c80 2024-03-12T17:07:49.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.815+0000 7f2105a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21000b7c50 con 0x7f20e40047c0 2024-03-12T17:07:49.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.815+0000 7f2105a00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21000b3a20 con 0x7f21000b7290 2024-03-12T17:07:49.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.816+0000 7f2105000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21000b2c80 0x7f21000b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:49.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.816+0000 7f2105000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21000b2c80 0x7f21000b3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55794/0 (socket says 172.21.15.52:55794) 2024-03-12T17:07:49.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.816+0000 7f2105000640 1 -- 172.21.15.52:0/1508202209 learned_addr learned my addr 172.21.15.52:0/1508202209 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:49.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.816+0000 7f20ff400640 1 --2- 172.21.15.52:0/1508202209 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f21000b7290 0x7f21000b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:49.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.816+0000 7f20ffe00640 1 --2- 172.21.15.52:0/1508202209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20e40047c0 0x7f21000c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:49.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.816+0000 7f2105000640 1 -- 172.21.15.52:0/1508202209 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f21000b7290 msgr2=0x7f21000b7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:49.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.816+0000 7f2105000640 1 --2- 172.21.15.52:0/1508202209 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f21000b7290 0x7f21000b7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:49.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.816+0000 7f2105000640 1 -- 172.21.15.52:0/1508202209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20e40047c0 msgr2=0x7f21000c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:49.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.816+0000 7f2105000640 1 --2- 172.21.15.52:0/1508202209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20e40047c0 0x7f21000c8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:49.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.816+0000 7f2105000640 1 -- 172.21.15.52:0/1508202209 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21000b3d00 con 0x7f21000b2c80 2024-03-12T17:07:49.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.816+0000 7f20ff400640 1 --2- 172.21.15.52:0/1508202209 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f21000b7290 0x7f21000b7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:07:49.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.817+0000 7f2105000640 1 --2- 172.21.15.52:0/1508202209 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21000b2c80 0x7f21000b3100 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f21000c61a0 tx=0x7f20f800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:49.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.817+0000 7f20f7e00640 1 -- 172.21.15.52:0/1508202209 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f800eed0 con 0x7f21000b2c80 2024-03-12T17:07:49.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.817+0000 7f20f7e00640 1 -- 172.21.15.52:0/1508202209 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f800c610 con 0x7f21000b2c80 2024-03-12T17:07:49.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.817+0000 7f20f7e00640 1 -- 172.21.15.52:0/1508202209 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f800c9a0 con 0x7f21000b2c80 2024-03-12T17:07:49.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.817+0000 7f2105a00640 1 -- 172.21.15.52:0/1508202209 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21000b3fe0 con 0x7f21000b2c80 2024-03-12T17:07:49.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.817+0000 7f2105a00640 1 -- 172.21.15.52:0/1508202209 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21000b42c0 con 0x7f21000b2c80 2024-03-12T17:07:49.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.818+0000 7f2105a00640 1 -- 172.21.15.52:0/1508202209 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21000c68b0 con 0x7f21000b2c80 2024-03-12T17:07:49.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.819+0000 7f20f7e00640 1 -- 172.21.15.52:0/1508202209 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f20f800cb00 con 0x7f21000b2c80 2024-03-12T17:07:49.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.821+0000 7f20f7e00640 1 --2- 172.21.15.52:0/1508202209 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f20dc06ac30 0x7f20dc06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:49.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.821+0000 7f20f7e00640 1 -- 172.21.15.52:0/1508202209 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f20f8089470 con 0x7f21000b2c80 2024-03-12T17:07:49.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.821+0000 7f20ffe00640 1 --2- 172.21.15.52:0/1508202209 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f20dc06ac30 0x7f20dc06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:49.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.822+0000 7f20ffe00640 1 --2- 172.21.15.52:0/1508202209 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f20dc06ac30 0x7f20dc06d0f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f20f0007fd0 tx=0x7f20f0003970 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:49.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:49.825+0000 7f20f7e00640 1 -- 172.21.15.52:0/1508202209 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f21000c68b0 con 0x7f21000b2c80 2024-03-12T17:07:50.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.038+0000 7f2105a00640 1 -- 172.21.15.52:0/1508202209 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21000c68b0 con 0x7f21000b2c80 2024-03-12T17:07:50.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.040+0000 7f20f7e00640 1 -- 172.21.15.52:0/1508202209 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18207 (secure 0 0 0) 0x7f21000c68b0 con 0x7f21000b2c80 2024-03-12T17:07:50.033 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:07:50.033 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":27,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:07:12.969614+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T17:06:19.218543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T17:06:23.525474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T17:06:24.911574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-12T17:06:28.017172+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.733031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.775178+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.591522+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.820838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.829252+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.582481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.18","mappings":[{"from":3,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:07:50.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.043+0000 7f2105a00640 1 -- 172.21.15.52:0/1508202209 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f20dc06ac30 msgr2=0x7f20dc06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:50.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.043+0000 7f2105a00640 1 --2- 172.21.15.52:0/1508202209 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f20dc06ac30 0x7f20dc06d0f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f20f0007fd0 tx=0x7f20f0003970 comp rx=0 tx=0).stop 2024-03-12T17:07:50.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.043+0000 7f2105a00640 1 -- 172.21.15.52:0/1508202209 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21000b2c80 msgr2=0x7f21000b3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:50.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.043+0000 7f2105a00640 1 --2- 172.21.15.52:0/1508202209 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21000b2c80 0x7f21000b3100 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f21000c61a0 tx=0x7f20f800eb90 comp rx=0 tx=0).stop 2024-03-12T17:07:50.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.043+0000 7f2105a00640 1 -- 172.21.15.52:0/1508202209 shutdown_connections 2024-03-12T17:07:50.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.043+0000 7f2105a00640 1 --2- 172.21.15.52:0/1508202209 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f20dc06ac30 0x7f20dc06d0f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:50.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.043+0000 7f2105a00640 1 --2- 172.21.15.52:0/1508202209 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f21000b7290 0x7f21000b7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:50.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.043+0000 7f2105a00640 1 --2- 172.21.15.52:0/1508202209 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21000b2c80 0x7f21000b3100 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:50.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.043+0000 7f2105a00640 1 --2- 172.21.15.52:0/1508202209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20e40047c0 0x7f21000c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:50.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.043+0000 7f2105a00640 1 -- 172.21.15.52:0/1508202209 >> 172.21.15.52:0/1508202209 conn(0x7f210006ae90 msgr2=0x7f21000d1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:50.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.043+0000 7f2105a00640 1 -- 172.21.15.52:0/1508202209 shutdown_connections 2024-03-12T17:07:50.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.043+0000 7f2105a00640 1 -- 172.21.15.52:0/1508202209 wait complete. 2024-03-12T17:07:50.055 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-12T17:07:50.055 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T17:07:50.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.271+0000 7f358dc00640 1 Processor -- start 2024-03-12T17:07:50.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.271+0000 7f358dc00640 1 -- start start 2024-03-12T17:07:50.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.273+0000 7f358dc00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35881571d0 0x7f35880b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:50.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.273+0000 7f358dc00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f35880cae50 0x7f35880cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:50.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.273+0000 7f358dc00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35881575b0 con 0x7f35880cdc00 2024-03-12T17:07:50.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.273+0000 7f358dc00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35880cb820 con 0x7f35880cae50 2024-03-12T17:07:50.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.273+0000 7f358dc00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3588159f20 con 0x7f35881571d0 2024-03-12T17:07:50.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.273+0000 7f3587400640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f35880cdc00 0x7f358815aaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:46430/0 (socket says 172.21.15.52:46430) 2024-03-12T17:07:50.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.273+0000 7f3587400640 1 -- 172.21.15.52:0/3584657390 learned_addr learned my addr 172.21.15.52:0/3584657390 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:50.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.274+0000 7f358d200640 1 --2- 172.21.15.52:0/3584657390 >> v2:172.21.15.102:3300/0 conn(0x7f35880cae50 0x7f35880cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:50.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.274+0000 7f3587e00640 1 --2- 172.21.15.52:0/3584657390 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35881571d0 0x7f35880b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:50.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.274+0000 7f3586a00640 1 -- 172.21.15.52:0/3584657390 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4122131065 0 0) 0x7f35881575b0 con 0x7f35880cdc00 2024-03-12T17:07:50.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.274+0000 7f3586a00640 1 -- 172.21.15.52:0/3584657390 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f356c003520 con 0x7f35880cdc00 2024-03-12T17:07:50.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.274+0000 7f3586a00640 1 -- 172.21.15.52:0/3584657390 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1538350081 0 0) 0x7f356c003520 con 0x7f35880cdc00 2024-03-12T17:07:50.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.274+0000 7f3586a00640 1 -- 172.21.15.52:0/3584657390 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35881571d0 msgr2=0x7f35880b35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:50.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.274+0000 7f3586a00640 1 --2- 172.21.15.52:0/3584657390 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35881571d0 0x7f35880b35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:50.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.274+0000 7f3586a00640 1 -- 172.21.15.52:0/3584657390 >> v2:172.21.15.102:3300/0 conn(0x7f35880cae50 msgr2=0x7f35880cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:50.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.274+0000 7f3586a00640 1 --2- 172.21.15.52:0/3584657390 >> v2:172.21.15.102:3300/0 conn(0x7f35880cae50 0x7f35880cb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:50.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.274+0000 7f3586a00640 1 -- 172.21.15.52:0/3584657390 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358815a200 con 0x7f35880cdc00 2024-03-12T17:07:50.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.274+0000 7f3586a00640 1 -- 172.21.15.52:0/3584657390 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f35740019f0 con 0x7f35880cdc00 2024-03-12T17:07:50.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.274+0000 7f3586a00640 1 -- 172.21.15.52:0/3584657390 >> v1:172.21.15.52:6789/0 conn(0x7f35880cdc00 legacy=0x7f358815aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:50.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.274+0000 7f3586a00640 1 --2- 172.21.15.52:0/3584657390 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35881571d0 0x7f356c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:50.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.274+0000 7f3586a00640 1 --2- 172.21.15.52:0/3584657390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f356c003ef0 0x7f356c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:50.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.274+0000 7f3586a00640 1 --2- 172.21.15.52:0/3584657390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f356c004d00 0x7f356c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:50.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.274+0000 7f3586a00640 1 -- 172.21.15.52:0/3584657390 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35880cb820 con 0x7f356c003ef0 2024-03-12T17:07:50.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.274+0000 7f3586a00640 1 -- 172.21.15.52:0/3584657390 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3588159f20 con 0x7f356c004d00 2024-03-12T17:07:50.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.274+0000 7f3586a00640 1 -- 172.21.15.52:0/3584657390 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f356c00db90 con 0x7f35881571d0 2024-03-12T17:07:50.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.275+0000 7f3587400640 1 --2- 172.21.15.52:0/3584657390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f356c003ef0 0x7f356c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:50.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.275+0000 7f3587400640 1 -- 172.21.15.52:0/3584657390 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35881571d0 msgr2=0x7f356c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:50.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.275+0000 7f3587400640 1 --2- 172.21.15.52:0/3584657390 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35881571d0 0x7f356c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:50.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.275+0000 7f3587400640 1 -- 172.21.15.52:0/3584657390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f356c004d00 msgr2=0x7f356c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:50.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.275+0000 7f3587400640 1 --2- 172.21.15.52:0/3584657390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f356c004d00 0x7f356c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:50.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.275+0000 7f3587400640 1 -- 172.21.15.52:0/3584657390 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f356c003520 con 0x7f356c003ef0 2024-03-12T17:07:50.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.276+0000 7f3587400640 1 --2- 172.21.15.52:0/3584657390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f356c003ef0 0x7f356c0047c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f357400a860 tx=0x7f356c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c13fc1557cce81 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:50.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.276+0000 7f3586a00640 1 -- 172.21.15.52:0/3584657390 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3574034630 con 0x7f356c003ef0 2024-03-12T17:07:50.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.276+0000 7f3586a00640 1 -- 172.21.15.52:0/3584657390 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f357402f070 con 0x7f356c003ef0 2024-03-12T17:07:50.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.276+0000 7f3586a00640 1 -- 172.21.15.52:0/3584657390 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3574034a30 con 0x7f356c003ef0 2024-03-12T17:07:50.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.277+0000 7f358dc00640 1 -- 172.21.15.52:0/3584657390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f356c003ef0 msgr2=0x7f356c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:50.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.277+0000 7f358dc00640 1 --2- 172.21.15.52:0/3584657390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f356c003ef0 0x7f356c0047c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f357400a860 tx=0x7f356c0057f0 comp rx=0 tx=0).stop 2024-03-12T17:07:50.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.277+0000 7f358d200640 1 -- 172.21.15.52:0/3584657390 reap_dead start 2024-03-12T17:07:50.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.277+0000 7f358dc00640 1 -- 172.21.15.52:0/3584657390 shutdown_connections 2024-03-12T17:07:50.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.277+0000 7f358dc00640 1 -- 172.21.15.52:0/3584657390 >> 172.21.15.52:0/3584657390 conn(0x7f3588065df0 msgr2=0x7f3588069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:50.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.277+0000 7f358dc00640 1 -- 172.21.15.52:0/3584657390 shutdown_connections 2024-03-12T17:07:50.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.277+0000 7f358dc00640 1 -- 172.21.15.52:0/3584657390 wait complete. 2024-03-12T17:07:50.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.279+0000 7f358dc00640 1 Processor -- start 2024-03-12T17:07:50.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.279+0000 7f358dc00640 1 -- start start 2024-03-12T17:07:50.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.280+0000 7f358dc00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f356c003ef0 0x7f3588155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:50.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.280+0000 7f358dc00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35881ee570 0x7f35880a74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:50.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.280+0000 7f358dc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35880afe50 0x7f35880b02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:50.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.280+0000 7f358dc00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35881eeb30 con 0x7f35881ee570 2024-03-12T17:07:50.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.280+0000 7f358dc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35880aead0 con 0x7f35880afe50 2024-03-12T17:07:50.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.280+0000 7f358dc00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35880aedb0 con 0x7f356c003ef0 2024-03-12T17:07:50.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.280+0000 7f3587400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35881ee570 0x7f35880a74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:50.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.280+0000 7f3587400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35881ee570 0x7f35880a74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55812/0 (socket says 172.21.15.52:55812) 2024-03-12T17:07:50.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.280+0000 7f3587400640 1 -- 172.21.15.52:0/3753604523 learned_addr learned my addr 172.21.15.52:0/3753604523 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:50.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.281+0000 7f3587e00640 1 --2- 172.21.15.52:0/3753604523 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f356c003ef0 0x7f3588155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:50.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.281+0000 7f358d200640 1 --2- 172.21.15.52:0/3753604523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35880afe50 0x7f35880b02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:50.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.281+0000 7f3587400640 1 -- 172.21.15.52:0/3753604523 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f356c003ef0 msgr2=0x7f3588155a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:50.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.281+0000 7f3587400640 1 --2- 172.21.15.52:0/3753604523 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f356c003ef0 0x7f3588155a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:50.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.281+0000 7f3587400640 1 -- 172.21.15.52:0/3753604523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35880afe50 msgr2=0x7f35880b02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:50.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.281+0000 7f3587400640 1 --2- 172.21.15.52:0/3753604523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35880afe50 0x7f35880b02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:50.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.281+0000 7f3587400640 1 -- 172.21.15.52:0/3753604523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35880af090 con 0x7f35881ee570 2024-03-12T17:07:50.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.281+0000 7f3587400640 1 --2- 172.21.15.52:0/3753604523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35881ee570 0x7f35880a74a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f357402cc00 tx=0x7f3574001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:50.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.281+0000 7f357fe00640 1 -- 172.21.15.52:0/3753604523 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3574034640 con 0x7f35881ee570 2024-03-12T17:07:50.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.282+0000 7f357fe00640 1 -- 172.21.15.52:0/3753604523 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35740347a0 con 0x7f35881ee570 2024-03-12T17:07:50.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.282+0000 7f357fe00640 1 -- 172.21.15.52:0/3753604523 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3574034b50 con 0x7f35881ee570 2024-03-12T17:07:50.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.282+0000 7f358dc00640 1 -- 172.21.15.52:0/3753604523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35880af370 con 0x7f35881ee570 2024-03-12T17:07:50.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.282+0000 7f358dc00640 1 -- 172.21.15.52:0/3753604523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35880af930 con 0x7f35881ee570 2024-03-12T17:07:50.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.283+0000 7f358dc00640 1 -- 172.21.15.52:0/3753604523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35880afaf0 con 0x7f35881ee570 2024-03-12T17:07:50.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.284+0000 7f357fe00640 1 -- 172.21.15.52:0/3753604523 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3574031030 con 0x7f35881ee570 2024-03-12T17:07:50.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.284+0000 7f357fe00640 1 --2- 172.21.15.52:0/3753604523 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f356806ac30 0x7f356806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:50.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.285+0000 7f357fe00640 1 -- 172.21.15.52:0/3753604523 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f35740abb30 con 0x7f35881ee570 2024-03-12T17:07:50.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.285+0000 7f3587e00640 1 --2- 172.21.15.52:0/3753604523 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f356806ac30 0x7f356806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:50.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.286+0000 7f3587e00640 1 --2- 172.21.15.52:0/3753604523 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f356806ac30 0x7f356806d0f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3578009970 tx=0x7f3578008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:50.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.289+0000 7f357fe00640 1 -- 172.21.15.52:0/3753604523 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f357407a3d0 con 0x7f35881ee570 2024-03-12T17:07:50.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.526+0000 7f358dc00640 1 -- 172.21.15.52:0/3753604523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f358804e030 con 0x7f35881ee570 2024-03-12T17:07:50.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.527+0000 7f357fe00640 1 -- 172.21.15.52:0/3753604523 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4733 (secure 0 0 0) 0x7f3574043020 con 0x7f35881ee570 2024-03-12T17:07:50.520 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:07:50.520 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2950099092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2950099092},{"type":"v1","addr":"172.21.15.52:6834","nonce":2950099092}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/846108993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":846108993},{"type":"v1","addr":"172.21.15.52:6835","nonce":846108993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6836/3002500679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3002500679},{"type":"v1","addr":"172.21.15.102:6836","nonce":3002500679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-12T17:06:25.905186+0000","modified":"2024-03-12T17:06:54.075752+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6837/3333451504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6835","nonce":3333451504},{"type":"v1","addr":"172.21.15.102:6837","nonce":3333451504}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T17:07:50.521 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 9 2024-03-12T17:07:50.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.530+0000 7f358dc00640 1 -- 172.21.15.52:0/3753604523 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f356806ac30 msgr2=0x7f356806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:50.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.530+0000 7f358dc00640 1 --2- 172.21.15.52:0/3753604523 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f356806ac30 0x7f356806d0f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3578009970 tx=0x7f3578008040 comp rx=0 tx=0).stop 2024-03-12T17:07:50.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.530+0000 7f358dc00640 1 -- 172.21.15.52:0/3753604523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35881ee570 msgr2=0x7f35880a74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:50.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.530+0000 7f358dc00640 1 --2- 172.21.15.52:0/3753604523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35881ee570 0x7f35880a74a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f357402cc00 tx=0x7f3574001d80 comp rx=0 tx=0).stop 2024-03-12T17:07:50.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.530+0000 7f358dc00640 1 -- 172.21.15.52:0/3753604523 shutdown_connections 2024-03-12T17:07:50.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.530+0000 7f358dc00640 1 --2- 172.21.15.52:0/3753604523 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f356806ac30 0x7f356806d0f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:50.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.530+0000 7f358dc00640 1 --2- 172.21.15.52:0/3753604523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35880afe50 0x7f35880b02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:50.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.530+0000 7f358dc00640 1 --2- 172.21.15.52:0/3753604523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35881ee570 0x7f35880a74a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:50.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.530+0000 7f358dc00640 1 --2- 172.21.15.52:0/3753604523 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f356c003ef0 0x7f3588155a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:50.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.530+0000 7f358dc00640 1 -- 172.21.15.52:0/3753604523 >> 172.21.15.52:0/3753604523 conn(0x7f3588065df0 msgr2=0x7f358815c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:50.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.530+0000 7f358dc00640 1 -- 172.21.15.52:0/3753604523 shutdown_connections 2024-03-12T17:07:50.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.530+0000 7f358dc00640 1 -- 172.21.15.52:0/3753604523 wait complete. 2024-03-12T17:07:50.543 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T17:07:50.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.772+0000 7f259f200640 1 Processor -- start 2024-03-12T17:07:50.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.772+0000 7f259f200640 1 -- start start 2024-03-12T17:07:50.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.774+0000 7f259f200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f25a0143930 0x7f25a00cd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:50.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.774+0000 7f259f200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f25a00cdc50 0x7f25a0070020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:50.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.774+0000 7f259f200640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25a015b2b0 con 0x7f25a0143460 2024-03-12T17:07:50.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.774+0000 7f259f200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25a00cc6b0 con 0x7f25a0143930 2024-03-12T17:07:50.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.774+0000 7f259f200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25a00cc950 con 0x7f25a00cdc50 2024-03-12T17:07:50.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.774+0000 7f259de00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f25a0143460 0x7f25a015aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:46434/0 (socket says 172.21.15.52:46434) 2024-03-12T17:07:50.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.774+0000 7f259de00640 1 -- 172.21.15.52:0/803616117 learned_addr learned my addr 172.21.15.52:0/803616117 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:50.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.775+0000 7f2597e00640 1 --2- 172.21.15.52:0/803616117 >> v2:172.21.15.52:3300/0 conn(0x7f25a0143930 0x7f25a00cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:50.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.775+0000 7f259e800640 1 --2- 172.21.15.52:0/803616117 >> v2:172.21.15.102:3300/0 conn(0x7f25a00cdc50 0x7f25a0070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:50.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.775+0000 7f259d400640 1 -- 172.21.15.52:0/803616117 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2023597659 0 0) 0x7f25a015b2b0 con 0x7f25a0143460 2024-03-12T17:07:50.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.775+0000 7f259d400640 1 -- 172.21.15.52:0/803616117 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2584003520 con 0x7f25a0143460 2024-03-12T17:07:50.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.775+0000 7f2597e00640 1 -- 172.21.15.52:0/803616117 >> v2:172.21.15.102:3300/0 conn(0x7f25a00cdc50 msgr2=0x7f25a0070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:50.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.775+0000 7f2597e00640 1 --2- 172.21.15.52:0/803616117 >> v2:172.21.15.102:3300/0 conn(0x7f25a00cdc50 0x7f25a0070020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:50.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.775+0000 7f259d400640 1 -- 172.21.15.52:0/803616117 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 87503362 0 0) 0x7f2584003520 con 0x7f25a0143460 2024-03-12T17:07:50.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.775+0000 7f2597e00640 1 -- 172.21.15.52:0/803616117 >> v1:172.21.15.52:6789/0 conn(0x7f25a0143460 legacy=0x7f25a015aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:50.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.775+0000 7f2597e00640 1 -- 172.21.15.52:0/803616117 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a00ccc30 con 0x7f25a0143930 2024-03-12T17:07:50.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.777+0000 7f2597e00640 1 --2- 172.21.15.52:0/803616117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25a0143930 0x7f25a00cd710 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f25a00b1a50 tx=0x7f258802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334e338dd8fd48f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:50.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.777+0000 7f259d400640 1 -- 172.21.15.52:0/803616117 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2590001670 con 0x7f25a0143930 2024-03-12T17:07:50.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.777+0000 7f259d400640 1 -- 172.21.15.52:0/803616117 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2588031520 con 0x7f25a0143930 2024-03-12T17:07:50.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.777+0000 7f259d400640 1 -- 172.21.15.52:0/803616117 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25880316b0 con 0x7f25a0143930 2024-03-12T17:07:50.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.778+0000 7f259f200640 1 -- 172.21.15.52:0/803616117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25a0143930 msgr2=0x7f25a00cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:50.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.778+0000 7f259f200640 1 --2- 172.21.15.52:0/803616117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25a0143930 0x7f25a00cd710 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f25a00b1a50 tx=0x7f258802b840 comp rx=0 tx=0).stop 2024-03-12T17:07:50.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.778+0000 7f259f200640 1 -- 172.21.15.52:0/803616117 shutdown_connections 2024-03-12T17:07:50.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.778+0000 7f259f200640 1 --2- 172.21.15.52:0/803616117 >> v2:172.21.15.102:3300/0 conn(0x7f25a00cdc50 0x7f25a0070020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:50.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.778+0000 7f259f200640 1 --2- 172.21.15.52:0/803616117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25a0143930 0x7f25a00cd710 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:50.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.778+0000 7f259f200640 1 -- 172.21.15.52:0/803616117 >> 172.21.15.52:0/803616117 conn(0x7f25a0065df0 msgr2=0x7f25a0069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:50.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.778+0000 7f259f200640 1 -- 172.21.15.52:0/803616117 shutdown_connections 2024-03-12T17:07:50.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.778+0000 7f259f200640 1 -- 172.21.15.52:0/803616117 wait complete. 2024-03-12T17:07:50.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.780+0000 7f259f200640 1 Processor -- start 2024-03-12T17:07:50.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.780+0000 7f259f200640 1 -- start start 2024-03-12T17:07:50.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.781+0000 7f259f200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25a00cdc50 0x7f25a015a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:50.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.781+0000 7f259f200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f25a0143460 0x7f25a006cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:50.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.781+0000 7f259f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25a0143930 0x7f25a006d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:50.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.781+0000 7f259f200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a0139500 con 0x7f25a00cdc50 2024-03-12T17:07:50.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.781+0000 7f259f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a01e2e80 con 0x7f25a0143930 2024-03-12T17:07:50.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.781+0000 7f259f200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a0141a20 con 0x7f25a0143460 2024-03-12T17:07:50.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.782+0000 7f259de00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25a00cdc50 0x7f25a015a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:50.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.782+0000 7f259de00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25a00cdc50 0x7f25a015a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55834/0 (socket says 172.21.15.52:55834) 2024-03-12T17:07:50.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.782+0000 7f259de00640 1 -- 172.21.15.52:0/920050744 learned_addr learned my addr 172.21.15.52:0/920050744 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:50.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.782+0000 7f2597e00640 1 --2- 172.21.15.52:0/920050744 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f25a0143460 0x7f25a006cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:50.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.782+0000 7f259e800640 1 --2- 172.21.15.52:0/920050744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25a0143930 0x7f25a006d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:50.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.782+0000 7f259de00640 1 -- 172.21.15.52:0/920050744 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f25a0143460 msgr2=0x7f25a006cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:50.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.782+0000 7f259de00640 1 --2- 172.21.15.52:0/920050744 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f25a0143460 0x7f25a006cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:50.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.782+0000 7f259de00640 1 -- 172.21.15.52:0/920050744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25a0143930 msgr2=0x7f25a006d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:50.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.782+0000 7f259de00640 1 --2- 172.21.15.52:0/920050744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25a0143930 0x7f25a006d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:50.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.782+0000 7f259de00640 1 -- 172.21.15.52:0/920050744 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2588009400 con 0x7f25a00cdc50 2024-03-12T17:07:50.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.782+0000 7f259e800640 1 --2- 172.21.15.52:0/920050744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25a0143930 0x7f25a006d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:50.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.782+0000 7f259de00640 1 --2- 172.21.15.52:0/920050744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25a00cdc50 0x7f25a015a0c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f259000c790 tx=0x7f259000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:50.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.783+0000 7f2596000640 1 -- 172.21.15.52:0/920050744 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2590005ad0 con 0x7f25a00cdc50 2024-03-12T17:07:50.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.783+0000 7f2596000640 1 -- 172.21.15.52:0/920050744 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2590005c30 con 0x7f25a00cdc50 2024-03-12T17:07:50.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.783+0000 7f259f200640 1 -- 172.21.15.52:0/920050744 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a0141d00 con 0x7f25a00cdc50 2024-03-12T17:07:50.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.783+0000 7f2596000640 1 -- 172.21.15.52:0/920050744 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2590007070 con 0x7f25a00cdc50 2024-03-12T17:07:50.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.783+0000 7f259f200640 1 -- 172.21.15.52:0/920050744 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a01422c0 con 0x7f25a00cdc50 2024-03-12T17:07:50.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.784+0000 7f259f200640 1 -- 172.21.15.52:0/920050744 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25a01424b0 con 0x7f25a00cdc50 2024-03-12T17:07:50.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.785+0000 7f2596000640 1 -- 172.21.15.52:0/920050744 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f25900071d0 con 0x7f25a00cdc50 2024-03-12T17:07:50.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.786+0000 7f2596000640 1 --2- 172.21.15.52:0/920050744 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f257c06ac00 0x7f257c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:50.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.787+0000 7f2596000640 1 -- 172.21.15.52:0/920050744 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2590089540 con 0x7f25a00cdc50 2024-03-12T17:07:50.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.787+0000 7f2597e00640 1 --2- 172.21.15.52:0/920050744 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f257c06ac00 0x7f257c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:50.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.788+0000 7f2597e00640 1 --2- 172.21.15.52:0/920050744 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f257c06ac00 0x7f257c06d0c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2588035000 tx=0x7f2588030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:50.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:50.791+0000 7f2596000640 1 -- 172.21.15.52:0/920050744 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f25900542c0 con 0x7f25a00cdc50 2024-03-12T17:07:51.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.031+0000 7f259f200640 1 -- 172.21.15.52:0/920050744 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25a004e030 con 0x7f25a00cdc50 2024-03-12T17:07:51.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.032+0000 7f2596000640 1 -- 172.21.15.52:0/920050744 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4733 (secure 0 0 0) 0x7f25900578e0 con 0x7f25a00cdc50 2024-03-12T17:07:51.025 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:07:51.025 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2950099092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2950099092},{"type":"v1","addr":"172.21.15.52:6834","nonce":2950099092}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/846108993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":846108993},{"type":"v1","addr":"172.21.15.52:6835","nonce":846108993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6836/3002500679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3002500679},{"type":"v1","addr":"172.21.15.102:6836","nonce":3002500679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-12T17:06:25.905186+0000","modified":"2024-03-12T17:06:54.075752+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6837/3333451504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6835","nonce":3333451504},{"type":"v1","addr":"172.21.15.102:6837","nonce":3333451504}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-12T17:07:51.025 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 9 2024-03-12T17:07:51.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.035+0000 7f259f200640 1 -- 172.21.15.52:0/920050744 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f257c06ac00 msgr2=0x7f257c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:51.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.035+0000 7f259f200640 1 --2- 172.21.15.52:0/920050744 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f257c06ac00 0x7f257c06d0c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2588035000 tx=0x7f2588030000 comp rx=0 tx=0).stop 2024-03-12T17:07:51.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.035+0000 7f259f200640 1 -- 172.21.15.52:0/920050744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25a00cdc50 msgr2=0x7f25a015a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:51.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.035+0000 7f259f200640 1 --2- 172.21.15.52:0/920050744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25a00cdc50 0x7f25a015a0c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f259000c790 tx=0x7f259000cc50 comp rx=0 tx=0).stop 2024-03-12T17:07:51.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.035+0000 7f259f200640 1 -- 172.21.15.52:0/920050744 shutdown_connections 2024-03-12T17:07:51.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.035+0000 7f259f200640 1 --2- 172.21.15.52:0/920050744 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f257c06ac00 0x7f257c06d0c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:51.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.035+0000 7f259f200640 1 --2- 172.21.15.52:0/920050744 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f25a0143460 0x7f25a006cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:51.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.035+0000 7f259f200640 1 --2- 172.21.15.52:0/920050744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25a0143930 0x7f25a006d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:51.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.035+0000 7f259f200640 1 --2- 172.21.15.52:0/920050744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25a00cdc50 0x7f25a015a0c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:51.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.035+0000 7f259f200640 1 -- 172.21.15.52:0/920050744 >> 172.21.15.52:0/920050744 conn(0x7f25a0065df0 msgr2=0x7f25a00aea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:51.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.035+0000 7f259f200640 1 -- 172.21.15.52:0/920050744 shutdown_connections 2024-03-12T17:07:51.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.035+0000 7f259f200640 1 -- 172.21.15.52:0/920050744 wait complete. 2024-03-12T17:07:51.047 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:07:51.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.262+0000 7f7e09400640 1 Processor -- start 2024-03-12T17:07:51.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.263+0000 7f7e09400640 1 -- start start 2024-03-12T17:07:51.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.264+0000 7f7e09400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f7e0415cd70 0x7f7e040d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:51.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.264+0000 7f7e09400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e040d1330 0x7f7e040b4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:51.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.264+0000 7f7e09400640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e040b4ca0 con 0x7f7e040b4880 2024-03-12T17:07:51.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.264+0000 7f7e09400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e0415d150 con 0x7f7e0415cd70 2024-03-12T17:07:51.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.264+0000 7f7e09400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e040d17d0 con 0x7f7e040d1330 2024-03-12T17:07:51.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.265+0000 7f7e03200640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f7e040b4880 0x7f7e0406b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:46442/0 (socket says 172.21.15.52:46442) 2024-03-12T17:07:51.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.265+0000 7f7e03200640 1 -- 172.21.15.52:0/1031115820 learned_addr learned my addr 172.21.15.52:0/1031115820 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:51.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.265+0000 7f7e02800640 1 --2- 172.21.15.52:0/1031115820 >> v2:172.21.15.102:3300/0 conn(0x7f7e0415cd70 0x7f7e040d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:51.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.265+0000 7f7dfbe00640 1 --2- 172.21.15.52:0/1031115820 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e040d1330 0x7f7e040b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:51.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.265+0000 7f7e01e00640 1 -- 172.21.15.52:0/1031115820 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2314906777 0 0) 0x7f7e040b4ca0 con 0x7f7e040b4880 2024-03-12T17:07:51.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.265+0000 7f7e01e00640 1 -- 172.21.15.52:0/1031115820 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7de4003520 con 0x7f7e040b4880 2024-03-12T17:07:51.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.266+0000 7f7e01e00640 1 -- 172.21.15.52:0/1031115820 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1063998291 0 0) 0x7f7de4003520 con 0x7f7e040b4880 2024-03-12T17:07:51.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.266+0000 7f7e01e00640 1 -- 172.21.15.52:0/1031115820 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e040d1330 msgr2=0x7f7e040b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:51.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.266+0000 7f7e01e00640 1 --2- 172.21.15.52:0/1031115820 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e040d1330 0x7f7e040b4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:51.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.266+0000 7f7e01e00640 1 -- 172.21.15.52:0/1031115820 >> v2:172.21.15.102:3300/0 conn(0x7f7e0415cd70 msgr2=0x7f7e040d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:51.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.266+0000 7f7e01e00640 1 --2- 172.21.15.52:0/1031115820 >> v2:172.21.15.102:3300/0 conn(0x7f7e0415cd70 0x7f7e040d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:51.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.266+0000 7f7e01e00640 1 -- 172.21.15.52:0/1031115820 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e040d1ab0 con 0x7f7e040b4880 2024-03-12T17:07:51.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.266+0000 7f7e01e00640 1 -- 172.21.15.52:0/1031115820 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f7dfc0019f0 con 0x7f7e040b4880 2024-03-12T17:07:51.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.266+0000 7f7e01e00640 1 -- 172.21.15.52:0/1031115820 >> v1:172.21.15.52:6789/0 conn(0x7f7e040b4880 legacy=0x7f7e0406b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:51.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.266+0000 7f7e01e00640 1 --2- 172.21.15.52:0/1031115820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7de4003970 0x7f7de4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:51.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.266+0000 7f7e01e00640 1 --2- 172.21.15.52:0/1031115820 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e040d1330 0x7f7de40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:51.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.266+0000 7f7e01e00640 1 --2- 172.21.15.52:0/1031115820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7de4004d00 0x7f7de4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:51.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.266+0000 7f7e01e00640 1 -- 172.21.15.52:0/1031115820 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e040d17d0 con 0x7f7de4004d00 2024-03-12T17:07:51.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.266+0000 7f7e01e00640 1 -- 172.21.15.52:0/1031115820 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de400b980 con 0x7f7de4003970 2024-03-12T17:07:51.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.266+0000 7f7e01e00640 1 -- 172.21.15.52:0/1031115820 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7de400bac0 con 0x7f7e040d1330 2024-03-12T17:07:51.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.266+0000 7f7e03200640 1 --2- 172.21.15.52:0/1031115820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7de4004d00 0x7f7de4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:51.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.266+0000 7f7e02800640 1 --2- 172.21.15.52:0/1031115820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7de4003970 0x7f7de4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:51.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.267+0000 7f7dfbe00640 1 --2- 172.21.15.52:0/1031115820 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e040d1330 0x7f7de40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:51.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.267+0000 7f7e03200640 1 -- 172.21.15.52:0/1031115820 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e040d1330 msgr2=0x7f7de40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:51.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.267+0000 7f7e03200640 1 --2- 172.21.15.52:0/1031115820 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e040d1330 0x7f7de40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:51.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.267+0000 7f7e03200640 1 -- 172.21.15.52:0/1031115820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7de4003970 msgr2=0x7f7de4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:51.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.267+0000 7f7e03200640 1 --2- 172.21.15.52:0/1031115820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7de4003970 0x7f7de4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:51.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.267+0000 7f7e03200640 1 -- 172.21.15.52:0/1031115820 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de4003520 con 0x7f7de4004d00 2024-03-12T17:07:51.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.267+0000 7f7e02800640 1 --2- 172.21.15.52:0/1031115820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7de4003970 0x7f7de4003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:07:51.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.268+0000 7f7e03200640 1 --2- 172.21.15.52:0/1031115820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7de4004d00 0x7f7de4005180 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc00a900 tx=0x7f7de40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac7bff624f89f817 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:51.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.268+0000 7f7e01e00640 1 -- 172.21.15.52:0/1031115820 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc034630 con 0x7f7de4004d00 2024-03-12T17:07:51.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.268+0000 7f7e01e00640 1 -- 172.21.15.52:0/1031115820 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dfc02f070 con 0x7f7de4004d00 2024-03-12T17:07:51.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.268+0000 7f7e01e00640 1 -- 172.21.15.52:0/1031115820 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc0349d0 con 0x7f7de4004d00 2024-03-12T17:07:51.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.269+0000 7f7e09400640 1 -- 172.21.15.52:0/1031115820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7de4004d00 msgr2=0x7f7de4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:51.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.269+0000 7f7e09400640 1 --2- 172.21.15.52:0/1031115820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7de4004d00 0x7f7de4005180 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc00a900 tx=0x7f7de40057f0 comp rx=0 tx=0).stop 2024-03-12T17:07:51.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.269+0000 7f7e03200640 1 -- 172.21.15.52:0/1031115820 reap_dead start 2024-03-12T17:07:51.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.269+0000 7f7e09400640 1 -- 172.21.15.52:0/1031115820 shutdown_connections 2024-03-12T17:07:51.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.269+0000 7f7e09400640 1 -- 172.21.15.52:0/1031115820 >> 172.21.15.52:0/1031115820 conn(0x7f7e040cf340 msgr2=0x7f7e040cf750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:51.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.269+0000 7f7e09400640 1 -- 172.21.15.52:0/1031115820 shutdown_connections 2024-03-12T17:07:51.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.269+0000 7f7e09400640 1 -- 172.21.15.52:0/1031115820 wait complete. 2024-03-12T17:07:51.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.271+0000 7f7e09400640 1 Processor -- start 2024-03-12T17:07:51.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.271+0000 7f7e09400640 1 -- start start 2024-03-12T17:07:51.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.272+0000 7f7e09400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7de4004d00 0x7f7e0415f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:51.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.272+0000 7f7e09400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e041f1770 0x7f7e041556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:51.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.272+0000 7f7e09400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e041f1bf0 0x7f7e04152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:51.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.272+0000 7f7e09400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0415fb20 con 0x7f7e041f1bf0 2024-03-12T17:07:51.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.272+0000 7f7e09400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0415e540 con 0x7f7e041f1770 2024-03-12T17:07:51.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.272+0000 7f7e09400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0415e820 con 0x7f7de4004d00 2024-03-12T17:07:51.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.272+0000 7f7e03200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e041f1bf0 0x7f7e04152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:51.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.272+0000 7f7e03200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e041f1bf0 0x7f7e04152b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55856/0 (socket says 172.21.15.52:55856) 2024-03-12T17:07:51.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.272+0000 7f7e03200640 1 -- 172.21.15.52:0/3882326320 learned_addr learned my addr 172.21.15.52:0/3882326320 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:51.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.273+0000 7f7dfbe00640 1 --2- 172.21.15.52:0/3882326320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e041f1770 0x7f7e041556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:51.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.273+0000 7f7e02800640 1 --2- 172.21.15.52:0/3882326320 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7de4004d00 0x7f7e0415f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:51.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.273+0000 7f7e03200640 1 -- 172.21.15.52:0/3882326320 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7de4004d00 msgr2=0x7f7e0415f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:51.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.273+0000 7f7e03200640 1 --2- 172.21.15.52:0/3882326320 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7de4004d00 0x7f7e0415f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:51.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.273+0000 7f7e03200640 1 -- 172.21.15.52:0/3882326320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e041f1770 msgr2=0x7f7e041556b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:51.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.273+0000 7f7e03200640 1 --2- 172.21.15.52:0/3882326320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e041f1770 0x7f7e041556b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:51.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.273+0000 7f7e03200640 1 -- 172.21.15.52:0/3882326320 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e0415eb00 con 0x7f7e041f1bf0 2024-03-12T17:07:51.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.273+0000 7f7e03200640 1 --2- 172.21.15.52:0/3882326320 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e041f1bf0 0x7f7e04152b40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc000f80 tx=0x7f7dfc001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:51.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.273+0000 7f7dfaa00640 1 -- 172.21.15.52:0/3882326320 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc043070 con 0x7f7e041f1bf0 2024-03-12T17:07:51.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.273+0000 7f7dfaa00640 1 -- 172.21.15.52:0/3882326320 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dfc002000 con 0x7f7e041f1bf0 2024-03-12T17:07:51.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.273+0000 7f7dfaa00640 1 -- 172.21.15.52:0/3882326320 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc034860 con 0x7f7e041f1bf0 2024-03-12T17:07:51.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.273+0000 7f7e09400640 1 -- 172.21.15.52:0/3882326320 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e0415ede0 con 0x7f7e041f1bf0 2024-03-12T17:07:51.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.274+0000 7f7e09400640 1 -- 172.21.15.52:0/3882326320 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e0415e1a0 con 0x7f7e041f1bf0 2024-03-12T17:07:51.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.275+0000 7f7e09400640 1 -- 172.21.15.52:0/3882326320 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e0415f300 con 0x7f7e041f1bf0 2024-03-12T17:07:51.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.276+0000 7f7dfaa00640 1 -- 172.21.15.52:0/3882326320 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7dfc031030 con 0x7f7e041f1bf0 2024-03-12T17:07:51.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.277+0000 7f7dfaa00640 1 --2- 172.21.15.52:0/3882326320 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f7de006ac00 0x7f7de006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:51.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.277+0000 7f7dfaa00640 1 -- 172.21.15.52:0/3882326320 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7dfc0aad10 con 0x7f7e041f1bf0 2024-03-12T17:07:51.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.277+0000 7f7e02800640 1 --2- 172.21.15.52:0/3882326320 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f7de006ac00 0x7f7de006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:51.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.278+0000 7f7e02800640 1 --2- 172.21.15.52:0/3882326320 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f7de006ac00 0x7f7de006d0c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7e040b1630 tx=0x7f7df4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:51.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.281+0000 7f7dfaa00640 1 -- 172.21.15.52:0/3882326320 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f7e0415f300 con 0x7f7e041f1bf0 2024-03-12T17:07:51.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.482+0000 7f7e09400640 1 -- 172.21.15.52:0/3882326320 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e0415f300 con 0x7f7e041f1bf0 2024-03-12T17:07:51.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.485+0000 7f7dfaa00640 1 -- 172.21.15.52:0/3882326320 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18207 (secure 0 0 0) 0x7f7e0415f300 con 0x7f7e041f1bf0 2024-03-12T17:07:51.478 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:07:51.479 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":27,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:07:12.969614+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T17:06:19.218543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-12T17:06:23.525474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-12T17:06:24.911574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-12T17:06:28.017172+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.733031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.775178+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.591522+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.820838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.829252+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.582481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.18","mappings":[{"from":3,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:07:51.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.487+0000 7f7e09400640 1 -- 172.21.15.52:0/3882326320 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f7de006ac00 msgr2=0x7f7de006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:51.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.487+0000 7f7e09400640 1 --2- 172.21.15.52:0/3882326320 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f7de006ac00 0x7f7de006d0c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7e040b1630 tx=0x7f7df4008040 comp rx=0 tx=0).stop 2024-03-12T17:07:51.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.487+0000 7f7e09400640 1 -- 172.21.15.52:0/3882326320 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e041f1bf0 msgr2=0x7f7e04152b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:51.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.487+0000 7f7e09400640 1 --2- 172.21.15.52:0/3882326320 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e041f1bf0 0x7f7e04152b40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc000f80 tx=0x7f7dfc001360 comp rx=0 tx=0).stop 2024-03-12T17:07:51.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.488+0000 7f7e09400640 1 -- 172.21.15.52:0/3882326320 shutdown_connections 2024-03-12T17:07:51.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.488+0000 7f7e09400640 1 --2- 172.21.15.52:0/3882326320 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f7de006ac00 0x7f7de006d0c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:51.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.488+0000 7f7e09400640 1 --2- 172.21.15.52:0/3882326320 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e041f1bf0 0x7f7e04152b40 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:51.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.488+0000 7f7e09400640 1 --2- 172.21.15.52:0/3882326320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e041f1770 0x7f7e041556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:51.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.488+0000 7f7e09400640 1 --2- 172.21.15.52:0/3882326320 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7de4004d00 0x7f7e0415f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:51.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.488+0000 7f7e09400640 1 -- 172.21.15.52:0/3882326320 >> 172.21.15.52:0/3882326320 conn(0x7f7e040cf340 msgr2=0x7f7e04159640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:51.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.488+0000 7f7e09400640 1 -- 172.21.15.52:0/3882326320 shutdown_connections 2024-03-12T17:07:51.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.488+0000 7f7e09400640 1 -- 172.21.15.52:0/3882326320 wait complete. 2024-03-12T17:07:51.500 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-12T17:07:51.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.723+0000 7f89d1400640 1 Processor -- start 2024-03-12T17:07:51.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.723+0000 7f89d1400640 1 -- start start 2024-03-12T17:07:51.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.725+0000 7f89d1400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f89cc15cd00 0x7f89cc0b2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:51.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.725+0000 7f89d1400640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89cc0b2f70 con 0x7f89cc0707e0 2024-03-12T17:07:51.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.725+0000 7f89d1400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89cc0b3250 con 0x7f89cc146070 2024-03-12T17:07:51.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.725+0000 7f89d1400640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89cc0b3530 con 0x7f89cc15cd00 2024-03-12T17:07:51.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.726+0000 7f89d0a00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f89cc15cd00 0x7f89cc0b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:51.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.726+0000 7f89d0a00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f89cc15cd00 0x7f89cc0b2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55860/0 (socket says 172.21.15.52:55860) 2024-03-12T17:07:51.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.726+0000 7f89d0a00640 1 -- 172.21.15.52:0/3538823226 learned_addr learned my addr 172.21.15.52:0/3538823226 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:51.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.726+0000 7f89cb400640 1 --1- 172.21.15.52:0/3538823226 >> v1:172.21.15.52:6789/0 conn(0x7f89cc0707e0 0x7f89cc0ad0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:46458/0 (socket says 172.21.15.52:46458) 2024-03-12T17:07:51.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.726+0000 7f89d0a00640 1 -- 172.21.15.52:0/3538823226 >> v1:172.21.15.102:6789/0 conn(0x7f89cc146070 legacy=0x7f89cc0700d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:51.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.726+0000 7f89d0a00640 1 -- 172.21.15.52:0/3538823226 >> v1:172.21.15.52:6789/0 conn(0x7f89cc0707e0 legacy=0x7f89cc0ad0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:51.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.726+0000 7f89d0a00640 1 -- 172.21.15.52:0/3538823226 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89cc0b3810 con 0x7f89cc15cd00 2024-03-12T17:07:51.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.727+0000 7f89d0a00640 1 --2- 172.21.15.52:0/3538823226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89cc15cd00 0x7f89cc0b2870 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f89c4009e50 tx=0x7f89c402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ec2808dcc3a4ce server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:51.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.727+0000 7f89caa00640 1 -- 172.21.15.52:0/3538823226 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c402ca80 con 0x7f89cc15cd00 2024-03-12T17:07:51.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.727+0000 7f89caa00640 1 -- 172.21.15.52:0/3538823226 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c402cbe0 con 0x7f89cc15cd00 2024-03-12T17:07:51.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.727+0000 7f89caa00640 1 -- 172.21.15.52:0/3538823226 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c40343f0 con 0x7f89cc15cd00 2024-03-12T17:07:51.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.728+0000 7f89d1400640 1 -- 172.21.15.52:0/3538823226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89cc15cd00 msgr2=0x7f89cc0b2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:51.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.728+0000 7f89d1400640 1 --2- 172.21.15.52:0/3538823226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89cc15cd00 0x7f89cc0b2870 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f89c4009e50 tx=0x7f89c402bee0 comp rx=0 tx=0).stop 2024-03-12T17:07:51.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.728+0000 7f89d1400640 1 -- 172.21.15.52:0/3538823226 shutdown_connections 2024-03-12T17:07:51.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.728+0000 7f89d1400640 1 --2- 172.21.15.52:0/3538823226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89cc15cd00 0x7f89cc0b2870 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:51.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.728+0000 7f89d1400640 1 -- 172.21.15.52:0/3538823226 >> 172.21.15.52:0/3538823226 conn(0x7f89cc065df0 msgr2=0x7f89cc069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:51.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.728+0000 7f89d1400640 1 -- 172.21.15.52:0/3538823226 shutdown_connections 2024-03-12T17:07:51.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.728+0000 7f89d1400640 1 -- 172.21.15.52:0/3538823226 wait complete. 2024-03-12T17:07:51.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.730+0000 7f89d1400640 1 Processor -- start 2024-03-12T17:07:51.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.730+0000 7f89d1400640 1 -- start start 2024-03-12T17:07:51.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.731+0000 7f89d1400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f89cc0707e0 0x7f89cc140760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:51.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.731+0000 7f89d1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89cc146070 0x7f89cc159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:51.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.731+0000 7f89d1400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89cc15cd00 0x7f89cc139f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:51.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.731+0000 7f89d1400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc140340 con 0x7f89cc15cd00 2024-03-12T17:07:51.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.731+0000 7f89d1400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc13a480 con 0x7f89cc146070 2024-03-12T17:07:51.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.731+0000 7f89d1400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc13bfb0 con 0x7f89cc0707e0 2024-03-12T17:07:51.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.731+0000 7f89d0a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89cc15cd00 0x7f89cc139f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:51.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.731+0000 7f89d0a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89cc15cd00 0x7f89cc139f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55870/0 (socket says 172.21.15.52:55870) 2024-03-12T17:07:51.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.731+0000 7f89d0a00640 1 -- 172.21.15.52:0/1157315570 learned_addr learned my addr 172.21.15.52:0/1157315570 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:51.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.731+0000 7f89cb400640 1 --2- 172.21.15.52:0/1157315570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89cc146070 0x7f89cc159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:51.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.731+0000 7f89cbe00640 1 --2- 172.21.15.52:0/1157315570 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f89cc0707e0 0x7f89cc140760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:51.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.732+0000 7f89d0a00640 1 -- 172.21.15.52:0/1157315570 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f89cc0707e0 msgr2=0x7f89cc140760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:51.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.732+0000 7f89d0a00640 1 --2- 172.21.15.52:0/1157315570 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f89cc0707e0 0x7f89cc140760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:51.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.732+0000 7f89d0a00640 1 -- 172.21.15.52:0/1157315570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89cc146070 msgr2=0x7f89cc159e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:51.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.732+0000 7f89d0a00640 1 --2- 172.21.15.52:0/1157315570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89cc146070 0x7f89cc159e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:51.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.732+0000 7f89d0a00640 1 -- 172.21.15.52:0/1157315570 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c4009b30 con 0x7f89cc15cd00 2024-03-12T17:07:51.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.732+0000 7f89cbe00640 1 --2- 172.21.15.52:0/1157315570 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f89cc0707e0 0x7f89cc140760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:51.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.732+0000 7f89cb400640 1 --2- 172.21.15.52:0/1157315570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89cc146070 0x7f89cc159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:51.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.732+0000 7f89d0a00640 1 --2- 172.21.15.52:0/1157315570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89cc15cd00 0x7f89cc139f40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f89c4009f70 tx=0x7f89c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:51.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.732+0000 7f89c3e00640 1 -- 172.21.15.52:0/1157315570 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c403e450 con 0x7f89cc15cd00 2024-03-12T17:07:51.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.732+0000 7f89c3e00640 1 -- 172.21.15.52:0/1157315570 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c403e5d0 con 0x7f89cc15cd00 2024-03-12T17:07:51.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.732+0000 7f89c3e00640 1 -- 172.21.15.52:0/1157315570 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c403e980 con 0x7f89cc15cd00 2024-03-12T17:07:51.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.732+0000 7f89d1400640 1 -- 172.21.15.52:0/1157315570 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89cc13f6c0 con 0x7f89cc15cd00 2024-03-12T17:07:51.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.732+0000 7f89d1400640 1 -- 172.21.15.52:0/1157315570 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89cc13fc80 con 0x7f89cc15cd00 2024-03-12T17:07:51.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.734+0000 7f89d1400640 1 -- 172.21.15.52:0/1157315570 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89cc140480 con 0x7f89cc15cd00 2024-03-12T17:07:51.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.735+0000 7f89c3e00640 1 -- 172.21.15.52:0/1157315570 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f89c402cd50 con 0x7f89cc15cd00 2024-03-12T17:07:51.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.736+0000 7f89c3e00640 1 --2- 172.21.15.52:0/1157315570 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f89ac06ac00 0x7f89ac06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:51.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.736+0000 7f89c3e00640 1 -- 172.21.15.52:0/1157315570 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f89c40ac380 con 0x7f89cc15cd00 2024-03-12T17:07:51.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.737+0000 7f89cbe00640 1 --2- 172.21.15.52:0/1157315570 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f89ac06ac00 0x7f89ac06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:51.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.738+0000 7f89cbe00640 1 --2- 172.21.15.52:0/1157315570 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f89ac06ac00 0x7f89ac06d0c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f89bc003eb0 tx=0x7f89bc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:51.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.741+0000 7f89c3e00640 1 -- 172.21.15.52:0/1157315570 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f89cc140480 con 0x7f89cc15cd00 2024-03-12T17:07:51.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.977+0000 7f89d1400640 1 -- 172.21.15.52:0/1157315570 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f89cc140480 con 0x7f89cc15cd00 2024-03-12T17:07:51.975 INFO:tasks.ceph.mon.a.smithi052.stderr:2024-03-12T17:07:51.982+0000 7f074c400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-12T17:07:51.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.997+0000 7f89c3e00640 1 -- 172.21.15.52:0/1157315570 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7f89cc140480 con 0x7f89cc15cd00 2024-03-12T17:07:51.990 INFO:teuthology.orchestra.run.smithi052.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-12T17:07:51.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.999+0000 7f89d1400640 1 -- 172.21.15.52:0/1157315570 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f89ac06ac00 msgr2=0x7f89ac06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:51.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.999+0000 7f89d1400640 1 --2- 172.21.15.52:0/1157315570 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f89ac06ac00 0x7f89ac06d0c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f89bc003eb0 tx=0x7f89bc009040 comp rx=0 tx=0).stop 2024-03-12T17:07:51.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.999+0000 7f89d1400640 1 -- 172.21.15.52:0/1157315570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89cc15cd00 msgr2=0x7f89cc139f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:51.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:51.999+0000 7f89d1400640 1 --2- 172.21.15.52:0/1157315570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89cc15cd00 0x7f89cc139f40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f89c4009f70 tx=0x7f89c4004a40 comp rx=0 tx=0).stop 2024-03-12T17:07:51.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.000+0000 7f89d1400640 1 -- 172.21.15.52:0/1157315570 shutdown_connections 2024-03-12T17:07:51.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.000+0000 7f89d1400640 1 --2- 172.21.15.52:0/1157315570 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f89ac06ac00 0x7f89ac06d0c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:51.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.000+0000 7f89d1400640 1 --2- 172.21.15.52:0/1157315570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89cc15cd00 0x7f89cc139f40 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:51.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.000+0000 7f89d1400640 1 --2- 172.21.15.52:0/1157315570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89cc146070 0x7f89cc159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:51.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.000+0000 7f89d1400640 1 --2- 172.21.15.52:0/1157315570 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f89cc0707e0 0x7f89cc140760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:51.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.000+0000 7f89d1400640 1 -- 172.21.15.52:0/1157315570 >> 172.21.15.52:0/1157315570 conn(0x7f89cc065df0 msgr2=0x7f89cc0b1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:51.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.000+0000 7f89d1400640 1 -- 172.21.15.52:0/1157315570 shutdown_connections 2024-03-12T17:07:51.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.000+0000 7f89d1400640 1 -- 172.21.15.52:0/1157315570 wait complete. 2024-03-12T17:07:52.014 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-12T17:07:52.021 INFO:tasks.ceph.mds.b.smithi102.stderr: -5847> 2024-03-12T17:06:29.367+0000 7f5214c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-12T17:07:52.103 INFO:tasks.ceph.mds.b.smithi102.stdout:starting mds.b at 2024-03-12T17:07:52.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.246+0000 7fee8b000640 1 Processor -- start 2024-03-12T17:07:52.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.246+0000 7fee8b000640 1 -- start start 2024-03-12T17:07:52.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.247+0000 7fee8b000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fee840cd150 0x7fee840b01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:52.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.247+0000 7fee8b000640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fee840b0730 0x7fee840cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:52.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.247+0000 7fee8b000640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee840cc3f0 con 0x7fee840cd530 2024-03-12T17:07:52.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.247+0000 7fee8b000640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee840cbe30 con 0x7fee840b0730 2024-03-12T17:07:52.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.247+0000 7fee8b000640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee840cc110 con 0x7fee840cd150 2024-03-12T17:07:52.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.248+0000 7fee8a600640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fee840cd530 0x7fee8415a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:46466/0 (socket says 172.21.15.52:46466) 2024-03-12T17:07:52.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.248+0000 7fee8a600640 1 -- 172.21.15.52:0/460232808 learned_addr learned my addr 172.21.15.52:0/460232808 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:52.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.248+0000 7fee89200640 1 --2- 172.21.15.52:0/460232808 >> v2:172.21.15.52:3300/0 conn(0x7fee840b0730 0x7fee840cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:52.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.248+0000 7fee89c00640 1 --2- 172.21.15.52:0/460232808 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fee840cd150 0x7fee840b01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:52.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.248+0000 7fee83e00640 1 -- 172.21.15.52:0/460232808 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 937582641 0 0) 0x7fee840cc3f0 con 0x7fee840cd530 2024-03-12T17:07:52.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.248+0000 7fee83e00640 1 -- 172.21.15.52:0/460232808 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee6c003520 con 0x7fee840cd530 2024-03-12T17:07:52.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.248+0000 7fee89200640 1 -- 172.21.15.52:0/460232808 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fee840cd150 msgr2=0x7fee840b01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:52.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.248+0000 7fee89200640 1 --2- 172.21.15.52:0/460232808 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fee840cd150 0x7fee840b01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:52.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.248+0000 7fee89200640 1 -- 172.21.15.52:0/460232808 >> v1:172.21.15.52:6789/0 conn(0x7fee840cd530 legacy=0x7fee8415a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:52.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.248+0000 7fee89200640 1 -- 172.21.15.52:0/460232808 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee840ce280 con 0x7fee840b0730 2024-03-12T17:07:52.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.250+0000 7fee89200640 1 --2- 172.21.15.52:0/460232808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee840b0730 0x7fee840cea80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fee700097a0 tx=0x7fee7002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7140587ca9e305e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:52.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.250+0000 7fee83e00640 1 -- 172.21.15.52:0/460232808 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7002e070 con 0x7fee840b0730 2024-03-12T17:07:52.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.250+0000 7fee83e00640 1 -- 172.21.15.52:0/460232808 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee700315a0 con 0x7fee840b0730 2024-03-12T17:07:52.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.250+0000 7fee83e00640 1 -- 172.21.15.52:0/460232808 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee70031930 con 0x7fee840b0730 2024-03-12T17:07:52.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.250+0000 7fee8b000640 1 -- 172.21.15.52:0/460232808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee840b0730 msgr2=0x7fee840cea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:52.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.250+0000 7fee8b000640 1 --2- 172.21.15.52:0/460232808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee840b0730 0x7fee840cea80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fee700097a0 tx=0x7fee7002b840 comp rx=0 tx=0).stop 2024-03-12T17:07:52.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.251+0000 7fee8b000640 1 -- 172.21.15.52:0/460232808 shutdown_connections 2024-03-12T17:07:52.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.251+0000 7fee8b000640 1 --2- 172.21.15.52:0/460232808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee840b0730 0x7fee840cea80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:52.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.251+0000 7fee8b000640 1 --2- 172.21.15.52:0/460232808 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fee840cd150 0x7fee840b01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:52.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.251+0000 7fee8b000640 1 -- 172.21.15.52:0/460232808 >> 172.21.15.52:0/460232808 conn(0x7fee840d0580 msgr2=0x7fee840d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:52.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.251+0000 7fee8b000640 1 -- 172.21.15.52:0/460232808 shutdown_connections 2024-03-12T17:07:52.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.251+0000 7fee8b000640 1 -- 172.21.15.52:0/460232808 wait complete. 2024-03-12T17:07:52.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.252+0000 7fee8b000640 1 Processor -- start 2024-03-12T17:07:52.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.253+0000 7fee8b000640 1 -- start start 2024-03-12T17:07:52.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.254+0000 7fee8b000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee840b0730 0x7fee841f4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:52.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.254+0000 7fee8b000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fee840cd150 0x7fee841f2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:52.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.254+0000 7fee8b000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee840cd530 0x7fee8406b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:52.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.254+0000 7fee8b000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee84154530 con 0x7fee840b0730 2024-03-12T17:07:52.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.254+0000 7fee8b000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8406c260 con 0x7fee840cd530 2024-03-12T17:07:52.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.254+0000 7fee8b000640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee8406bcf0 con 0x7fee840cd150 2024-03-12T17:07:52.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.254+0000 7fee89c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee840b0730 0x7fee841f4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:52.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.254+0000 7fee89c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee840b0730 0x7fee841f4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55884/0 (socket says 172.21.15.52:55884) 2024-03-12T17:07:52.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.254+0000 7fee89c00640 1 -- 172.21.15.52:0/3950264031 learned_addr learned my addr 172.21.15.52:0/3950264031 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:52.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.254+0000 7fee89200640 1 --2- 172.21.15.52:0/3950264031 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fee840cd150 0x7fee841f2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:52.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.254+0000 7fee8a600640 1 --2- 172.21.15.52:0/3950264031 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee840cd530 0x7fee8406b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:52.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.254+0000 7fee89c00640 1 -- 172.21.15.52:0/3950264031 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fee840cd150 msgr2=0x7fee841f2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:52.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.254+0000 7fee89c00640 1 --2- 172.21.15.52:0/3950264031 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fee840cd150 0x7fee841f2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:52.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.254+0000 7fee89c00640 1 -- 172.21.15.52:0/3950264031 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee840cd530 msgr2=0x7fee8406b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:52.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.254+0000 7fee89c00640 1 --2- 172.21.15.52:0/3950264031 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee840cd530 0x7fee8406b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:52.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.254+0000 7fee89c00640 1 -- 172.21.15.52:0/3950264031 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee70009400 con 0x7fee840b0730 2024-03-12T17:07:52.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.254+0000 7fee8a600640 1 --2- 172.21.15.52:0/3950264031 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee840cd530 0x7fee8406b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:52.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.255+0000 7fee89c00640 1 --2- 172.21.15.52:0/3950264031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee840b0730 0x7fee841f4db0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fee7800e9b0 tx=0x7fee7800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:52.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.255+0000 7fee81600640 1 -- 172.21.15.52:0/3950264031 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee78009670 con 0x7fee840b0730 2024-03-12T17:07:52.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.255+0000 7fee81600640 1 -- 172.21.15.52:0/3950264031 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee7800c9a0 con 0x7fee840b0730 2024-03-12T17:07:52.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.255+0000 7fee81600640 1 -- 172.21.15.52:0/3950264031 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7800cd50 con 0x7fee840b0730 2024-03-12T17:07:52.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.255+0000 7fee8b000640 1 -- 172.21.15.52:0/3950264031 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee841f22e0 con 0x7fee840b0730 2024-03-12T17:07:52.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.255+0000 7fee8b000640 1 -- 172.21.15.52:0/3950264031 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee841f28a0 con 0x7fee840b0730 2024-03-12T17:07:52.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.257+0000 7fee8b000640 1 -- 172.21.15.52:0/3950264031 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee840b1e70 con 0x7fee840b0730 2024-03-12T17:07:52.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.258+0000 7fee81600640 1 -- 172.21.15.52:0/3950264031 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fee78013470 con 0x7fee840b0730 2024-03-12T17:07:52.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.258+0000 7fee81600640 1 --2- 172.21.15.52:0/3950264031 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fee6406ac00 0x7fee6406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:52.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.259+0000 7fee81600640 1 -- 172.21.15.52:0/3950264031 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6749+0+0 (secure 0 0 0) 0x7fee78089a60 con 0x7fee840b0730 2024-03-12T17:07:52.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.259+0000 7fee89200640 1 --2- 172.21.15.52:0/3950264031 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fee6406ac00 0x7fee6406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:52.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.260+0000 7fee89200640 1 --2- 172.21.15.52:0/3950264031 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fee6406ac00 0x7fee6406d0c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fee70036000 tx=0x7fee7003d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:52.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.263+0000 7fee81600640 1 -- 172.21.15.52:0/3950264031 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fee78054830 con 0x7fee840b0730 2024-03-12T17:07:52.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:52.506+0000 7fee8b000640 1 -- 172.21.15.52:0/3950264031 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fee8404e030 con 0x7fee840b0730 2024-03-12T17:07:52.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.001+0000 7fee81600640 1 -- 172.21.15.52:0/3950264031 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fee78057e50 con 0x7fee840b0730 2024-03-12T17:07:52.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.005+0000 7fee8b000640 1 -- 172.21.15.52:0/3950264031 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fee6406ac00 msgr2=0x7fee6406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:52.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.005+0000 7fee8b000640 1 --2- 172.21.15.52:0/3950264031 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fee6406ac00 0x7fee6406d0c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fee70036000 tx=0x7fee7003d040 comp rx=0 tx=0).stop 2024-03-12T17:07:52.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.005+0000 7fee8b000640 1 -- 172.21.15.52:0/3950264031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee840b0730 msgr2=0x7fee841f4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:52.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.005+0000 7fee8b000640 1 --2- 172.21.15.52:0/3950264031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee840b0730 0x7fee841f4db0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fee7800e9b0 tx=0x7fee7800ee70 comp rx=0 tx=0).stop 2024-03-12T17:07:52.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.006+0000 7fee8b000640 1 -- 172.21.15.52:0/3950264031 shutdown_connections 2024-03-12T17:07:52.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.006+0000 7fee8b000640 1 --2- 172.21.15.52:0/3950264031 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fee6406ac00 0x7fee6406d0c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:52.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.006+0000 7fee8b000640 1 --2- 172.21.15.52:0/3950264031 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fee840cd150 0x7fee841f2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:53.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.006+0000 7fee8b000640 1 --2- 172.21.15.52:0/3950264031 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee840cd530 0x7fee8406b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:53.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.006+0000 7fee8b000640 1 --2- 172.21.15.52:0/3950264031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee840b0730 0x7fee841f4db0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:53.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.006+0000 7fee8b000640 1 -- 172.21.15.52:0/3950264031 >> 172.21.15.52:0/3950264031 conn(0x7fee840d0580 msgr2=0x7fee84157960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:53.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.006+0000 7fee8b000640 1 -- 172.21.15.52:0/3950264031 shutdown_connections 2024-03-12T17:07:53.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.006+0000 7fee8b000640 1 -- 172.21.15.52:0/3950264031 wait complete. 2024-03-12T17:07:53.022 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-12T17:07:53.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.262+0000 7f9325a00640 1 Processor -- start 2024-03-12T17:07:53.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.262+0000 7f9325a00640 1 -- start start 2024-03-12T17:07:53.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.263+0000 7f9325a00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f932015a850 0x7f93200cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:53.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.263+0000 7f9325a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f93200cacf0 0x7f93200cb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:53.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.264+0000 7f9325a00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f932015e320 con 0x7f93200b22b0 2024-03-12T17:07:53.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.264+0000 7f9325a00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93200cb6c0 con 0x7f932015a850 2024-03-12T17:07:53.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.264+0000 7f9325a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f932015ac30 con 0x7f93200cacf0 2024-03-12T17:07:53.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.264+0000 7f931fe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f932015a850 0x7f93200cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:53.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.264+0000 7f931fe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f932015a850 0x7f93200cd1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55896/0 (socket says 172.21.15.52:55896) 2024-03-12T17:07:53.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.264+0000 7f931fe00640 1 -- 172.21.15.52:0/4012679018 learned_addr learned my addr 172.21.15.52:0/4012679018 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:53.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.264+0000 7f931fe00640 1 -- 172.21.15.52:0/4012679018 >> v2:172.21.15.102:3300/0 conn(0x7f93200cacf0 msgr2=0x7f93200cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:53.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.264+0000 7f931fe00640 1 --2- 172.21.15.52:0/4012679018 >> v2:172.21.15.102:3300/0 conn(0x7f93200cacf0 0x7f93200cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:53.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.264+0000 7f931fe00640 1 -- 172.21.15.52:0/4012679018 >> v1:172.21.15.102:6789/0 conn(0x7f93200b22b0 legacy=0x7f932015eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:53.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.264+0000 7f931fe00640 1 -- 172.21.15.52:0/4012679018 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93200cda20 con 0x7f932015a850 2024-03-12T17:07:53.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.265+0000 7f931fe00640 1 --2- 172.21.15.52:0/4012679018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f932015a850 0x7f93200cd1d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f932006ad60 tx=0x7f931002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4310e1b5066228b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:53.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.266+0000 7f931ea00640 1 -- 172.21.15.52:0/4012679018 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931002ca80 con 0x7f932015a850 2024-03-12T17:07:53.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.266+0000 7f931ea00640 1 -- 172.21.15.52:0/4012679018 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931002cbe0 con 0x7f932015a850 2024-03-12T17:07:53.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.266+0000 7f931ea00640 1 -- 172.21.15.52:0/4012679018 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93100343f0 con 0x7f932015a850 2024-03-12T17:07:53.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.266+0000 7f9325a00640 1 -- 172.21.15.52:0/4012679018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f932015a850 msgr2=0x7f93200cd1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:53.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.266+0000 7f9325a00640 1 --2- 172.21.15.52:0/4012679018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f932015a850 0x7f93200cd1d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f932006ad60 tx=0x7f931002bee0 comp rx=0 tx=0).stop 2024-03-12T17:07:53.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.267+0000 7f9325a00640 1 -- 172.21.15.52:0/4012679018 shutdown_connections 2024-03-12T17:07:53.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.267+0000 7f9325a00640 1 --2- 172.21.15.52:0/4012679018 >> v2:172.21.15.102:3300/0 conn(0x7f93200cacf0 0x7f93200cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:53.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.267+0000 7f9325a00640 1 --2- 172.21.15.52:0/4012679018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f932015a850 0x7f93200cd1d0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:53.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.267+0000 7f9325a00640 1 -- 172.21.15.52:0/4012679018 >> 172.21.15.52:0/4012679018 conn(0x7f93200b01b0 msgr2=0x7f93200b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:53.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.267+0000 7f9325a00640 1 -- 172.21.15.52:0/4012679018 shutdown_connections 2024-03-12T17:07:53.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.267+0000 7f9325a00640 1 -- 172.21.15.52:0/4012679018 wait complete. 2024-03-12T17:07:53.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.269+0000 7f9325a00640 1 Processor -- start 2024-03-12T17:07:53.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.269+0000 7f9325a00640 1 -- start start 2024-03-12T17:07:53.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.270+0000 7f9325a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93200b22b0 0x7f932013c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:53.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.270+0000 7f9325a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93200cacf0 0x7f9320156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:53.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.270+0000 7f9325a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f932015a850 0x7f9320157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:53.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.270+0000 7f9325a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932014e8a0 con 0x7f93200cacf0 2024-03-12T17:07:53.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.270+0000 7f9325a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9320155cc0 con 0x7f93200b22b0 2024-03-12T17:07:53.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.270+0000 7f9325a00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9320155fa0 con 0x7f932015a850 2024-03-12T17:07:53.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.270+0000 7f931f400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93200cacf0 0x7f9320156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:53.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.270+0000 7f931f400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93200cacf0 0x7f9320156d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55910/0 (socket says 172.21.15.52:55910) 2024-03-12T17:07:53.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.270+0000 7f931f400640 1 -- 172.21.15.52:0/368195478 learned_addr learned my addr 172.21.15.52:0/368195478 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:53.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.271+0000 7f9325000640 1 --2- 172.21.15.52:0/368195478 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f932015a850 0x7f9320157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:53.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.271+0000 7f931fe00640 1 --2- 172.21.15.52:0/368195478 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93200b22b0 0x7f932013c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:53.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.271+0000 7f931f400640 1 -- 172.21.15.52:0/368195478 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f932015a850 msgr2=0x7f9320157280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:53.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.271+0000 7f931f400640 1 --2- 172.21.15.52:0/368195478 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f932015a850 0x7f9320157280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:53.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.271+0000 7f931f400640 1 -- 172.21.15.52:0/368195478 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93200b22b0 msgr2=0x7f932013c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:53.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.271+0000 7f931f400640 1 --2- 172.21.15.52:0/368195478 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93200b22b0 0x7f932013c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:53.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.271+0000 7f931f400640 1 -- 172.21.15.52:0/368195478 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9310009b30 con 0x7f93200cacf0 2024-03-12T17:07:53.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.271+0000 7f9325000640 1 --2- 172.21.15.52:0/368195478 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f932015a850 0x7f9320157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:07:53.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.271+0000 7f931f400640 1 --2- 172.21.15.52:0/368195478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93200cacf0 0x7f9320156d40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f930800b3e0 tx=0x7f930800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:53.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.271+0000 7f9317e00640 1 -- 172.21.15.52:0/368195478 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9308011830 con 0x7f93200cacf0 2024-03-12T17:07:53.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.271+0000 7f9317e00640 1 -- 172.21.15.52:0/368195478 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93080119b0 con 0x7f93200cacf0 2024-03-12T17:07:53.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.271+0000 7f9317e00640 1 -- 172.21.15.52:0/368195478 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9308011d60 con 0x7f93200cacf0 2024-03-12T17:07:53.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.272+0000 7f9325a00640 1 -- 172.21.15.52:0/368195478 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9320156280 con 0x7f93200cacf0 2024-03-12T17:07:53.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.272+0000 7f9325a00640 1 -- 172.21.15.52:0/368195478 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9320156840 con 0x7f93200cacf0 2024-03-12T17:07:53.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.273+0000 7f9325a00640 1 -- 172.21.15.52:0/368195478 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f932004e030 con 0x7f93200cacf0 2024-03-12T17:07:53.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.274+0000 7f9317e00640 1 -- 172.21.15.52:0/368195478 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f930801a400 con 0x7f93200cacf0 2024-03-12T17:07:53.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.275+0000 7f9317e00640 1 --2- 172.21.15.52:0/368195478 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f92fc06f010 0x7f92fc0714d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:53.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.275+0000 7f9317e00640 1 -- 172.21.15.52:0/368195478 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f93080899b0 con 0x7f93200cacf0 2024-03-12T17:07:53.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.275+0000 7f931fe00640 1 --2- 172.21.15.52:0/368195478 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f92fc06f010 0x7f92fc0714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:53.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.277+0000 7f931fe00640 1 --2- 172.21.15.52:0/368195478 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f92fc06f010 0x7f92fc0714d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9310035000 tx=0x7f9310030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:53.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.280+0000 7f9317e00640 1 -- 172.21.15.52:0/368195478 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f93080546e0 con 0x7f93200cacf0 2024-03-12T17:07:53.471 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-12T17:07:53.480 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-12T17:07:53.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:53.482+0000 7f9325a00640 1 -- 172.21.15.52:0/368195478 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f93200ccaa0 con 0x7f93200cacf0 2024-03-12T17:07:54.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.219+0000 7f9317e00640 1 -- 172.21.15.52:0/368195478 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) v1 ==== 181+0+0 (secure 0 0 0) 0x7f93080150d0 con 0x7f93200cacf0 2024-03-12T17:07:54.212 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_metadata' removed 2024-03-12T17:07:54.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.223+0000 7f9325a00640 1 -- 172.21.15.52:0/368195478 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f92fc06f010 msgr2=0x7f92fc0714d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:54.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.223+0000 7f9325a00640 1 --2- 172.21.15.52:0/368195478 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f92fc06f010 0x7f92fc0714d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9310035000 tx=0x7f9310030000 comp rx=0 tx=0).stop 2024-03-12T17:07:54.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.223+0000 7f9325a00640 1 -- 172.21.15.52:0/368195478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93200cacf0 msgr2=0x7f9320156d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:54.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.223+0000 7f9325a00640 1 --2- 172.21.15.52:0/368195478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93200cacf0 0x7f9320156d40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f930800b3e0 tx=0x7f930800b8a0 comp rx=0 tx=0).stop 2024-03-12T17:07:54.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.224+0000 7f9325a00640 1 -- 172.21.15.52:0/368195478 shutdown_connections 2024-03-12T17:07:54.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.224+0000 7f9325a00640 1 --2- 172.21.15.52:0/368195478 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f92fc06f010 0x7f92fc0714d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:54.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.224+0000 7f9325a00640 1 --2- 172.21.15.52:0/368195478 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f932015a850 0x7f9320157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:54.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.224+0000 7f9325a00640 1 --2- 172.21.15.52:0/368195478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93200cacf0 0x7f9320156d40 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:54.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.224+0000 7f9325a00640 1 --2- 172.21.15.52:0/368195478 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93200b22b0 0x7f932013c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:54.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.224+0000 7f9325a00640 1 -- 172.21.15.52:0/368195478 >> 172.21.15.52:0/368195478 conn(0x7f93200b01b0 msgr2=0x7f93200b0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:54.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.224+0000 7f9325a00640 1 -- 172.21.15.52:0/368195478 shutdown_connections 2024-03-12T17:07:54.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.224+0000 7f9325a00640 1 -- 172.21.15.52:0/368195478 wait complete. 2024-03-12T17:07:54.242 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-12T17:07:54.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.477+0000 7fa2c7c00640 1 Processor -- start 2024-03-12T17:07:54.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.477+0000 7fa2c7c00640 1 -- start start 2024-03-12T17:07:54.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.478+0000 7fa2c7c00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa2c00ce930 0x7fa2c00cedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:54.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.478+0000 7fa2c7c00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c0159c10 con 0x7fa2c0159830 2024-03-12T17:07:54.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.479+0000 7fa2c7c00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c00cd840 con 0x7fa2c00ca7b0 2024-03-12T17:07:54.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.479+0000 7fa2c7c00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2c006de50 con 0x7fa2c00ce930 2024-03-12T17:07:54.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.479+0000 7fa2c5c00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa2c00ce930 0x7fa2c00cedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:54.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.479+0000 7fa2c5200640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fa2c0159830 0x7fa2c00cc3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:46468/0 (socket says 172.21.15.52:46468) 2024-03-12T17:07:54.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.479+0000 7fa2c5c00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa2c00ce930 0x7fa2c00cedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55916/0 (socket says 172.21.15.52:55916) 2024-03-12T17:07:54.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.479+0000 7fa2c5200640 1 -- 172.21.15.52:0/105417743 learned_addr learned my addr 172.21.15.52:0/105417743 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:07:54.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.479+0000 7fa2c5c00640 1 -- 172.21.15.52:0/105417743 >> v1:172.21.15.102:6789/0 conn(0x7fa2c00ca7b0 legacy=0x7fa2c00cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:54.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.479+0000 7fa2c5c00640 1 -- 172.21.15.52:0/105417743 >> v1:172.21.15.52:6789/0 conn(0x7fa2c0159830 legacy=0x7fa2c00cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:54.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.479+0000 7fa2c5c00640 1 -- 172.21.15.52:0/105417743 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c00ccc30 con 0x7fa2c00ce930 2024-03-12T17:07:54.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.479+0000 7fa2bf400640 1 -- 172.21.15.52:0/105417743 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1228950302 0 0) 0x7fa2c0159c10 con 0x7fa2c0159830 2024-03-12T17:07:54.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.481+0000 7fa2c5c00640 1 --2- 172.21.15.52:0/105417743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2c00ce930 0x7fa2c00cedc0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8009dc0 tx=0x7fa2b802bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20961d763c788ad5 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:54.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.481+0000 7fa2bf400640 1 -- 172.21.15.52:0/105417743 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b802ccd0 con 0x7fa2c00ce930 2024-03-12T17:07:54.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.481+0000 7fa2bf400640 1 -- 172.21.15.52:0/105417743 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b802ce30 con 0x7fa2c00ce930 2024-03-12T17:07:54.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.481+0000 7fa2bf400640 1 -- 172.21.15.52:0/105417743 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8034660 con 0x7fa2c00ce930 2024-03-12T17:07:54.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.482+0000 7fa2c7c00640 1 -- 172.21.15.52:0/105417743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2c00ce930 msgr2=0x7fa2c00cedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:54.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.482+0000 7fa2c7c00640 1 --2- 172.21.15.52:0/105417743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2c00ce930 0x7fa2c00cedc0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8009dc0 tx=0x7fa2b802bf20 comp rx=0 tx=0).stop 2024-03-12T17:07:54.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.482+0000 7fa2c7c00640 1 -- 172.21.15.52:0/105417743 shutdown_connections 2024-03-12T17:07:54.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.482+0000 7fa2c7c00640 1 --2- 172.21.15.52:0/105417743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2c00ce930 0x7fa2c00cedc0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:54.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.482+0000 7fa2c7c00640 1 -- 172.21.15.52:0/105417743 >> 172.21.15.52:0/105417743 conn(0x7fa2c006a0b0 msgr2=0x7fa2c006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:54.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.482+0000 7fa2c7c00640 1 -- 172.21.15.52:0/105417743 shutdown_connections 2024-03-12T17:07:54.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.482+0000 7fa2c7c00640 1 -- 172.21.15.52:0/105417743 wait complete. 2024-03-12T17:07:54.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.484+0000 7fa2c7c00640 1 Processor -- start 2024-03-12T17:07:54.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.484+0000 7fa2c7c00640 1 -- start start 2024-03-12T17:07:54.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.485+0000 7fa2c7c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2c00ca7b0 0x7fa2c0140dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:54.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.485+0000 7fa2c7c00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa2c00ce930 0x7fa2c00afac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:54.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.485+0000 7fa2c7c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2c0159830 0x7fa2c00b0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:54.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.485+0000 7fa2c7c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c0141300 con 0x7fa2c0159830 2024-03-12T17:07:54.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.485+0000 7fa2c7c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c01f2110 con 0x7fa2c00ca7b0 2024-03-12T17:07:54.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.485+0000 7fa2c7c00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c01f2380 con 0x7fa2c00ce930 2024-03-12T17:07:54.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.485+0000 7fa2c5c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2c0159830 0x7fa2c00b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:54.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.485+0000 7fa2c5c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2c0159830 0x7fa2c00b0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55930/0 (socket says 172.21.15.52:55930) 2024-03-12T17:07:54.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.485+0000 7fa2c5c00640 1 -- 172.21.15.52:0/184927764 learned_addr learned my addr 172.21.15.52:0/184927764 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:54.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.485+0000 7fa2c5200640 1 --2- 172.21.15.52:0/184927764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2c00ca7b0 0x7fa2c0140dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:54.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.485+0000 7fa2bfe00640 1 --2- 172.21.15.52:0/184927764 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa2c00ce930 0x7fa2c00afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:54.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.485+0000 7fa2c5c00640 1 -- 172.21.15.52:0/184927764 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa2c00ce930 msgr2=0x7fa2c00afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:54.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.485+0000 7fa2c5c00640 1 --2- 172.21.15.52:0/184927764 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa2c00ce930 0x7fa2c00afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:54.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.485+0000 7fa2c5c00640 1 -- 172.21.15.52:0/184927764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2c00ca7b0 msgr2=0x7fa2c0140dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:54.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.485+0000 7fa2c5c00640 1 --2- 172.21.15.52:0/184927764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2c00ca7b0 0x7fa2c0140dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:54.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.485+0000 7fa2c5c00640 1 -- 172.21.15.52:0/184927764 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b8009a70 con 0x7fa2c0159830 2024-03-12T17:07:54.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.486+0000 7fa2c5c00640 1 --2- 172.21.15.52:0/184927764 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2c0159830 0x7fa2c00b0000 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa2b802c530 tx=0x7fa2b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:54.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.486+0000 7fa2bcc00640 1 -- 172.21.15.52:0/184927764 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8003b90 con 0x7fa2c0159830 2024-03-12T17:07:54.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.486+0000 7fa2bcc00640 1 -- 172.21.15.52:0/184927764 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b8003cf0 con 0x7fa2c0159830 2024-03-12T17:07:54.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.486+0000 7fa2bcc00640 1 -- 172.21.15.52:0/184927764 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8004260 con 0x7fa2c0159830 2024-03-12T17:07:54.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.486+0000 7fa2c7c00640 1 -- 172.21.15.52:0/184927764 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c01f2660 con 0x7fa2c0159830 2024-03-12T17:07:54.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.486+0000 7fa2c7c00640 1 -- 172.21.15.52:0/184927764 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c0144490 con 0x7fa2c0159830 2024-03-12T17:07:54.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.488+0000 7fa2c7c00640 1 -- 172.21.15.52:0/184927764 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2c0073390 con 0x7fa2c0159830 2024-03-12T17:07:54.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.488+0000 7fa2bcc00640 1 -- 172.21.15.52:0/184927764 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa2b803b040 con 0x7fa2c0159830 2024-03-12T17:07:54.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.489+0000 7fa2bcc00640 1 --2- 172.21.15.52:0/184927764 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa2a006ac00 0x7fa2a006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:54.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.489+0000 7fa2bcc00640 1 -- 172.21.15.52:0/184927764 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6311+0+0 (secure 0 0 0) 0x7fa2b80ac3e0 con 0x7fa2c0159830 2024-03-12T17:07:54.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.490+0000 7fa2c5200640 1 --2- 172.21.15.52:0/184927764 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa2a006ac00 0x7fa2a006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:54.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.491+0000 7fa2c5200640 1 --2- 172.21.15.52:0/184927764 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa2a006ac00 0x7fa2a006d0c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0000f60 tx=0x7fa2b0007c00 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:54.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.494+0000 7fa2bcc00640 1 -- 172.21.15.52:0/184927764 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fa2b8077510 con 0x7fa2c0159830 2024-03-12T17:07:54.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:54.694+0000 7fa2c7c00640 1 -- 172.21.15.52:0/184927764 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa2c01f2b20 con 0x7fa2c0159830 2024-03-12T17:07:55.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.233+0000 7fa2bcc00640 1 -- 172.21.15.52:0/184927764 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa2b807ab30 con 0x7fa2c0159830 2024-03-12T17:07:55.229 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_data' removed 2024-03-12T17:07:55.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.240+0000 7fa2c7c00640 1 -- 172.21.15.52:0/184927764 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa2a006ac00 msgr2=0x7fa2a006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:55.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.241+0000 7fa2c7c00640 1 --2- 172.21.15.52:0/184927764 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa2a006ac00 0x7fa2a006d0c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0000f60 tx=0x7fa2b0007c00 comp rx=0 tx=0).stop 2024-03-12T17:07:55.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.241+0000 7fa2c7c00640 1 -- 172.21.15.52:0/184927764 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2c0159830 msgr2=0x7fa2c00b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:55.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.241+0000 7fa2c7c00640 1 --2- 172.21.15.52:0/184927764 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2c0159830 0x7fa2c00b0000 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa2b802c530 tx=0x7fa2b8004a40 comp rx=0 tx=0).stop 2024-03-12T17:07:55.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.241+0000 7fa2c7c00640 1 -- 172.21.15.52:0/184927764 shutdown_connections 2024-03-12T17:07:55.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.241+0000 7fa2c7c00640 1 --2- 172.21.15.52:0/184927764 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa2a006ac00 0x7fa2a006d0c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:55.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.241+0000 7fa2c7c00640 1 --2- 172.21.15.52:0/184927764 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa2c00ce930 0x7fa2c00afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:55.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.241+0000 7fa2c7c00640 1 --2- 172.21.15.52:0/184927764 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2c0159830 0x7fa2c00b0000 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:55.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.241+0000 7fa2c7c00640 1 --2- 172.21.15.52:0/184927764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2c00ca7b0 0x7fa2c0140dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:55.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.241+0000 7fa2c7c00640 1 -- 172.21.15.52:0/184927764 >> 172.21.15.52:0/184927764 conn(0x7fa2c006a0b0 msgr2=0x7fa2c0070150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:55.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.242+0000 7fa2c7c00640 1 -- 172.21.15.52:0/184927764 shutdown_connections 2024-03-12T17:07:55.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.242+0000 7fa2c7c00640 1 -- 172.21.15.52:0/184927764 wait complete. 2024-03-12T17:07:55.260 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-03-12T17:07:55.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.489+0000 7fd56d800640 1 Processor -- start 2024-03-12T17:07:55.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.489+0000 7fd56d800640 1 -- start start 2024-03-12T17:07:55.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.490+0000 7fd56d800640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd5680cc7d0 0x7fd5680b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:55.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.490+0000 7fd56d800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd5680b1410 0x7fd5680c3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:55.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.490+0000 7fd56d800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd5680c3a50 0x7fd568164830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:55.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.490+0000 7fd56d800640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5680c3ee0 con 0x7fd5680cc7d0 2024-03-12T17:07:55.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.490+0000 7fd56d800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd56806ada0 con 0x7fd5680b1410 2024-03-12T17:07:55.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.490+0000 7fd56d800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5680b1e70 con 0x7fd5680c3a50 2024-03-12T17:07:55.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.491+0000 7fd567e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd5680cc7d0 0x7fd5680b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:55.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.491+0000 7fd567e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd5680cc7d0 0x7fd5680b1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60714/0 (socket says 172.21.15.52:60714) 2024-03-12T17:07:55.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.491+0000 7fd567e00640 1 -- 172.21.15.52:0/955595273 learned_addr learned my addr 172.21.15.52:0/955595273 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:55.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.491+0000 7fd567400640 1 --2- 172.21.15.52:0/955595273 >> v2:172.21.15.102:3300/0 conn(0x7fd5680b1410 0x7fd5680c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:55.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.491+0000 7fd56ce00640 1 --2- 172.21.15.52:0/955595273 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd5680c3a50 0x7fd568164830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:55.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.491+0000 7fd567e00640 1 -- 172.21.15.52:0/955595273 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd5680c3a50 msgr2=0x7fd568164830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:55.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.491+0000 7fd567e00640 1 --2- 172.21.15.52:0/955595273 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd5680c3a50 0x7fd568164830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:55.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.492+0000 7fd567e00640 1 -- 172.21.15.52:0/955595273 >> v2:172.21.15.102:3300/0 conn(0x7fd5680b1410 msgr2=0x7fd5680c3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:07:55.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.492+0000 7fd567e00640 1 --2- 172.21.15.52:0/955595273 >> v2:172.21.15.102:3300/0 conn(0x7fd5680b1410 0x7fd5680c3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:55.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.492+0000 7fd567e00640 1 -- 172.21.15.52:0/955595273 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5681645f0 con 0x7fd5680cc7d0 2024-03-12T17:07:55.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.493+0000 7fd567e00640 1 --2- 172.21.15.52:0/955595273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5680cc7d0 0x7fd5680b1930 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd56806db40 tx=0x7fd55802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9103cc26dc935891 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:55.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.493+0000 7fd566a00640 1 -- 172.21.15.52:0/955595273 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd558031870 con 0x7fd5680cc7d0 2024-03-12T17:07:55.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.493+0000 7fd566a00640 1 -- 172.21.15.52:0/955595273 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5580319d0 con 0x7fd5680cc7d0 2024-03-12T17:07:55.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.493+0000 7fd566a00640 1 -- 172.21.15.52:0/955595273 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd558031d60 con 0x7fd5680cc7d0 2024-03-12T17:07:55.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.494+0000 7fd56d800640 1 -- 172.21.15.52:0/955595273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5680cc7d0 msgr2=0x7fd5680b1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:55.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.494+0000 7fd56d800640 1 --2- 172.21.15.52:0/955595273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5680cc7d0 0x7fd5680b1930 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd56806db40 tx=0x7fd55802b910 comp rx=0 tx=0).stop 2024-03-12T17:07:55.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.494+0000 7fd56d800640 1 -- 172.21.15.52:0/955595273 shutdown_connections 2024-03-12T17:07:55.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.494+0000 7fd56d800640 1 --2- 172.21.15.52:0/955595273 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd5680c3a50 0x7fd568164830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:55.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.494+0000 7fd56d800640 1 --2- 172.21.15.52:0/955595273 >> v2:172.21.15.102:3300/0 conn(0x7fd5680b1410 0x7fd5680c3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:55.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.494+0000 7fd56d800640 1 --2- 172.21.15.52:0/955595273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5680cc7d0 0x7fd5680b1930 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:55.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.494+0000 7fd56d800640 1 -- 172.21.15.52:0/955595273 >> 172.21.15.52:0/955595273 conn(0x7fd5680d0540 msgr2=0x7fd5680d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:55.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.494+0000 7fd56d800640 1 -- 172.21.15.52:0/955595273 shutdown_connections 2024-03-12T17:07:55.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.494+0000 7fd56d800640 1 -- 172.21.15.52:0/955595273 wait complete. 2024-03-12T17:07:55.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.496+0000 7fd56d800640 1 Processor -- start 2024-03-12T17:07:55.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.496+0000 7fd56d800640 1 -- start start 2024-03-12T17:07:55.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.497+0000 7fd56d800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5680b1410 0x7fd5680c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:55.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.497+0000 7fd56d800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5680c3a50 0x7fd5680c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:55.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.497+0000 7fd56d800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd5680cc7d0 0x7fd5681586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:55.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.497+0000 7fd56d800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5680c82b0 con 0x7fd5680c3a50 2024-03-12T17:07:55.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.497+0000 7fd56d800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5680c8590 con 0x7fd5680b1410 2024-03-12T17:07:55.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.497+0000 7fd56d800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5680c8870 con 0x7fd5680cc7d0 2024-03-12T17:07:55.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.497+0000 7fd567400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5680c3a50 0x7fd5680c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:55.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.497+0000 7fd567400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5680c3a50 0x7fd5680c5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60724/0 (socket says 172.21.15.52:60724) 2024-03-12T17:07:55.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.497+0000 7fd567400640 1 -- 172.21.15.52:0/316905079 learned_addr learned my addr 172.21.15.52:0/316905079 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:07:55.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.497+0000 7fd567e00640 1 --2- 172.21.15.52:0/316905079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5680b1410 0x7fd5680c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:55.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.497+0000 7fd56ce00640 1 --2- 172.21.15.52:0/316905079 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd5680cc7d0 0x7fd5681586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:55.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.498+0000 7fd567400640 1 -- 172.21.15.52:0/316905079 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd5680cc7d0 msgr2=0x7fd5681586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:55.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.498+0000 7fd567400640 1 --2- 172.21.15.52:0/316905079 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd5680cc7d0 0x7fd5681586c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:55.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.498+0000 7fd567400640 1 -- 172.21.15.52:0/316905079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5680b1410 msgr2=0x7fd5680c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:55.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.498+0000 7fd567400640 1 --2- 172.21.15.52:0/316905079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5680b1410 0x7fd5680c9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:55.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.498+0000 7fd567400640 1 -- 172.21.15.52:0/316905079 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5580094d0 con 0x7fd5680c3a50 2024-03-12T17:07:55.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.498+0000 7fd567400640 1 --2- 172.21.15.52:0/316905079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5680c3a50 0x7fd5680c5ef0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd55000e870 tx=0x7fd55000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:55.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.498+0000 7fd55fe00640 1 -- 172.21.15.52:0/316905079 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55000ee60 con 0x7fd5680c3a50 2024-03-12T17:07:55.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.498+0000 7fd55fe00640 1 -- 172.21.15.52:0/316905079 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd55000c940 con 0x7fd5680c3a50 2024-03-12T17:07:55.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.498+0000 7fd55fe00640 1 -- 172.21.15.52:0/316905079 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55000ccf0 con 0x7fd5680c3a50 2024-03-12T17:07:55.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.498+0000 7fd56d800640 1 -- 172.21.15.52:0/316905079 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5680c8b50 con 0x7fd5680c3a50 2024-03-12T17:07:55.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.499+0000 7fd56d800640 1 -- 172.21.15.52:0/316905079 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5680c6430 con 0x7fd5680c3a50 2024-03-12T17:07:55.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.500+0000 7fd56d800640 1 -- 172.21.15.52:0/316905079 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5680cceb0 con 0x7fd5680c3a50 2024-03-12T17:07:55.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.501+0000 7fd55fe00640 1 -- 172.21.15.52:0/316905079 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd550013470 con 0x7fd5680c3a50 2024-03-12T17:07:55.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.502+0000 7fd55fe00640 1 --2- 172.21.15.52:0/316905079 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd54406ac00 0x7fd54406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:07:55.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.502+0000 7fd55fe00640 1 -- 172.21.15.52:0/316905079 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5781+0+0 (secure 0 0 0) 0x7fd5500894d0 con 0x7fd5680c3a50 2024-03-12T17:07:55.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.503+0000 7fd567e00640 1 --2- 172.21.15.52:0/316905079 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd54406ac00 0x7fd54406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:07:55.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.504+0000 7fd567e00640 1 --2- 172.21.15.52:0/316905079 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd54406ac00 0x7fd54406d0c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd558036000 tx=0x7fd55803c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:07:55.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.507+0000 7fd55fe00640 1 -- 172.21.15.52:0/316905079 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fd5500545c0 con 0x7fd5680c3a50 2024-03-12T17:07:55.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:55.704+0000 7fd56d800640 1 -- 172.21.15.52:0/316905079 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd56804e030 con 0x7fd5680c3a50 2024-03-12T17:07:56.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:56.235+0000 7fd55fe00640 1 -- 172.21.15.52:0/316905079 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) v1 ==== 178+0+0 (secure 0 0 0) 0x7fd550057be0 con 0x7fd5680c3a50 2024-03-12T17:07:56.230 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_data_ec' removed 2024-03-12T17:07:56.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:56.240+0000 7fd56d800640 1 -- 172.21.15.52:0/316905079 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd54406ac00 msgr2=0x7fd54406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:56.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:56.240+0000 7fd56d800640 1 --2- 172.21.15.52:0/316905079 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd54406ac00 0x7fd54406d0c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd558036000 tx=0x7fd55803c040 comp rx=0 tx=0).stop 2024-03-12T17:07:56.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:56.240+0000 7fd56d800640 1 -- 172.21.15.52:0/316905079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5680c3a50 msgr2=0x7fd5680c5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:07:56.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:56.240+0000 7fd56d800640 1 --2- 172.21.15.52:0/316905079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5680c3a50 0x7fd5680c5ef0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd55000e870 tx=0x7fd55000ed30 comp rx=0 tx=0).stop 2024-03-12T17:07:56.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:56.240+0000 7fd56d800640 1 -- 172.21.15.52:0/316905079 shutdown_connections 2024-03-12T17:07:56.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:56.240+0000 7fd56d800640 1 --2- 172.21.15.52:0/316905079 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd54406ac00 0x7fd54406d0c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:56.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:56.241+0000 7fd56d800640 1 --2- 172.21.15.52:0/316905079 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd5680cc7d0 0x7fd5681586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:56.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:56.241+0000 7fd56d800640 1 --2- 172.21.15.52:0/316905079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5680c3a50 0x7fd5680c5ef0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:56.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:56.241+0000 7fd56d800640 1 --2- 172.21.15.52:0/316905079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5680b1410 0x7fd5680c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:07:56.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:56.241+0000 7fd56d800640 1 -- 172.21.15.52:0/316905079 >> 172.21.15.52:0/316905079 conn(0x7fd5680d0540 msgr2=0x7fd5680af240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:07:56.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:56.241+0000 7fd56d800640 1 -- 172.21.15.52:0/316905079 shutdown_connections 2024-03-12T17:07:56.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:07:56.241+0000 7fd56d800640 1 -- 172.21.15.52:0/316905079 wait complete. 2024-03-12T17:07:56.258 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-12T17:07:56.259 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-12T17:07:56.259 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-12T17:07:56.259 INFO:teuthology.orchestra.run:waiting for 300 2024-03-12T17:07:56.259 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-12T17:07:56.259 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-12T17:07:56.259 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-12T17:07:56.259 INFO:teuthology.orchestra.run:waiting for 300 2024-03-12T17:07:56.260 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-12T17:07:56.260 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-12T17:07:56.260 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-12T17:07:56.260 INFO:teuthology.orchestra.run:waiting for 300 2024-03-12T17:07:56.260 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-12T17:07:56.260 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-12T17:07:56.261 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-12T17:07:56.261 INFO:teuthology.orchestra.run:waiting for 300 2024-03-12T17:07:56.278 INFO:tasks.ceph.mds.c:Stopped 2024-03-12T17:07:56.278 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-12T17:07:56.282 INFO:tasks.ceph.mds.d:Stopped 2024-03-12T17:07:56.282 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-12T17:07:56.284 INFO:tasks.ceph.mds.b:Stopped 2024-03-12T17:07:56.284 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-12T17:07:56.326 INFO:tasks.ceph.mds.c:Started 2024-03-12T17:07:56.326 DEBUG:teuthology.parallel:result is None 2024-03-12T17:07:56.332 INFO:tasks.ceph.mds.d:Started 2024-03-12T17:07:56.332 DEBUG:teuthology.parallel:result is None 2024-03-12T17:07:56.338 INFO:tasks.ceph.mds.b:Started 2024-03-12T17:07:56.339 DEBUG:teuthology.parallel:result is None 2024-03-12T17:07:56.490 INFO:tasks.ceph.mds.b.smithi102.stdout:starting mds.b at 2024-03-12T17:07:56.490 INFO:tasks.ceph.mds.d.smithi102.stdout:starting mds.d at 2024-03-12T17:07:56.503 INFO:tasks.ceph.mds.c.smithi052.stdout:starting mds.c at 2024-03-12T17:07:58.106 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-03-12T17:08:02.278 INFO:tasks.ceph.mds.a:Stopped 2024-03-12T17:08:02.278 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-12T17:08:02.286 INFO:tasks.ceph.mds.a:Started 2024-03-12T17:08:02.287 DEBUG:teuthology.parallel:result is None 2024-03-12T17:08:02.287 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-12T17:08:02.460 INFO:tasks.ceph.mds.a.smithi052.stdout:starting mds.a at 2024-03-12T17:08:02.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.490+0000 7fd92f400640 1 Processor -- start 2024-03-12T17:08:02.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.490+0000 7fd92f400640 1 -- start start 2024-03-12T17:08:02.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.492+0000 7fd92f400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd92815b560 0x7fd92815b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:02.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.492+0000 7fd92f400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9280b39b0 0x7fd9280b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:02.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.492+0000 7fd92f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd92815e500 0x7fd92815e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:02.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.492+0000 7fd92f400640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9280b4370 con 0x7fd92815b560 2024-03-12T17:08:02.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.492+0000 7fd92f400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd92806c6f0 con 0x7fd92815e500 2024-03-12T17:08:02.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.492+0000 7fd92f400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd92806c9d0 con 0x7fd9280b39b0 2024-03-12T17:08:02.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.492+0000 7fd92ca00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd92815b560 0x7fd92815b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:02.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.492+0000 7fd92ca00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd92815b560 0x7fd92815b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60750/0 (socket says 172.21.15.52:60750) 2024-03-12T17:08:02.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.492+0000 7fd92ca00640 1 -- 172.21.15.52:0/820875342 learned_addr learned my addr 172.21.15.52:0/820875342 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:02.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.492+0000 7fd92ca00640 1 -- 172.21.15.52:0/820875342 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9280b39b0 msgr2=0x7fd9280b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:02.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.492+0000 7fd92ca00640 1 --2- 172.21.15.52:0/820875342 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9280b39b0 0x7fd9280b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:02.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.492+0000 7fd92ca00640 1 -- 172.21.15.52:0/820875342 >> v2:172.21.15.102:3300/0 conn(0x7fd92815e500 msgr2=0x7fd92815e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:02.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.492+0000 7fd92ca00640 1 --2- 172.21.15.52:0/820875342 >> v2:172.21.15.102:3300/0 conn(0x7fd92815e500 0x7fd92815e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:02.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.492+0000 7fd92ca00640 1 -- 172.21.15.52:0/820875342 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92806ccb0 con 0x7fd92815b560 2024-03-12T17:08:02.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.494+0000 7fd92ca00640 1 --2- 172.21.15.52:0/820875342 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd92815b560 0x7fd92815b940 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd9280cab40 tx=0x7fd91802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4752c9547af5bfaf server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:02.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.494+0000 7fd927400640 1 -- 172.21.15.52:0/820875342 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd918031860 con 0x7fd92815b560 2024-03-12T17:08:02.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.494+0000 7fd927400640 1 -- 172.21.15.52:0/820875342 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9180319c0 con 0x7fd92815b560 2024-03-12T17:08:02.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.494+0000 7fd927400640 1 -- 172.21.15.52:0/820875342 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd918031d50 con 0x7fd92815b560 2024-03-12T17:08:02.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.494+0000 7fd92f400640 1 -- 172.21.15.52:0/820875342 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd92815b560 msgr2=0x7fd92815b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:02.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.494+0000 7fd92f400640 1 --2- 172.21.15.52:0/820875342 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd92815b560 0x7fd92815b940 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd9280cab40 tx=0x7fd91802b900 comp rx=0 tx=0).stop 2024-03-12T17:08:02.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.495+0000 7fd92f400640 1 -- 172.21.15.52:0/820875342 shutdown_connections 2024-03-12T17:08:02.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.495+0000 7fd92f400640 1 --2- 172.21.15.52:0/820875342 >> v2:172.21.15.102:3300/0 conn(0x7fd92815e500 0x7fd92815e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:02.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.495+0000 7fd92f400640 1 --2- 172.21.15.52:0/820875342 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9280b39b0 0x7fd9280b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:02.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.495+0000 7fd92f400640 1 --2- 172.21.15.52:0/820875342 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd92815b560 0x7fd92815b940 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:02.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.495+0000 7fd92f400640 1 -- 172.21.15.52:0/820875342 >> 172.21.15.52:0/820875342 conn(0x7fd92806a3d0 msgr2=0x7fd9280b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:02.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.495+0000 7fd92f400640 1 -- 172.21.15.52:0/820875342 shutdown_connections 2024-03-12T17:08:02.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.495+0000 7fd92f400640 1 -- 172.21.15.52:0/820875342 wait complete. 2024-03-12T17:08:02.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.496+0000 7fd92f400640 1 Processor -- start 2024-03-12T17:08:02.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.497+0000 7fd92f400640 1 -- start start 2024-03-12T17:08:02.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.498+0000 7fd92f400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9280b39b0 0x7fd928152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:02.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.498+0000 7fd92f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd92815b560 0x7fd9281f5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:02.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.498+0000 7fd92f400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd92815e500 0x7fd9281f6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:02.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.498+0000 7fd92f400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9281534b0 con 0x7fd92815e500 2024-03-12T17:08:02.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.498+0000 7fd92f400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd928153790 con 0x7fd92815b560 2024-03-12T17:08:02.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.498+0000 7fd92f400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd928153a70 con 0x7fd9280b39b0 2024-03-12T17:08:02.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.498+0000 7fd92d400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd92815e500 0x7fd9281f6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:02.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.498+0000 7fd92d400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd92815e500 0x7fd9281f6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60762/0 (socket says 172.21.15.52:60762) 2024-03-12T17:08:02.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.498+0000 7fd92d400640 1 -- 172.21.15.52:0/3688979346 learned_addr learned my addr 172.21.15.52:0/3688979346 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:02.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.498+0000 7fd92ca00640 1 --2- 172.21.15.52:0/3688979346 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9280b39b0 0x7fd928152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:02.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.498+0000 7fd927e00640 1 --2- 172.21.15.52:0/3688979346 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd92815b560 0x7fd9281f5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:02.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.498+0000 7fd92d400640 1 -- 172.21.15.52:0/3688979346 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9280b39b0 msgr2=0x7fd928152eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:02.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.498+0000 7fd92d400640 1 --2- 172.21.15.52:0/3688979346 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9280b39b0 0x7fd928152eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:02.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.498+0000 7fd92d400640 1 -- 172.21.15.52:0/3688979346 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd92815b560 msgr2=0x7fd9281f5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:02.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.498+0000 7fd92d400640 1 --2- 172.21.15.52:0/3688979346 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd92815b560 0x7fd9281f5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:02.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.498+0000 7fd92d400640 1 -- 172.21.15.52:0/3688979346 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9180094d0 con 0x7fd92815e500 2024-03-12T17:08:02.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.498+0000 7fd927e00640 1 --2- 172.21.15.52:0/3688979346 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd92815b560 0x7fd9281f5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:08:02.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.499+0000 7fd92d400640 1 --2- 172.21.15.52:0/3688979346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd92815e500 0x7fd9281f6050 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd92000b410 tx=0x7fd92000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:02.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.499+0000 7fd924c00640 1 -- 172.21.15.52:0/3688979346 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd920011830 con 0x7fd92815e500 2024-03-12T17:08:02.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.499+0000 7fd924c00640 1 -- 172.21.15.52:0/3688979346 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9200119b0 con 0x7fd92815e500 2024-03-12T17:08:02.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.499+0000 7fd924c00640 1 -- 172.21.15.52:0/3688979346 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd920011d60 con 0x7fd92815e500 2024-03-12T17:08:02.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.499+0000 7fd92f400640 1 -- 172.21.15.52:0/3688979346 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd928153d50 con 0x7fd92815e500 2024-03-12T17:08:02.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.499+0000 7fd92f400640 1 -- 172.21.15.52:0/3688979346 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9280b36b0 con 0x7fd92815e500 2024-03-12T17:08:02.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.501+0000 7fd92f400640 1 -- 172.21.15.52:0/3688979346 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd928154270 con 0x7fd92815e500 2024-03-12T17:08:02.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.501+0000 7fd924c00640 1 -- 172.21.15.52:0/3688979346 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd92001a400 con 0x7fd92815e500 2024-03-12T17:08:02.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.502+0000 7fd924c00640 1 --2- 172.21.15.52:0/3688979346 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd90806ac00 0x7fd90806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:02.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.502+0000 7fd924c00640 1 -- 172.21.15.52:0/3688979346 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fd920089260 con 0x7fd92815e500 2024-03-12T17:08:02.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.503+0000 7fd92ca00640 1 --2- 172.21.15.52:0/3688979346 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd90806ac00 0x7fd90806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:02.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.504+0000 7fd92ca00640 1 --2- 172.21.15.52:0/3688979346 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd90806ac00 0x7fd90806d0c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd918036000 tx=0x7fd91803c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:02.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.507+0000 7fd924c00640 1 -- 172.21.15.52:0/3688979346 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fd928154270 con 0x7fd92815e500 2024-03-12T17:08:02.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:02.721+0000 7fd92f400640 1 -- 172.21.15.52:0/3688979346 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd928154270 con 0x7fd92815e500 2024-03-12T17:08:03.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.491+0000 7fd924c00640 1 -- 172.21.15.52:0/3688979346 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v36) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd928154270 con 0x7fd92815e500 2024-03-12T17:08:03.486 INFO:teuthology.orchestra.run.smithi052.stderr: removed all blocklist entries 2024-03-12T17:08:03.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.495+0000 7fd92f400640 1 -- 172.21.15.52:0/3688979346 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd90806ac00 msgr2=0x7fd90806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:03.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.495+0000 7fd92f400640 1 --2- 172.21.15.52:0/3688979346 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd90806ac00 0x7fd90806d0c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd918036000 tx=0x7fd91803c040 comp rx=0 tx=0).stop 2024-03-12T17:08:03.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.495+0000 7fd92f400640 1 -- 172.21.15.52:0/3688979346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd92815e500 msgr2=0x7fd9281f6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:03.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.495+0000 7fd92f400640 1 --2- 172.21.15.52:0/3688979346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd92815e500 0x7fd9281f6050 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd92000b410 tx=0x7fd92000b8d0 comp rx=0 tx=0).stop 2024-03-12T17:08:03.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.496+0000 7fd92f400640 1 -- 172.21.15.52:0/3688979346 shutdown_connections 2024-03-12T17:08:03.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.496+0000 7fd92f400640 1 --2- 172.21.15.52:0/3688979346 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd90806ac00 0x7fd90806d0c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:03.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.496+0000 7fd92f400640 1 --2- 172.21.15.52:0/3688979346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd92815e500 0x7fd9281f6050 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:03.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.496+0000 7fd92f400640 1 --2- 172.21.15.52:0/3688979346 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd92815b560 0x7fd9281f5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:03.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.496+0000 7fd92f400640 1 --2- 172.21.15.52:0/3688979346 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9280b39b0 0x7fd928152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:03.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.496+0000 7fd92f400640 1 -- 172.21.15.52:0/3688979346 >> 172.21.15.52:0/3688979346 conn(0x7fd92806a3d0 msgr2=0x7fd9280b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:03.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.496+0000 7fd92f400640 1 -- 172.21.15.52:0/3688979346 shutdown_connections 2024-03-12T17:08:03.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.496+0000 7fd92f400640 1 -- 172.21.15.52:0/3688979346 wait complete. 2024-03-12T17:08:03.513 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-12T17:08:03.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.742+0000 7f19c0e00640 1 Processor -- start 2024-03-12T17:08:03.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.743+0000 7f19c0e00640 1 -- start start 2024-03-12T17:08:03.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.744+0000 7f19c0e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f19bc0b80a0 0x7f19bc0cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:03.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.744+0000 7f19c0e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f19bc0cd9b0 0x7f19bc0d10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:03.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.744+0000 7f19c0e00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19bc160b90 con 0x7f19bc15c890 2024-03-12T17:08:03.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.744+0000 7f19c0e00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19bc0d0ca0 con 0x7f19bc0b80a0 2024-03-12T17:08:03.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.744+0000 7f19c0e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19bc0cde60 con 0x7f19bc0cd9b0 2024-03-12T17:08:03.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.744+0000 7f19bb400640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f19bc15c890 0x7f19bc160480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:48016/0 (socket says 172.21.15.52:48016) 2024-03-12T17:08:03.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.744+0000 7f19bb400640 1 -- 172.21.15.52:0/2856381421 learned_addr learned my addr 172.21.15.52:0/2856381421 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:08:03.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.744+0000 7f19baa00640 1 --2- 172.21.15.52:0/2856381421 >> v2:172.21.15.52:3300/0 conn(0x7f19bc0b80a0 0x7f19bc0cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:03.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.745+0000 7f19bbe00640 1 --2- 172.21.15.52:0/2856381421 >> v2:172.21.15.102:3300/0 conn(0x7f19bc0cd9b0 0x7f19bc0d10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:03.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.745+0000 7f19ba000640 1 -- 172.21.15.52:0/2856381421 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1786878786 0 0) 0x7f19a4001110 con 0x7f19bc15c890 2024-03-12T17:08:03.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.745+0000 7f19ba000640 1 -- 172.21.15.52:0/2856381421 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19a0003520 con 0x7f19bc15c890 2024-03-12T17:08:03.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.745+0000 7f19baa00640 1 -- 172.21.15.52:0/2856381421 >> v2:172.21.15.102:3300/0 conn(0x7f19bc0cd9b0 msgr2=0x7f19bc0d10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:03.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.745+0000 7f19baa00640 1 --2- 172.21.15.52:0/2856381421 >> v2:172.21.15.102:3300/0 conn(0x7f19bc0cd9b0 0x7f19bc0d10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:03.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.745+0000 7f19baa00640 1 -- 172.21.15.52:0/2856381421 >> v1:172.21.15.52:6789/0 conn(0x7f19bc15c890 legacy=0x7f19bc160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:03.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.745+0000 7f19baa00640 1 -- 172.21.15.52:0/2856381421 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19bc161390 con 0x7f19bc0b80a0 2024-03-12T17:08:03.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.746+0000 7f19baa00640 1 --2- 172.21.15.52:0/2856381421 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19bc0b80a0 0x7f19bc0cd470 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f19bc0d0ba0 tx=0x7f19ac02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8deea868272c4331 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:03.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.747+0000 7f19ba000640 1 -- 172.21.15.52:0/2856381421 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ac02cbf0 con 0x7f19bc0b80a0 2024-03-12T17:08:03.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.747+0000 7f19ba000640 1 -- 172.21.15.52:0/2856381421 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19ac02cd50 con 0x7f19bc0b80a0 2024-03-12T17:08:03.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.747+0000 7f19ba000640 1 -- 172.21.15.52:0/2856381421 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ac034660 con 0x7f19bc0b80a0 2024-03-12T17:08:03.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.747+0000 7f19c0e00640 1 -- 172.21.15.52:0/2856381421 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19bc0b80a0 msgr2=0x7f19bc0cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:03.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.747+0000 7f19c0e00640 1 --2- 172.21.15.52:0/2856381421 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19bc0b80a0 0x7f19bc0cd470 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f19bc0d0ba0 tx=0x7f19ac02be40 comp rx=0 tx=0).stop 2024-03-12T17:08:03.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.748+0000 7f19c0e00640 1 -- 172.21.15.52:0/2856381421 shutdown_connections 2024-03-12T17:08:03.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.748+0000 7f19c0e00640 1 --2- 172.21.15.52:0/2856381421 >> v2:172.21.15.102:3300/0 conn(0x7f19bc0cd9b0 0x7f19bc0d10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:03.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.748+0000 7f19c0e00640 1 --2- 172.21.15.52:0/2856381421 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19bc0b80a0 0x7f19bc0cd470 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:03.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.748+0000 7f19c0e00640 1 -- 172.21.15.52:0/2856381421 >> 172.21.15.52:0/2856381421 conn(0x7f19bc06ae90 msgr2=0x7f19bc06b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:03.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.748+0000 7f19c0e00640 1 -- 172.21.15.52:0/2856381421 shutdown_connections 2024-03-12T17:08:03.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.748+0000 7f19c0e00640 1 -- 172.21.15.52:0/2856381421 wait complete. 2024-03-12T17:08:03.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.750+0000 7f19c0e00640 1 Processor -- start 2024-03-12T17:08:03.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.750+0000 7f19c0e00640 1 -- start start 2024-03-12T17:08:03.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.751+0000 7f19c0e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f19bc0b80a0 0x7f19bc0b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:03.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.751+0000 7f19c0e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19bc0cd9b0 0x7f19bc1f1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:03.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.751+0000 7f19c0e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19bc15c890 0x7f19bc0cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:03.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.751+0000 7f19c0e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc0cad00 con 0x7f19bc0cd9b0 2024-03-12T17:08:03.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.751+0000 7f19c0e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc0cc470 con 0x7f19bc15c890 2024-03-12T17:08:03.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.751+0000 7f19c0e00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc0ca130 con 0x7f19bc0b80a0 2024-03-12T17:08:03.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.751+0000 7f19baa00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19bc0cd9b0 0x7f19bc1f1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:03.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.751+0000 7f19baa00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19bc0cd9b0 0x7f19bc1f1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60782/0 (socket says 172.21.15.52:60782) 2024-03-12T17:08:03.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.751+0000 7f19baa00640 1 -- 172.21.15.52:0/747619250 learned_addr learned my addr 172.21.15.52:0/747619250 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:03.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.751+0000 7f19bb400640 1 --2- 172.21.15.52:0/747619250 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f19bc0b80a0 0x7f19bc0b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:03.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.752+0000 7f19bbe00640 1 --2- 172.21.15.52:0/747619250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19bc15c890 0x7f19bc0cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:03.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.752+0000 7f19baa00640 1 -- 172.21.15.52:0/747619250 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f19bc0b80a0 msgr2=0x7f19bc0b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:03.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.752+0000 7f19baa00640 1 --2- 172.21.15.52:0/747619250 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f19bc0b80a0 0x7f19bc0b49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:03.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.752+0000 7f19baa00640 1 -- 172.21.15.52:0/747619250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19bc15c890 msgr2=0x7f19bc0cb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:03.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.752+0000 7f19baa00640 1 --2- 172.21.15.52:0/747619250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19bc15c890 0x7f19bc0cb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:03.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.752+0000 7f19baa00640 1 -- 172.21.15.52:0/747619250 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ac009a70 con 0x7f19bc0cd9b0 2024-03-12T17:08:03.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.752+0000 7f19baa00640 1 --2- 172.21.15.52:0/747619250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19bc0cd9b0 0x7f19bc1f1b20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f19ac037040 tx=0x7f19ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:03.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.752+0000 7f19b3400640 1 -- 172.21.15.52:0/747619250 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ac040650 con 0x7f19bc0cd9b0 2024-03-12T17:08:03.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.752+0000 7f19b3400640 1 -- 172.21.15.52:0/747619250 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19ac0407b0 con 0x7f19bc0cd9b0 2024-03-12T17:08:03.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.752+0000 7f19c0e00640 1 -- 172.21.15.52:0/747619250 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19bc0ca3d0 con 0x7f19bc0cd9b0 2024-03-12T17:08:03.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.752+0000 7f19b3400640 1 -- 172.21.15.52:0/747619250 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ac040b60 con 0x7f19bc0cd9b0 2024-03-12T17:08:03.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.753+0000 7f19c0e00640 1 -- 172.21.15.52:0/747619250 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19bc0ca990 con 0x7f19bc0cd9b0 2024-03-12T17:08:03.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.754+0000 7f19c0e00640 1 -- 172.21.15.52:0/747619250 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19bc06c420 con 0x7f19bc0cd9b0 2024-03-12T17:08:03.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.755+0000 7f19b3400640 1 -- 172.21.15.52:0/747619250 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f19ac040cc0 con 0x7f19bc0cd9b0 2024-03-12T17:08:03.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.756+0000 7f19b3400640 1 --2- 172.21.15.52:0/747619250 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f199806ac00 0x7f199806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:03.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.756+0000 7f19b3400640 1 -- 172.21.15.52:0/747619250 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f19ac0ac010 con 0x7f19bc0cd9b0 2024-03-12T17:08:03.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.756+0000 7f19bb400640 1 --2- 172.21.15.52:0/747619250 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f199806ac00 0x7f199806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:03.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.757+0000 7f19bb400640 1 --2- 172.21.15.52:0/747619250 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f199806ac00 0x7f199806d0c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f19a4007fd0 tx=0x7f19a4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:03.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:03.760+0000 7f19b3400640 1 -- 172.21.15.52:0/747619250 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f19ac0775b0 con 0x7f19bc0cd9b0 2024-03-12T17:08:04.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.044+0000 7f19c0e00640 1 -- 172.21.15.52:0/747619250 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f19bc0d0340 con 0x7f19bc0cd9b0 2024-03-12T17:08:04.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.045+0000 7f19b3400640 1 -- 172.21.15.52:0/747619250 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f19ac07abd0 con 0x7f19bc0cd9b0 2024-03-12T17:08:04.039 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:08:04.040 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-03-12T17:08:04.040 INFO:teuthology.orchestra.run.smithi052.stdout: "auth_dump": [ 2024-03-12T17:08:04.040 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-12T17:08:04.040 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "mds.a", 2024-03-12T17:08:04.040 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBWi/Bl4S66IRAAwl2dMUzSrsczo+luzg/UDg==", 2024-03-12T17:08:04.040 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-03-12T17:08:04.040 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow", 2024-03-12T17:08:04.040 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow *", 2024-03-12T17:08:04.040 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow *", 2024-03-12T17:08:04.040 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-03-12T17:08:04.040 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-12T17:08:04.040 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-12T17:08:04.041 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-12T17:08:04.041 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "mds.b", 2024-03-12T17:08:04.041 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBXi/Bla8vvCBAAnvXNXGDljXRihZONHSKF/w==", 2024-03-12T17:08:04.041 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-03-12T17:08:04.041 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow", 2024-03-12T17:08:04.041 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow *", 2024-03-12T17:08:04.041 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow *", 2024-03-12T17:08:04.041 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-03-12T17:08:04.041 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-12T17:08:04.041 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-12T17:08:04.041 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-12T17:08:04.041 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "mds.c", 2024-03-12T17:08:04.042 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBWi/Bllf0XNBAAcd9lZCjmGesX1rT6Nac5gQ==", 2024-03-12T17:08:04.042 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-03-12T17:08:04.042 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow", 2024-03-12T17:08:04.042 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow *", 2024-03-12T17:08:04.042 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow *", 2024-03-12T17:08:04.042 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-03-12T17:08:04.042 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-12T17:08:04.042 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-12T17:08:04.042 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-12T17:08:04.042 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "mds.d", 2024-03-12T17:08:04.042 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBXi/BlGHK2FxAAvywOwM5ZkXa16lyfAYB+NQ==", 2024-03-12T17:08:04.042 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-03-12T17:08:04.042 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow", 2024-03-12T17:08:04.043 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow *", 2024-03-12T17:08:04.043 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow *", 2024-03-12T17:08:04.043 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-03-12T17:08:04.043 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-12T17:08:04.043 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-12T17:08:04.043 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-12T17:08:04.043 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "osd.0", 2024-03-12T17:08:04.043 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBai/BlarXWBxAAfIQgbiJtEi3/J8gN4e0TGw==", 2024-03-12T17:08:04.043 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-03-12T17:08:04.043 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow profile osd", 2024-03-12T17:08:04.043 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile osd", 2024-03-12T17:08:04.043 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-03-12T17:08:04.043 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-12T17:08:04.043 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-12T17:08:04.044 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-12T17:08:04.044 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "osd.1", 2024-03-12T17:08:04.044 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBdi/BlzrueIRAApKYnmLqMVcuRhV/a4gFTLg==", 2024-03-12T17:08:04.044 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-03-12T17:08:04.044 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow profile osd", 2024-03-12T17:08:04.044 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile osd", 2024-03-12T17:08:04.044 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-03-12T17:08:04.044 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-12T17:08:04.044 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-12T17:08:04.044 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-12T17:08:04.044 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "osd.2", 2024-03-12T17:08:04.044 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBgi/BlDY3sLBAAmhhTmkfpCXbUWEKVNMsQ1w==", 2024-03-12T17:08:04.044 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-03-12T17:08:04.044 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow profile osd", 2024-03-12T17:08:04.045 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile osd", 2024-03-12T17:08:04.045 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-03-12T17:08:04.045 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-12T17:08:04.045 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-12T17:08:04.045 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-12T17:08:04.045 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "osd.3", 2024-03-12T17:08:04.045 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBki/BlaoypChAAZ8lV0XQxqrbfnZrYRE7XEw==", 2024-03-12T17:08:04.045 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-03-12T17:08:04.045 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow profile osd", 2024-03-12T17:08:04.045 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile osd", 2024-03-12T17:08:04.045 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-03-12T17:08:04.045 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-12T17:08:04.045 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-12T17:08:04.045 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-12T17:08:04.046 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "osd.4", 2024-03-12T17:08:04.046 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBri/Blle0NChAAB7AFUJK+mykQVMak16bp5A==", 2024-03-12T17:08:04.046 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-03-12T17:08:04.046 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow profile osd", 2024-03-12T17:08:04.046 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile osd", 2024-03-12T17:08:04.046 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-03-12T17:08:04.046 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-12T17:08:04.046 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-12T17:08:04.046 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-12T17:08:04.046 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "osd.5", 2024-03-12T17:08:04.046 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBui/Bl/xV3IhAAm44sVuUu6/Zqko1CHnOXkQ==", 2024-03-12T17:08:04.046 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-03-12T17:08:04.046 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow profile osd", 2024-03-12T17:08:04.046 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile osd", 2024-03-12T17:08:04.047 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-03-12T17:08:04.047 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-12T17:08:04.047 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-12T17:08:04.047 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-12T17:08:04.047 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "osd.6", 2024-03-12T17:08:04.047 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBxi/BloF8uLBAAiIYC2menNPYe+GZ/yH/Ozg==", 2024-03-12T17:08:04.047 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-03-12T17:08:04.047 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow profile osd", 2024-03-12T17:08:04.047 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile osd", 2024-03-12T17:08:04.047 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-03-12T17:08:04.047 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-12T17:08:04.047 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-12T17:08:04.047 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-12T17:08:04.048 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "osd.7", 2024-03-12T17:08:04.048 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQB0i/BlBNXDNRAAaQuAdjotgiiroWuymS9noQ==", 2024-03-12T17:08:04.048 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-03-12T17:08:04.048 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow profile osd", 2024-03-12T17:08:04.048 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile osd", 2024-03-12T17:08:04.048 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-03-12T17:08:04.048 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-12T17:08:04.048 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-12T17:08:04.048 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-12T17:08:04.048 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.0", 2024-03-12T17:08:04.048 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBXi/BlAy3lJBAAI6pQDgbz22pC+i7V7j7TxA==", 2024-03-12T17:08:04.048 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-03-12T17:08:04.048 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow", 2024-03-12T17:08:04.048 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow r", 2024-03-12T17:08:04.049 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow rw", 2024-03-12T17:08:04.049 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow rwx" 2024-03-12T17:08:04.049 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-12T17:08:04.049 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-12T17:08:04.049 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-12T17:08:04.049 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.1", 2024-03-12T17:08:04.049 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBXi/Bl+OJ1OxAALw+b2TNswYnjzIlx1uoiZA==", 2024-03-12T17:08:04.049 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-03-12T17:08:04.049 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow", 2024-03-12T17:08:04.049 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow r", 2024-03-12T17:08:04.049 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow rw", 2024-03-12T17:08:04.049 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow rwx" 2024-03-12T17:08:04.049 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-12T17:08:04.049 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-12T17:08:04.050 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-12T17:08:04.050 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.2", 2024-03-12T17:08:04.050 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBXi/Bl7W/5MBAAeI8bJMJRCkzAnki/Sxod7A==", 2024-03-12T17:08:04.050 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-03-12T17:08:04.050 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow", 2024-03-12T17:08:04.050 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow r", 2024-03-12T17:08:04.050 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow rw", 2024-03-12T17:08:04.050 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow rwx" 2024-03-12T17:08:04.050 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-12T17:08:04.050 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-12T17:08:04.050 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-12T17:08:04.050 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.3", 2024-03-12T17:08:04.050 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBYi/Bl7fdOChAA5ExYqh/p4m1mRyJ5Xd3M8Q==", 2024-03-12T17:08:04.051 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-03-12T17:08:04.051 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow", 2024-03-12T17:08:04.051 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow r", 2024-03-12T17:08:04.051 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow rw", 2024-03-12T17:08:04.051 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow rwx" 2024-03-12T17:08:04.051 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-12T17:08:04.051 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-12T17:08:04.051 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-12T17:08:04.051 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.admin", 2024-03-12T17:08:04.051 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBVi/Bl0DH4ERAAr0Oaf2K8P8SdXQjliAt2TA==", 2024-03-12T17:08:04.051 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-03-12T17:08:04.051 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow *", 2024-03-12T17:08:04.051 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow *", 2024-03-12T17:08:04.052 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow *", 2024-03-12T17:08:04.052 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-03-12T17:08:04.052 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-12T17:08:04.052 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-12T17:08:04.052 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-12T17:08:04.052 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.bootstrap-mds", 2024-03-12T17:08:04.052 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQB9i/BlVOrjOBAAKpz4bpvJ6f+61jz16F/COA==", 2024-03-12T17:08:04.052 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-03-12T17:08:04.052 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile bootstrap-mds" 2024-03-12T17:08:04.052 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-12T17:08:04.053 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-12T17:08:04.053 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-12T17:08:04.053 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.bootstrap-mgr", 2024-03-12T17:08:04.053 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQB9i/BlhfrjOBAAM17emq5oO1g6/u1ypdvxPw==", 2024-03-12T17:08:04.053 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-03-12T17:08:04.053 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile bootstrap-mgr" 2024-03-12T17:08:04.053 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-12T17:08:04.053 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-12T17:08:04.053 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-12T17:08:04.053 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.bootstrap-osd", 2024-03-12T17:08:04.053 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQB9i/BlmwnkOBAAjr/XzB+hDHspZTsrc6Y1Vg==", 2024-03-12T17:08:04.054 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-03-12T17:08:04.054 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile bootstrap-osd" 2024-03-12T17:08:04.054 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-12T17:08:04.054 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-12T17:08:04.054 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-12T17:08:04.054 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.bootstrap-rbd", 2024-03-12T17:08:04.054 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQB9i/BlkxjkOBAAU+8VG3tZA0rvZkmC2GG2+A==", 2024-03-12T17:08:04.054 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-03-12T17:08:04.054 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile bootstrap-rbd" 2024-03-12T17:08:04.055 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-12T17:08:04.055 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-12T17:08:04.055 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-12T17:08:04.055 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-03-12T17:08:04.055 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQB9i/BlFSnkOBAAwOY/HUm205Bi36Kj6FOZPg==", 2024-03-12T17:08:04.055 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-03-12T17:08:04.055 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-03-12T17:08:04.055 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-12T17:08:04.055 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-12T17:08:04.055 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-12T17:08:04.056 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.bootstrap-rgw", 2024-03-12T17:08:04.056 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQB9i/BlIjjkOBAAvJnFBGl5DxyN5if2aANjlA==", 2024-03-12T17:08:04.056 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-03-12T17:08:04.056 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile bootstrap-rgw" 2024-03-12T17:08:04.056 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-12T17:08:04.056 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-12T17:08:04.056 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-12T17:08:04.056 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "mgr.x", 2024-03-12T17:08:04.056 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBWi/BlKiFAFRAAwlsqCR8ar9YEpwpYQ1AcAQ==", 2024-03-12T17:08:04.056 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-03-12T17:08:04.057 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow *", 2024-03-12T17:08:04.057 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile mgr", 2024-03-12T17:08:04.057 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-03-12T17:08:04.057 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-12T17:08:04.057 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-12T17:08:04.057 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-12T17:08:04.057 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "mgr.y", 2024-03-12T17:08:04.057 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQBWi/BlQIy0BxAAaeyxH1BvjKvpfEgvE41CnA==", 2024-03-12T17:08:04.057 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-03-12T17:08:04.057 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow *", 2024-03-12T17:08:04.058 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile mgr", 2024-03-12T17:08:04.058 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-03-12T17:08:04.058 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-12T17:08:04.058 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-12T17:08:04.058 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-03-12T17:08:04.058 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-03-12T17:08:04.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.048+0000 7f19c0e00640 1 -- 172.21.15.52:0/747619250 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f199806ac00 msgr2=0x7f199806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:04.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.048+0000 7f19c0e00640 1 --2- 172.21.15.52:0/747619250 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f199806ac00 0x7f199806d0c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f19a4007fd0 tx=0x7f19a4009040 comp rx=0 tx=0).stop 2024-03-12T17:08:04.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.048+0000 7f19c0e00640 1 -- 172.21.15.52:0/747619250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19bc0cd9b0 msgr2=0x7f19bc1f1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:04.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.048+0000 7f19c0e00640 1 --2- 172.21.15.52:0/747619250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19bc0cd9b0 0x7f19bc1f1b20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f19ac037040 tx=0x7f19ac004a40 comp rx=0 tx=0).stop 2024-03-12T17:08:04.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.048+0000 7f19c0e00640 1 -- 172.21.15.52:0/747619250 shutdown_connections 2024-03-12T17:08:04.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.048+0000 7f19c0e00640 1 --2- 172.21.15.52:0/747619250 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f199806ac00 0x7f199806d0c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:04.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.048+0000 7f19c0e00640 1 --2- 172.21.15.52:0/747619250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19bc15c890 0x7f19bc0cb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:04.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.048+0000 7f19c0e00640 1 --2- 172.21.15.52:0/747619250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19bc0cd9b0 0x7f19bc1f1b20 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:04.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.048+0000 7f19c0e00640 1 --2- 172.21.15.52:0/747619250 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f19bc0b80a0 0x7f19bc0b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:04.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.048+0000 7f19c0e00640 1 -- 172.21.15.52:0/747619250 >> 172.21.15.52:0/747619250 conn(0x7f19bc06ae90 msgr2=0x7f19bc15aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:04.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.049+0000 7f19c0e00640 1 -- 172.21.15.52:0/747619250 shutdown_connections 2024-03-12T17:08:04.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.049+0000 7f19c0e00640 1 -- 172.21.15.52:0/747619250 wait complete. 2024-03-12T17:08:04.062 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-03-12T17:08:04.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.266+0000 7f9f16e00640 1 Processor -- start 2024-03-12T17:08:04.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.267+0000 7f9f16e00640 1 -- start start 2024-03-12T17:08:04.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.268+0000 7f9f16e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9f100ccd40 0x7f9f1015b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:04.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.268+0000 7f9f16e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f9f1015e030 0x7f9f1015e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:04.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.268+0000 7f9f16e00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f1015b1a0 con 0x7f9f100700d0 2024-03-12T17:08:04.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.268+0000 7f9f16e00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f1006d940 con 0x7f9f1015e030 2024-03-12T17:08:04.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.268+0000 7f9f16e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f1015eba0 con 0x7f9f100ccd40 2024-03-12T17:08:04.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.269+0000 7f9f15000640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f9f1015e030 0x7f9f1015e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:04.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.269+0000 7f9f16400640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f9f100700d0 0x7f9f1006c130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:48020/0 (socket says 172.21.15.52:48020) 2024-03-12T17:08:04.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.269+0000 7f9f16400640 1 -- 172.21.15.52:0/1026560616 learned_addr learned my addr 172.21.15.52:0/1026560616 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:08:04.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.269+0000 7f9f15a00640 1 --2- 172.21.15.52:0/1026560616 >> v2:172.21.15.102:3300/0 conn(0x7f9f100ccd40 0x7f9f1015b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:04.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.269+0000 7f9f15000640 1 -- 172.21.15.52:0/1026560616 >> v2:172.21.15.102:3300/0 conn(0x7f9f100ccd40 msgr2=0x7f9f1015b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:04.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.269+0000 7f9f15000640 1 --2- 172.21.15.52:0/1026560616 >> v2:172.21.15.102:3300/0 conn(0x7f9f100ccd40 0x7f9f1015b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:04.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.269+0000 7f9f15000640 1 -- 172.21.15.52:0/1026560616 >> v1:172.21.15.52:6789/0 conn(0x7f9f100700d0 legacy=0x7f9f1006c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:04.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.269+0000 7f9f15000640 1 -- 172.21.15.52:0/1026560616 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f1015ab50 con 0x7f9f1015e030 2024-03-12T17:08:04.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.270+0000 7f9f15000640 1 --2- 172.21.15.52:0/1026560616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f1015e030 0x7f9f1015e4b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9f1006acb0 tx=0x7f9efc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426ccfb861c07deb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:04.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.271+0000 7f9f0fe00640 1 -- 172.21.15.52:0/1026560616 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc02e070 con 0x7f9f1015e030 2024-03-12T17:08:04.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.271+0000 7f9f0fe00640 1 -- 172.21.15.52:0/1026560616 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9efc0315a0 con 0x7f9f1015e030 2024-03-12T17:08:04.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.271+0000 7f9f0fe00640 1 -- 172.21.15.52:0/1026560616 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc031930 con 0x7f9f1015e030 2024-03-12T17:08:04.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.271+0000 7f9f16e00640 1 -- 172.21.15.52:0/1026560616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f1015e030 msgr2=0x7f9f1015e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:04.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.271+0000 7f9f16e00640 1 --2- 172.21.15.52:0/1026560616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f1015e030 0x7f9f1015e4b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9f1006acb0 tx=0x7f9efc02b840 comp rx=0 tx=0).stop 2024-03-12T17:08:04.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.271+0000 7f9f16e00640 1 -- 172.21.15.52:0/1026560616 shutdown_connections 2024-03-12T17:08:04.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.271+0000 7f9f16e00640 1 --2- 172.21.15.52:0/1026560616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f1015e030 0x7f9f1015e4b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:04.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.271+0000 7f9f16e00640 1 --2- 172.21.15.52:0/1026560616 >> v2:172.21.15.102:3300/0 conn(0x7f9f100ccd40 0x7f9f1015b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:04.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.272+0000 7f9f16e00640 1 -- 172.21.15.52:0/1026560616 >> 172.21.15.52:0/1026560616 conn(0x7f9f10065df0 msgr2=0x7f9f10069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:04.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.272+0000 7f9f16e00640 1 -- 172.21.15.52:0/1026560616 shutdown_connections 2024-03-12T17:08:04.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.272+0000 7f9f16e00640 1 -- 172.21.15.52:0/1026560616 wait complete. 2024-03-12T17:08:04.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.273+0000 7f9f16e00640 1 Processor -- start 2024-03-12T17:08:04.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.274+0000 7f9f16e00640 1 -- start start 2024-03-12T17:08:04.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.275+0000 7f9f16e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f100700d0 0x7f9f101f2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:04.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.275+0000 7f9f16e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f100ccd40 0x7f9f10157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:04.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.275+0000 7f9f16e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9f1015e030 0x7f9f101f2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:04.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.275+0000 7f9f16e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f100ca420 con 0x7f9f100700d0 2024-03-12T17:08:04.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.275+0000 7f9f16e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f101584d0 con 0x7f9f100ccd40 2024-03-12T17:08:04.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.275+0000 7f9f16e00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f101f2d50 con 0x7f9f1015e030 2024-03-12T17:08:04.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.275+0000 7f9f15a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f100700d0 0x7f9f101f2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:04.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.275+0000 7f9f15a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f100700d0 0x7f9f101f2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60796/0 (socket says 172.21.15.52:60796) 2024-03-12T17:08:04.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.275+0000 7f9f15a00640 1 -- 172.21.15.52:0/916840819 learned_addr learned my addr 172.21.15.52:0/916840819 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:04.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.275+0000 7f9f15000640 1 --2- 172.21.15.52:0/916840819 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f100ccd40 0x7f9f10157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:04.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.275+0000 7f9f15a00640 1 -- 172.21.15.52:0/916840819 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9f1015e030 msgr2=0x7f9f101f2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:04.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.275+0000 7f9f15a00640 1 --2- 172.21.15.52:0/916840819 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9f1015e030 0x7f9f101f2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:04.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.275+0000 7f9f15a00640 1 -- 172.21.15.52:0/916840819 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f100ccd40 msgr2=0x7f9f10157f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:04.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.275+0000 7f9f15a00640 1 --2- 172.21.15.52:0/916840819 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f100ccd40 0x7f9f10157f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:04.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.275+0000 7f9f15a00640 1 -- 172.21.15.52:0/916840819 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9efc009400 con 0x7f9f100700d0 2024-03-12T17:08:04.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.275+0000 7f9f15000640 1 --2- 172.21.15.52:0/916840819 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f100ccd40 0x7f9f10157f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:08:04.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.276+0000 7f9f15a00640 1 --2- 172.21.15.52:0/916840819 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f100700d0 0x7f9f101f2fa0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9f0400d800 tx=0x7f9f0400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:04.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.276+0000 7f9f0d600640 1 -- 172.21.15.52:0/916840819 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0400b6c0 con 0x7f9f100700d0 2024-03-12T17:08:04.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.276+0000 7f9f0d600640 1 -- 172.21.15.52:0/916840819 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f0400b820 con 0x7f9f100700d0 2024-03-12T17:08:04.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.276+0000 7f9f0d600640 1 -- 172.21.15.52:0/916840819 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0400bbb0 con 0x7f9f100700d0 2024-03-12T17:08:04.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.276+0000 7f9f16e00640 1 -- 172.21.15.52:0/916840819 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f100c9810 con 0x7f9f100700d0 2024-03-12T17:08:04.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.276+0000 7f9f16e00640 1 -- 172.21.15.52:0/916840819 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f100c9dd0 con 0x7f9f100700d0 2024-03-12T17:08:04.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.277+0000 7f9f16e00640 1 -- 172.21.15.52:0/916840819 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f10071b10 con 0x7f9f100700d0 2024-03-12T17:08:04.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.278+0000 7f9f0d600640 1 -- 172.21.15.52:0/916840819 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9f0400bd10 con 0x7f9f100700d0 2024-03-12T17:08:04.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.279+0000 7f9f0d600640 1 --2- 172.21.15.52:0/916840819 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f9ef006ac50 0x7f9ef006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:04.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.279+0000 7f9f0d600640 1 -- 172.21.15.52:0/916840819 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f9f04089720 con 0x7f9f100700d0 2024-03-12T17:08:04.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.280+0000 7f9f15000640 1 --2- 172.21.15.52:0/916840819 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f9ef006ac50 0x7f9ef006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:04.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.281+0000 7f9f15000640 1 --2- 172.21.15.52:0/916840819 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f9ef006ac50 0x7f9ef006d110 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9efc036000 tx=0x7f9efc008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:04.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.283+0000 7f9f0d600640 1 -- 172.21.15.52:0/916840819 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f9f04054af0 con 0x7f9f100700d0 2024-03-12T17:08:04.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.551+0000 7f9f16e00640 1 -- 172.21.15.52:0/916840819 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f9f1004e110 con 0x7f9f100700d0 2024-03-12T17:08:04.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.559+0000 7f9f0d600640 1 -- 172.21.15.52:0/916840819 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9f04058110 con 0x7f9f100700d0 2024-03-12T17:08:04.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.561+0000 7f9f16e00640 1 -- 172.21.15.52:0/916840819 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f9ef006ac50 msgr2=0x7f9ef006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:04.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.561+0000 7f9f16e00640 1 --2- 172.21.15.52:0/916840819 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f9ef006ac50 0x7f9ef006d110 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9efc036000 tx=0x7f9efc008030 comp rx=0 tx=0).stop 2024-03-12T17:08:04.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.561+0000 7f9f16e00640 1 -- 172.21.15.52:0/916840819 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f100700d0 msgr2=0x7f9f101f2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:04.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.561+0000 7f9f16e00640 1 --2- 172.21.15.52:0/916840819 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f100700d0 0x7f9f101f2fa0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9f0400d800 tx=0x7f9f0400dcc0 comp rx=0 tx=0).stop 2024-03-12T17:08:04.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.562+0000 7f9f16e00640 1 -- 172.21.15.52:0/916840819 shutdown_connections 2024-03-12T17:08:04.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.562+0000 7f9f16e00640 1 --2- 172.21.15.52:0/916840819 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f9ef006ac50 0x7f9ef006d110 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:04.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.562+0000 7f9f16e00640 1 --2- 172.21.15.52:0/916840819 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9f1015e030 0x7f9f101f2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:04.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.562+0000 7f9f16e00640 1 --2- 172.21.15.52:0/916840819 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f100ccd40 0x7f9f10157f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:04.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.562+0000 7f9f16e00640 1 --2- 172.21.15.52:0/916840819 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f100700d0 0x7f9f101f2fa0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:04.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.562+0000 7f9f16e00640 1 -- 172.21.15.52:0/916840819 >> 172.21.15.52:0/916840819 conn(0x7f9f10065df0 msgr2=0x7f9f10069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:04.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.562+0000 7f9f16e00640 1 -- 172.21.15.52:0/916840819 shutdown_connections 2024-03-12T17:08:04.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.562+0000 7f9f16e00640 1 -- 172.21.15.52:0/916840819 wait complete. 2024-03-12T17:08:04.572 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-03-12T17:08:04.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.791+0000 7f0afdc00640 1 Processor -- start 2024-03-12T17:08:04.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.791+0000 7f0afdc00640 1 -- start start 2024-03-12T17:08:04.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.792+0000 7f0afdc00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0af8146070 0x7f0af80b2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:04.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.792+0000 7f0afdc00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0af815cd00 0x7f0af806cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:04.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.793+0000 7f0afdc00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0af80b2f70 con 0x7f0af80700d0 2024-03-12T17:08:04.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.793+0000 7f0afdc00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0af806d130 con 0x7f0af8146070 2024-03-12T17:08:04.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.793+0000 7f0afdc00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af80b3250 con 0x7f0af815cd00 2024-03-12T17:08:04.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.793+0000 7f0afd200640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f0af80700d0 0x7f0af80b1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:48024/0 (socket says 172.21.15.52:48024) 2024-03-12T17:08:04.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.793+0000 7f0afd200640 1 -- 172.21.15.52:0/2586380757 learned_addr learned my addr 172.21.15.52:0/2586380757 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:08:04.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.793+0000 7f0af7e00640 1 --2- 172.21.15.52:0/2586380757 >> v2:172.21.15.102:3300/0 conn(0x7f0af8146070 0x7f0af80b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:04.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.793+0000 7f0aefe00640 1 --2- 172.21.15.52:0/2586380757 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0af815cd00 0x7f0af806cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:04.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.794+0000 7f0af7400640 1 -- 172.21.15.52:0/2586380757 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1914273339 0 0) 0x7f0af80b2f70 con 0x7f0af80700d0 2024-03-12T17:08:04.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.794+0000 7f0af7400640 1 -- 172.21.15.52:0/2586380757 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0adc003520 con 0x7f0af80700d0 2024-03-12T17:08:04.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.794+0000 7f0af7400640 1 -- 172.21.15.52:0/2586380757 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3899431477 0 0) 0x7f0adc003520 con 0x7f0af80700d0 2024-03-12T17:08:04.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.794+0000 7f0af7400640 1 -- 172.21.15.52:0/2586380757 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0af815cd00 msgr2=0x7f0af806cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:04.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.794+0000 7f0af7400640 1 --2- 172.21.15.52:0/2586380757 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0af815cd00 0x7f0af806cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:04.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.794+0000 7f0af7400640 1 -- 172.21.15.52:0/2586380757 >> v2:172.21.15.102:3300/0 conn(0x7f0af8146070 msgr2=0x7f0af80b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:04.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.794+0000 7f0af7400640 1 --2- 172.21.15.52:0/2586380757 >> v2:172.21.15.102:3300/0 conn(0x7f0af8146070 0x7f0af80b2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:04.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.794+0000 7f0af7400640 1 -- 172.21.15.52:0/2586380757 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af80b3530 con 0x7f0af80700d0 2024-03-12T17:08:04.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.794+0000 7f0af7400640 1 -- 172.21.15.52:0/2586380757 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f0af00019f0 con 0x7f0af80700d0 2024-03-12T17:08:04.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.794+0000 7f0af7400640 1 -- 172.21.15.52:0/2586380757 >> v1:172.21.15.52:6789/0 conn(0x7f0af80700d0 legacy=0x7f0af80b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:04.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.794+0000 7f0af7400640 1 --2- 172.21.15.52:0/2586380757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0adc003970 0x7f0adc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:04.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.794+0000 7f0af7400640 1 --2- 172.21.15.52:0/2586380757 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0adc0047c0 0x7f0adc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:04.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.794+0000 7f0af7400640 1 --2- 172.21.15.52:0/2586380757 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0af815cd00 0x7f0adc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:04.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.794+0000 7f0af7400640 1 -- 172.21.15.52:0/2586380757 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af806d130 con 0x7f0adc0047c0 2024-03-12T17:08:04.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.794+0000 7f0af7400640 1 -- 172.21.15.52:0/2586380757 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af80b3250 con 0x7f0adc003970 2024-03-12T17:08:04.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.794+0000 7f0af7400640 1 -- 172.21.15.52:0/2586380757 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0adc00b9d0 con 0x7f0af815cd00 2024-03-12T17:08:04.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.794+0000 7f0aefe00640 1 --2- 172.21.15.52:0/2586380757 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0adc0047c0 0x7f0adc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:04.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.795+0000 7f0af7e00640 1 --2- 172.21.15.52:0/2586380757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0adc003970 0x7f0adc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:04.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.795+0000 7f0afd200640 1 --2- 172.21.15.52:0/2586380757 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0af815cd00 0x7f0adc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:04.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.795+0000 7f0aefe00640 1 -- 172.21.15.52:0/2586380757 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0af815cd00 msgr2=0x7f0adc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:04.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.795+0000 7f0aefe00640 1 --2- 172.21.15.52:0/2586380757 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0af815cd00 0x7f0adc005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:04.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.795+0000 7f0aefe00640 1 -- 172.21.15.52:0/2586380757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0adc003970 msgr2=0x7f0adc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:04.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.795+0000 7f0aefe00640 1 --2- 172.21.15.52:0/2586380757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0adc003970 0x7f0adc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:04.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.795+0000 7f0aefe00640 1 -- 172.21.15.52:0/2586380757 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0adc003520 con 0x7f0adc0047c0 2024-03-12T17:08:04.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.796+0000 7f0aefe00640 1 --2- 172.21.15.52:0/2586380757 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0adc0047c0 0x7f0adc004c40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0009af0 tx=0x7f0adc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edef23aca6655b8f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:04.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.796+0000 7f0af7400640 1 -- 172.21.15.52:0/2586380757 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae002c6b0 con 0x7f0adc0047c0 2024-03-12T17:08:04.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.796+0000 7f0af7400640 1 -- 172.21.15.52:0/2586380757 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae002c810 con 0x7f0adc0047c0 2024-03-12T17:08:04.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.796+0000 7f0af7400640 1 -- 172.21.15.52:0/2586380757 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae002cba0 con 0x7f0adc0047c0 2024-03-12T17:08:04.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.797+0000 7f0afdc00640 1 -- 172.21.15.52:0/2586380757 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0adc0047c0 msgr2=0x7f0adc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:04.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.797+0000 7f0afdc00640 1 --2- 172.21.15.52:0/2586380757 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0adc0047c0 0x7f0adc004c40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0009af0 tx=0x7f0adc0057f0 comp rx=0 tx=0).stop 2024-03-12T17:08:04.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.797+0000 7f0afd200640 1 -- 172.21.15.52:0/2586380757 reap_dead start 2024-03-12T17:08:04.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.797+0000 7f0afdc00640 1 -- 172.21.15.52:0/2586380757 shutdown_connections 2024-03-12T17:08:04.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.797+0000 7f0afdc00640 1 -- 172.21.15.52:0/2586380757 >> 172.21.15.52:0/2586380757 conn(0x7f0af8065df0 msgr2=0x7f0af8069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:04.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.797+0000 7f0afdc00640 1 -- 172.21.15.52:0/2586380757 shutdown_connections 2024-03-12T17:08:04.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.797+0000 7f0afdc00640 1 -- 172.21.15.52:0/2586380757 wait complete. 2024-03-12T17:08:04.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.799+0000 7f0afdc00640 1 Processor -- start 2024-03-12T17:08:04.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.799+0000 7f0afdc00640 1 -- start start 2024-03-12T17:08:04.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.800+0000 7f0afdc00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0adc0047c0 0x7f0af81f6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:04.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.800+0000 7f0afdc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0af813c8b0 0x7f0af80aa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:04.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.800+0000 7f0afdc00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0af80cbcd0 0x7f0af80cc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:04.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.800+0000 7f0afdc00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af813e310 con 0x7f0af80cbcd0 2024-03-12T17:08:04.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.800+0000 7f0afdc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af80cc6c0 con 0x7f0af813c8b0 2024-03-12T17:08:04.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.800+0000 7f0afdc00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af813e000 con 0x7f0adc0047c0 2024-03-12T17:08:04.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.800+0000 7f0afd200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0af80cbcd0 0x7f0af80cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:04.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.800+0000 7f0afd200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0af80cbcd0 0x7f0af80cc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60800/0 (socket says 172.21.15.52:60800) 2024-03-12T17:08:04.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.800+0000 7f0afd200640 1 -- 172.21.15.52:0/3556159311 learned_addr learned my addr 172.21.15.52:0/3556159311 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:04.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.801+0000 7f0af7e00640 1 --2- 172.21.15.52:0/3556159311 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0adc0047c0 0x7f0af81f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:04.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.801+0000 7f0aefe00640 1 --2- 172.21.15.52:0/3556159311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0af813c8b0 0x7f0af80aa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:04.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.801+0000 7f0afd200640 1 -- 172.21.15.52:0/3556159311 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0adc0047c0 msgr2=0x7f0af81f6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:04.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.801+0000 7f0afd200640 1 --2- 172.21.15.52:0/3556159311 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0adc0047c0 0x7f0af81f6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:04.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.801+0000 7f0afd200640 1 -- 172.21.15.52:0/3556159311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0af813c8b0 msgr2=0x7f0af80aa5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:04.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.801+0000 7f0afd200640 1 --2- 172.21.15.52:0/3556159311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0af813c8b0 0x7f0af80aa5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:04.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.801+0000 7f0afd200640 1 -- 172.21.15.52:0/3556159311 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af80aaf10 con 0x7f0af80cbcd0 2024-03-12T17:08:04.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.801+0000 7f0afd200640 1 --2- 172.21.15.52:0/3556159311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0af80cbcd0 0x7f0af80cc180 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0af806a6b0 tx=0x7f0af000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:04.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.801+0000 7f0af4c00640 1 -- 172.21.15.52:0/3556159311 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af000de70 con 0x7f0af80cbcd0 2024-03-12T17:08:04.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.801+0000 7f0af4c00640 1 -- 172.21.15.52:0/3556159311 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af000bb80 con 0x7f0af80cbcd0 2024-03-12T17:08:04.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.801+0000 7f0af4c00640 1 -- 172.21.15.52:0/3556159311 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af0007070 con 0x7f0af80cbcd0 2024-03-12T17:08:04.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.801+0000 7f0afdc00640 1 -- 172.21.15.52:0/3556159311 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af80ab1f0 con 0x7f0af80cbcd0 2024-03-12T17:08:04.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.801+0000 7f0afdc00640 1 -- 172.21.15.52:0/3556159311 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af813cd10 con 0x7f0af80cbcd0 2024-03-12T17:08:04.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.803+0000 7f0afdc00640 1 -- 172.21.15.52:0/3556159311 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af8071b10 con 0x7f0af80cbcd0 2024-03-12T17:08:04.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.803+0000 7f0af4c00640 1 -- 172.21.15.52:0/3556159311 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0af00071d0 con 0x7f0af80cbcd0 2024-03-12T17:08:04.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.804+0000 7f0af4c00640 1 --2- 172.21.15.52:0/3556159311 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f0ad406ac30 0x7f0ad406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:04.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.804+0000 7f0af4c00640 1 -- 172.21.15.52:0/3556159311 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f0af00894d0 con 0x7f0af80cbcd0 2024-03-12T17:08:04.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.805+0000 7f0af7e00640 1 --2- 172.21.15.52:0/3556159311 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f0ad406ac30 0x7f0ad406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:04.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.806+0000 7f0af7e00640 1 --2- 172.21.15.52:0/3556159311 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f0ad406ac30 0x7f0ad406d0f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0ae80099d0 tx=0x7f0ae8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:04.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:04.809+0000 7f0af4c00640 1 -- 172.21.15.52:0/3556159311 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f0af0054950 con 0x7f0af80cbcd0 2024-03-12T17:08:04.818 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-03-12T17:08:05.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.083+0000 7f0afdc00640 1 -- 172.21.15.52:0/3556159311 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f0af804e110 con 0x7f0af80cbcd0 2024-03-12T17:08:05.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.090+0000 7f0af4c00640 1 -- 172.21.15.52:0/3556159311 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7f0af0057f70 con 0x7f0af80cbcd0 2024-03-12T17:08:05.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.093+0000 7f0afdc00640 1 -- 172.21.15.52:0/3556159311 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f0ad406ac30 msgr2=0x7f0ad406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:05.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.093+0000 7f0afdc00640 1 --2- 172.21.15.52:0/3556159311 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f0ad406ac30 0x7f0ad406d0f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0ae80099d0 tx=0x7f0ae8008040 comp rx=0 tx=0).stop 2024-03-12T17:08:05.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.093+0000 7f0afdc00640 1 -- 172.21.15.52:0/3556159311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0af80cbcd0 msgr2=0x7f0af80cc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:05.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.093+0000 7f0afdc00640 1 --2- 172.21.15.52:0/3556159311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0af80cbcd0 0x7f0af80cc180 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0af806a6b0 tx=0x7f0af000b4a0 comp rx=0 tx=0).stop 2024-03-12T17:08:05.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.093+0000 7f0afdc00640 1 -- 172.21.15.52:0/3556159311 shutdown_connections 2024-03-12T17:08:05.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.093+0000 7f0afdc00640 1 --2- 172.21.15.52:0/3556159311 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f0ad406ac30 0x7f0ad406d0f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:05.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.093+0000 7f0afdc00640 1 --2- 172.21.15.52:0/3556159311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0af80cbcd0 0x7f0af80cc180 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:05.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.093+0000 7f0afdc00640 1 --2- 172.21.15.52:0/3556159311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0af813c8b0 0x7f0af80aa5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:05.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.093+0000 7f0afdc00640 1 --2- 172.21.15.52:0/3556159311 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0adc0047c0 0x7f0af81f6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:05.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.093+0000 7f0afdc00640 1 -- 172.21.15.52:0/3556159311 >> 172.21.15.52:0/3556159311 conn(0x7f0af8065df0 msgr2=0x7f0af806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:05.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.093+0000 7f0afdc00640 1 -- 172.21.15.52:0/3556159311 shutdown_connections 2024-03-12T17:08:05.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.093+0000 7f0afdc00640 1 -- 172.21.15.52:0/3556159311 wait complete. 2024-03-12T17:08:05.107 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-03-12T17:08:05.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.324+0000 7fc0d8c00640 1 Processor -- start 2024-03-12T17:08:05.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.324+0000 7fc0d8c00640 1 -- start start 2024-03-12T17:08:05.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.326+0000 7fc0d8c00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc0d40ccd40 0x7fc0d415b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:05.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.326+0000 7fc0d8c00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc0d40707e0 0x7fc0d415e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:05.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.326+0000 7fc0d8c00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0d415b1a0 con 0x7fc0d415e030 2024-03-12T17:08:05.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.326+0000 7fc0d8c00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0d406d940 con 0x7fc0d40ccd40 2024-03-12T17:08:05.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.326+0000 7fc0d8c00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d415eba0 con 0x7fc0d40707e0 2024-03-12T17:08:05.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.326+0000 7fc0d3400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc0d40ccd40 0x7fc0d415b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:05.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.327+0000 7fc0d3400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc0d40ccd40 0x7fc0d415b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40554/0 (socket says 172.21.15.52:40554) 2024-03-12T17:08:05.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.327+0000 7fc0d3400640 1 -- 172.21.15.52:0/3941709554 learned_addr learned my addr 172.21.15.52:0/3941709554 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:05.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.327+0000 7fc0d3e00640 1 --2- 172.21.15.52:0/3941709554 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc0d40707e0 0x7fc0d415e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:05.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.327+0000 7fc0d3400640 1 -- 172.21.15.52:0/3941709554 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc0d40707e0 msgr2=0x7fc0d415e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:05.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.327+0000 7fc0d3400640 1 --2- 172.21.15.52:0/3941709554 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc0d40707e0 0x7fc0d415e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:05.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.327+0000 7fc0d3400640 1 -- 172.21.15.52:0/3941709554 >> v1:172.21.15.102:6789/0 conn(0x7fc0d415e030 legacy=0x7fc0d40700d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:05.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.327+0000 7fc0d3400640 1 -- 172.21.15.52:0/3941709554 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d415ab50 con 0x7fc0d40ccd40 2024-03-12T17:08:05.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.327+0000 7fc0d3e00640 1 --2- 172.21.15.52:0/3941709554 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc0d40707e0 0x7fc0d415e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:08:05.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.329+0000 7fc0d3400640 1 --2- 172.21.15.52:0/3941709554 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0d40ccd40 0x7fc0d415b630 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4009870 tx=0x7fc0c402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd3211b88f9fae0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:05.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.329+0000 7fc0d2000640 1 -- 172.21.15.52:0/3941709554 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4031640 con 0x7fc0d40ccd40 2024-03-12T17:08:05.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.329+0000 7fc0d2000640 1 -- 172.21.15.52:0/3941709554 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c40317a0 con 0x7fc0d40ccd40 2024-03-12T17:08:05.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.329+0000 7fc0d2000640 1 -- 172.21.15.52:0/3941709554 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4031b30 con 0x7fc0d40ccd40 2024-03-12T17:08:05.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.330+0000 7fc0d8c00640 1 -- 172.21.15.52:0/3941709554 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0d40ccd40 msgr2=0x7fc0d415b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:05.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.330+0000 7fc0d8c00640 1 --2- 172.21.15.52:0/3941709554 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0d40ccd40 0x7fc0d415b630 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4009870 tx=0x7fc0c402b8f0 comp rx=0 tx=0).stop 2024-03-12T17:08:05.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.330+0000 7fc0d8c00640 1 -- 172.21.15.52:0/3941709554 shutdown_connections 2024-03-12T17:08:05.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.330+0000 7fc0d8c00640 1 --2- 172.21.15.52:0/3941709554 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc0d40707e0 0x7fc0d415e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:05.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.330+0000 7fc0d8c00640 1 --2- 172.21.15.52:0/3941709554 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0d40ccd40 0x7fc0d415b630 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:05.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.330+0000 7fc0d8c00640 1 -- 172.21.15.52:0/3941709554 >> 172.21.15.52:0/3941709554 conn(0x7fc0d4065df0 msgr2=0x7fc0d4069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:05.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.330+0000 7fc0d8c00640 1 -- 172.21.15.52:0/3941709554 shutdown_connections 2024-03-12T17:08:05.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.330+0000 7fc0d8c00640 1 -- 172.21.15.52:0/3941709554 wait complete. 2024-03-12T17:08:05.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.332+0000 7fc0d8c00640 1 Processor -- start 2024-03-12T17:08:05.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.333+0000 7fc0d8c00640 1 -- start start 2024-03-12T17:08:05.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.334+0000 7fc0d8c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0d40707e0 0x7fc0d4144e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:05.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.334+0000 7fc0d8c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0d40ccd40 0x7fc0d41f0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:05.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.334+0000 7fc0d8c00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc0d415e030 0x7fc0d40ac7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:05.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.334+0000 7fc0d8c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d4145380 con 0x7fc0d40ccd40 2024-03-12T17:08:05.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.334+0000 7fc0d8c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d40abac0 con 0x7fc0d40707e0 2024-03-12T17:08:05.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.334+0000 7fc0d8c00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d41f0d60 con 0x7fc0d415e030 2024-03-12T17:08:05.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.334+0000 7fc0d2a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0d40ccd40 0x7fc0d41f0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:05.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.334+0000 7fc0d2a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0d40ccd40 0x7fc0d41f0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40560/0 (socket says 172.21.15.52:40560) 2024-03-12T17:08:05.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.334+0000 7fc0d2a00640 1 -- 172.21.15.52:0/510568926 learned_addr learned my addr 172.21.15.52:0/510568926 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:05.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.335+0000 7fc0d3400640 1 --2- 172.21.15.52:0/510568926 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0d40707e0 0x7fc0d4144e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:05.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.335+0000 7fc0d2a00640 1 -- 172.21.15.52:0/510568926 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc0d415e030 msgr2=0x7fc0d40ac7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:05.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.335+0000 7fc0d2a00640 1 --2- 172.21.15.52:0/510568926 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc0d415e030 0x7fc0d40ac7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:05.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.335+0000 7fc0d2a00640 1 -- 172.21.15.52:0/510568926 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0d40707e0 msgr2=0x7fc0d4144e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:05.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.335+0000 7fc0d2a00640 1 --2- 172.21.15.52:0/510568926 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0d40707e0 0x7fc0d4144e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:05.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.335+0000 7fc0d2a00640 1 -- 172.21.15.52:0/510568926 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c40094d0 con 0x7fc0d40ccd40 2024-03-12T17:08:05.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.335+0000 7fc0d2a00640 1 --2- 172.21.15.52:0/510568926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0d40ccd40 0x7fc0d41f0820 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000c400 tx=0x7fc0c000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:05.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.335+0000 7fc0cb400640 1 -- 172.21.15.52:0/510568926 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c00052d0 con 0x7fc0d40ccd40 2024-03-12T17:08:05.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.335+0000 7fc0cb400640 1 -- 172.21.15.52:0/510568926 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c0005430 con 0x7fc0d40ccd40 2024-03-12T17:08:05.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.335+0000 7fc0cb400640 1 -- 172.21.15.52:0/510568926 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0003830 con 0x7fc0d40ccd40 2024-03-12T17:08:05.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.335+0000 7fc0d8c00640 1 -- 172.21.15.52:0/510568926 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0d40ab6f0 con 0x7fc0d40ccd40 2024-03-12T17:08:05.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.336+0000 7fc0d8c00640 1 -- 172.21.15.52:0/510568926 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0d40abff0 con 0x7fc0d40ccd40 2024-03-12T17:08:05.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.337+0000 7fc0d8c00640 1 -- 172.21.15.52:0/510568926 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0d415ee60 con 0x7fc0d40ccd40 2024-03-12T17:08:05.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.338+0000 7fc0cb400640 1 -- 172.21.15.52:0/510568926 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc0c0003990 con 0x7fc0d40ccd40 2024-03-12T17:08:05.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.338+0000 7fc0cb400640 1 --2- 172.21.15.52:0/510568926 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc0b406ac00 0x7fc0b406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:05.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.338+0000 7fc0cb400640 1 -- 172.21.15.52:0/510568926 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fc0c0088760 con 0x7fc0d40ccd40 2024-03-12T17:08:05.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.339+0000 7fc0d3400640 1 --2- 172.21.15.52:0/510568926 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc0b406ac00 0x7fc0b406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:05.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.340+0000 7fc0d3400640 1 --2- 172.21.15.52:0/510568926 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc0b406ac00 0x7fc0b406d0c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4036000 tx=0x7fc0c403c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:05.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.343+0000 7fc0cb400640 1 -- 172.21.15.52:0/510568926 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fc0d415ee60 con 0x7fc0d40ccd40 2024-03-12T17:08:05.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.615+0000 7fc0d8c00640 1 -- 172.21.15.52:0/510568926 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fc0d415ee60 con 0x7fc0d40ccd40 2024-03-12T17:08:05.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.620+0000 7fc0cb400640 1 -- 172.21.15.52:0/510568926 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7fc0d415ee60 con 0x7fc0d40ccd40 2024-03-12T17:08:05.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.622+0000 7fc0d8c00640 1 -- 172.21.15.52:0/510568926 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc0b406ac00 msgr2=0x7fc0b406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:05.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.622+0000 7fc0d8c00640 1 --2- 172.21.15.52:0/510568926 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc0b406ac00 0x7fc0b406d0c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4036000 tx=0x7fc0c403c040 comp rx=0 tx=0).stop 2024-03-12T17:08:05.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.623+0000 7fc0d8c00640 1 -- 172.21.15.52:0/510568926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0d40ccd40 msgr2=0x7fc0d41f0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:05.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.623+0000 7fc0d8c00640 1 --2- 172.21.15.52:0/510568926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0d40ccd40 0x7fc0d41f0820 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000c400 tx=0x7fc0c000c8c0 comp rx=0 tx=0).stop 2024-03-12T17:08:05.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.623+0000 7fc0d8c00640 1 -- 172.21.15.52:0/510568926 shutdown_connections 2024-03-12T17:08:05.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.623+0000 7fc0d8c00640 1 --2- 172.21.15.52:0/510568926 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc0b406ac00 0x7fc0b406d0c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:05.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.623+0000 7fc0d8c00640 1 --2- 172.21.15.52:0/510568926 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc0d415e030 0x7fc0d40ac7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:05.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.623+0000 7fc0d8c00640 1 --2- 172.21.15.52:0/510568926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0d40ccd40 0x7fc0d41f0820 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:05.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.623+0000 7fc0d8c00640 1 --2- 172.21.15.52:0/510568926 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0d40707e0 0x7fc0d4144e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:05.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.623+0000 7fc0d8c00640 1 -- 172.21.15.52:0/510568926 >> 172.21.15.52:0/510568926 conn(0x7fc0d4065df0 msgr2=0x7fc0d40726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:05.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.623+0000 7fc0d8c00640 1 -- 172.21.15.52:0/510568926 shutdown_connections 2024-03-12T17:08:05.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.623+0000 7fc0d8c00640 1 -- 172.21.15.52:0/510568926 wait complete. 2024-03-12T17:08:05.637 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-03-12T17:08:05.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.869+0000 7fdf6be00640 1 Processor -- start 2024-03-12T17:08:05.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.870+0000 7fdf6be00640 1 -- start start 2024-03-12T17:08:05.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.871+0000 7fdf6be00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdf6c15c890 0x7fdf6c160480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:05.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.871+0000 7fdf6be00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf6c161390 con 0x7fdf6c0b80a0 2024-03-12T17:08:05.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.871+0000 7fdf6be00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf6c160dd0 con 0x7fdf6c0cdb80 2024-03-12T17:08:05.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.871+0000 7fdf6be00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6c0d0ca0 con 0x7fdf6c15c890 2024-03-12T17:08:05.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.872+0000 7fdf63e00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fdf6c0b80a0 0x7fdf6c0cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:44402/0 (socket says 172.21.15.52:44402) 2024-03-12T17:08:05.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.872+0000 7fdf63e00640 1 -- 172.21.15.52:0/483690830 learned_addr learned my addr 172.21.15.52:0/483690830 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:08:05.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.872+0000 7fdf6aa00640 1 --2- 172.21.15.52:0/483690830 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdf6c15c890 0x7fdf6c160480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:05.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.872+0000 7fdf6a000640 1 -- 172.21.15.52:0/483690830 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4209731911 0 0) 0x7fdf6c161390 con 0x7fdf6c0b80a0 2024-03-12T17:08:05.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.872+0000 7fdf6a000640 1 -- 172.21.15.52:0/483690830 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf50003520 con 0x7fdf6c0b80a0 2024-03-12T17:08:05.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.872+0000 7fdf6a000640 1 -- 172.21.15.52:0/483690830 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1303451520 0 0) 0x7fdf50003520 con 0x7fdf6c0b80a0 2024-03-12T17:08:05.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.873+0000 7fdf6a000640 1 -- 172.21.15.52:0/483690830 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdf6c15c890 msgr2=0x7fdf6c160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:05.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.873+0000 7fdf6a000640 1 --2- 172.21.15.52:0/483690830 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdf6c15c890 0x7fdf6c160480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:05.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.873+0000 7fdf6a000640 1 -- 172.21.15.52:0/483690830 >> v1:172.21.15.102:6789/0 conn(0x7fdf6c0cdb80 legacy=0x7fdf6c0d10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:05.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.873+0000 7fdf6a000640 1 -- 172.21.15.52:0/483690830 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf6c1610b0 con 0x7fdf6c0b80a0 2024-03-12T17:08:05.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.873+0000 7fdf6a000640 1 -- 172.21.15.52:0/483690830 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7fdf54001670 con 0x7fdf6c0b80a0 2024-03-12T17:08:05.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.873+0000 7fdf6a000640 1 -- 172.21.15.52:0/483690830 >> v1:172.21.15.52:6789/0 conn(0x7fdf6c0b80a0 legacy=0x7fdf6c0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:05.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.873+0000 7fdf6a000640 1 --2- 172.21.15.52:0/483690830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf500039a0 0x7fdf50003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:05.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.873+0000 7fdf6a000640 1 --2- 172.21.15.52:0/483690830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf500047c0 0x7fdf50004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:05.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.873+0000 7fdf6a000640 1 --2- 172.21.15.52:0/483690830 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdf6c15c890 0x7fdf50005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:05.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.873+0000 7fdf6a000640 1 -- 172.21.15.52:0/483690830 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6c0d0ca0 con 0x7fdf500039a0 2024-03-12T17:08:05.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.873+0000 7fdf6a000640 1 -- 172.21.15.52:0/483690830 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5000bb10 con 0x7fdf500047c0 2024-03-12T17:08:05.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.873+0000 7fdf6a000640 1 -- 172.21.15.52:0/483690830 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5000bcd0 con 0x7fdf6c15c890 2024-03-12T17:08:05.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.873+0000 7fdf6aa00640 1 --2- 172.21.15.52:0/483690830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf500039a0 0x7fdf50003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:05.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.873+0000 7fdf63e00640 1 --2- 172.21.15.52:0/483690830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf500047c0 0x7fdf50004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:05.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.873+0000 7fdf6b400640 1 --2- 172.21.15.52:0/483690830 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdf6c15c890 0x7fdf50005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:05.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.873+0000 7fdf6aa00640 1 -- 172.21.15.52:0/483690830 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdf6c15c890 msgr2=0x7fdf50005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:05.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.873+0000 7fdf6aa00640 1 --2- 172.21.15.52:0/483690830 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdf6c15c890 0x7fdf50005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:05.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.873+0000 7fdf6aa00640 1 -- 172.21.15.52:0/483690830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf500047c0 msgr2=0x7fdf50004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:05.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.873+0000 7fdf6aa00640 1 --2- 172.21.15.52:0/483690830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf500047c0 0x7fdf50004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:05.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.873+0000 7fdf6aa00640 1 -- 172.21.15.52:0/483690830 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf50003520 con 0x7fdf500039a0 2024-03-12T17:08:05.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.873+0000 7fdf6b400640 1 --2- 172.21.15.52:0/483690830 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdf6c15c890 0x7fdf50005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:08:05.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.873+0000 7fdf63e00640 1 --2- 172.21.15.52:0/483690830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf500047c0 0x7fdf50004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:08:05.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.875+0000 7fdf6aa00640 1 --2- 172.21.15.52:0/483690830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf500039a0 0x7fdf50003dc0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c009ac0 tx=0x7fdf500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511e7b0f8bc9dac9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:05.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.875+0000 7fdf6a000640 1 -- 172.21.15.52:0/483690830 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c02c6b0 con 0x7fdf500039a0 2024-03-12T17:08:05.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.875+0000 7fdf6a000640 1 -- 172.21.15.52:0/483690830 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5c02c810 con 0x7fdf500039a0 2024-03-12T17:08:05.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.875+0000 7fdf6a000640 1 -- 172.21.15.52:0/483690830 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c02cba0 con 0x7fdf500039a0 2024-03-12T17:08:05.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.876+0000 7fdf6be00640 1 -- 172.21.15.52:0/483690830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf500039a0 msgr2=0x7fdf50003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:05.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.876+0000 7fdf6be00640 1 --2- 172.21.15.52:0/483690830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf500039a0 0x7fdf50003dc0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c009ac0 tx=0x7fdf500057f0 comp rx=0 tx=0).stop 2024-03-12T17:08:05.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.876+0000 7fdf6b400640 1 -- 172.21.15.52:0/483690830 reap_dead start 2024-03-12T17:08:05.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.876+0000 7fdf6be00640 1 -- 172.21.15.52:0/483690830 shutdown_connections 2024-03-12T17:08:05.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.876+0000 7fdf6be00640 1 -- 172.21.15.52:0/483690830 >> 172.21.15.52:0/483690830 conn(0x7fdf6c06ae90 msgr2=0x7fdf6c06b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:05.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.876+0000 7fdf6be00640 1 -- 172.21.15.52:0/483690830 shutdown_connections 2024-03-12T17:08:05.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.876+0000 7fdf6be00640 1 -- 172.21.15.52:0/483690830 wait complete. 2024-03-12T17:08:05.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.878+0000 7fdf6be00640 1 Processor -- start 2024-03-12T17:08:05.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.878+0000 7fdf6be00640 1 -- start start 2024-03-12T17:08:05.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.879+0000 7fdf6be00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdf500039a0 0x7fdf6c14dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:05.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.879+0000 7fdf6be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf6c159210 0x7fdf6c159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:05.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.879+0000 7fdf6be00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf6c15aaf0 0x7fdf6c15af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:05.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.879+0000 7fdf6be00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6c0c99d0 con 0x7fdf6c15aaf0 2024-03-12T17:08:05.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.879+0000 7fdf6be00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6c15b4b0 con 0x7fdf6c159210 2024-03-12T17:08:05.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.879+0000 7fdf6be00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6c0c9f50 con 0x7fdf500039a0 2024-03-12T17:08:05.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.879+0000 7fdf6b400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf6c15aaf0 0x7fdf6c15af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:05.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.879+0000 7fdf6b400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf6c15aaf0 0x7fdf6c15af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40584/0 (socket says 172.21.15.52:40584) 2024-03-12T17:08:05.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.879+0000 7fdf6b400640 1 -- 172.21.15.52:0/1957004239 learned_addr learned my addr 172.21.15.52:0/1957004239 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:05.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.879+0000 7fdf63e00640 1 --2- 172.21.15.52:0/1957004239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf6c159210 0x7fdf6c159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:05.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.879+0000 7fdf6aa00640 1 --2- 172.21.15.52:0/1957004239 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdf500039a0 0x7fdf6c14dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:05.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.880+0000 7fdf6b400640 1 -- 172.21.15.52:0/1957004239 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdf500039a0 msgr2=0x7fdf6c14dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:05.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.880+0000 7fdf6b400640 1 --2- 172.21.15.52:0/1957004239 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdf500039a0 0x7fdf6c14dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:05.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.880+0000 7fdf6b400640 1 -- 172.21.15.52:0/1957004239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf6c159210 msgr2=0x7fdf6c159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:05.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.880+0000 7fdf6b400640 1 --2- 172.21.15.52:0/1957004239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf6c159210 0x7fdf6c159690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:05.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.880+0000 7fdf6b400640 1 -- 172.21.15.52:0/1957004239 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf6c15cdf0 con 0x7fdf6c15aaf0 2024-03-12T17:08:05.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.880+0000 7fdf6b400640 1 --2- 172.21.15.52:0/1957004239 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf6c15aaf0 0x7fdf6c15af70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c0b5240 tx=0x7fdf6400d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:05.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.880+0000 7fdf62a00640 1 -- 172.21.15.52:0/1957004239 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf64017070 con 0x7fdf6c15aaf0 2024-03-12T17:08:05.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.880+0000 7fdf62a00640 1 -- 172.21.15.52:0/1957004239 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf64006d60 con 0x7fdf6c15aaf0 2024-03-12T17:08:05.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.880+0000 7fdf6be00640 1 -- 172.21.15.52:0/1957004239 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf6c0cc7b0 con 0x7fdf6c15aaf0 2024-03-12T17:08:05.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.880+0000 7fdf62a00640 1 -- 172.21.15.52:0/1957004239 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf64003bd0 con 0x7fdf6c15aaf0 2024-03-12T17:08:05.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.880+0000 7fdf6be00640 1 -- 172.21.15.52:0/1957004239 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf6c159d80 con 0x7fdf6c15aaf0 2024-03-12T17:08:05.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.881+0000 7fdf6be00640 1 -- 172.21.15.52:0/1957004239 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf6c0ca0e0 con 0x7fdf6c15aaf0 2024-03-12T17:08:05.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.882+0000 7fdf62a00640 1 -- 172.21.15.52:0/1957004239 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fdf64003d30 con 0x7fdf6c15aaf0 2024-03-12T17:08:05.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.883+0000 7fdf62a00640 1 --2- 172.21.15.52:0/1957004239 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fdf4806ac00 0x7fdf4806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:05.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.884+0000 7fdf62a00640 1 -- 172.21.15.52:0/1957004239 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fdf64088870 con 0x7fdf6c15aaf0 2024-03-12T17:08:05.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.884+0000 7fdf6aa00640 1 --2- 172.21.15.52:0/1957004239 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fdf4806ac00 0x7fdf4806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:05.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.885+0000 7fdf6aa00640 1 --2- 172.21.15.52:0/1957004239 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fdf4806ac00 0x7fdf4806d0c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c009ac0 tx=0x7fdf5c030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:05.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:05.888+0000 7fdf62a00640 1 -- 172.21.15.52:0/1957004239 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fdf64053c50 con 0x7fdf6c15aaf0 2024-03-12T17:08:06.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.158+0000 7fdf6be00640 1 -- 172.21.15.52:0/1957004239 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fdf6c06c420 con 0x7fdf6c15aaf0 2024-03-12T17:08:06.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.166+0000 7fdf62a00640 1 -- 172.21.15.52:0/1957004239 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7fdf64057270 con 0x7fdf6c15aaf0 2024-03-12T17:08:06.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.169+0000 7fdf6be00640 1 -- 172.21.15.52:0/1957004239 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fdf4806ac00 msgr2=0x7fdf4806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:06.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.169+0000 7fdf6be00640 1 --2- 172.21.15.52:0/1957004239 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fdf4806ac00 0x7fdf4806d0c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c009ac0 tx=0x7fdf5c030000 comp rx=0 tx=0).stop 2024-03-12T17:08:06.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.169+0000 7fdf6be00640 1 -- 172.21.15.52:0/1957004239 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf6c15aaf0 msgr2=0x7fdf6c15af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:06.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.169+0000 7fdf6be00640 1 --2- 172.21.15.52:0/1957004239 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf6c15aaf0 0x7fdf6c15af70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c0b5240 tx=0x7fdf6400d9d0 comp rx=0 tx=0).stop 2024-03-12T17:08:06.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.169+0000 7fdf6be00640 1 -- 172.21.15.52:0/1957004239 shutdown_connections 2024-03-12T17:08:06.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.169+0000 7fdf6be00640 1 --2- 172.21.15.52:0/1957004239 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fdf4806ac00 0x7fdf4806d0c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:06.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.169+0000 7fdf6be00640 1 --2- 172.21.15.52:0/1957004239 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf6c15aaf0 0x7fdf6c15af70 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:06.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.169+0000 7fdf6be00640 1 --2- 172.21.15.52:0/1957004239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf6c159210 0x7fdf6c159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:06.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.169+0000 7fdf6be00640 1 --2- 172.21.15.52:0/1957004239 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdf500039a0 0x7fdf6c14dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:06.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.169+0000 7fdf6be00640 1 -- 172.21.15.52:0/1957004239 >> 172.21.15.52:0/1957004239 conn(0x7fdf6c06ae90 msgr2=0x7fdf6c0b69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:06.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.169+0000 7fdf6be00640 1 -- 172.21.15.52:0/1957004239 shutdown_connections 2024-03-12T17:08:06.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.169+0000 7fdf6be00640 1 -- 172.21.15.52:0/1957004239 wait complete. 2024-03-12T17:08:06.180 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-03-12T17:08:06.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.394+0000 7f821ca00640 1 Processor -- start 2024-03-12T17:08:06.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.395+0000 7f821ca00640 1 -- start start 2024-03-12T17:08:06.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.396+0000 7f821ca00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f82180cad70 0x7f82180cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:06.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.396+0000 7f821ca00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8218141410 0x7f8218141880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:06.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.396+0000 7f821ca00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82180b25b0 con 0x7f8218140cf0 2024-03-12T17:08:06.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.396+0000 7f821ca00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82180cb720 con 0x7f82180cad70 2024-03-12T17:08:06.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.396+0000 7f821ca00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82181410d0 con 0x7f8218141410 2024-03-12T17:08:06.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.397+0000 7f820ea00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f82180cad70 0x7f82180cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:06.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.397+0000 7f820ea00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f82180cad70 0x7f82180cb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40592/0 (socket says 172.21.15.52:40592) 2024-03-12T17:08:06.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.397+0000 7f820ea00640 1 -- 172.21.15.52:0/1588191224 learned_addr learned my addr 172.21.15.52:0/1588191224 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:06.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.397+0000 7f8217e00640 1 --2- 172.21.15.52:0/1588191224 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8218141410 0x7f8218141880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:06.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.397+0000 7f820ea00640 1 -- 172.21.15.52:0/1588191224 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8218141410 msgr2=0x7f8218141880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:06.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.397+0000 7f820ea00640 1 --2- 172.21.15.52:0/1588191224 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8218141410 0x7f8218141880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:06.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.397+0000 7f820ea00640 1 -- 172.21.15.52:0/1588191224 >> v1:172.21.15.52:6789/0 conn(0x7f8218140cf0 legacy=0x7f82180b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:06.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.397+0000 7f820ea00640 1 -- 172.21.15.52:0/1588191224 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82180cd1b0 con 0x7f82180cad70 2024-03-12T17:08:06.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.397+0000 7f8217e00640 1 --2- 172.21.15.52:0/1588191224 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8218141410 0x7f8218141880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:08:06.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.398+0000 7f820ea00640 1 --2- 172.21.15.52:0/1588191224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82180cad70 0x7f82180cb1e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8200009870 tx=0x7f820002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2417bf4595be82e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:06.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.399+0000 7f8216a00640 1 -- 172.21.15.52:0/1588191224 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8200030840 con 0x7f82180cad70 2024-03-12T17:08:06.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.399+0000 7f8216a00640 1 -- 172.21.15.52:0/1588191224 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82000309a0 con 0x7f82180cad70 2024-03-12T17:08:06.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.399+0000 7f8216a00640 1 -- 172.21.15.52:0/1588191224 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8200030d30 con 0x7f82180cad70 2024-03-12T17:08:06.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.399+0000 7f821ca00640 1 -- 172.21.15.52:0/1588191224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82180cad70 msgr2=0x7f82180cb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:06.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.399+0000 7f821ca00640 1 --2- 172.21.15.52:0/1588191224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82180cad70 0x7f82180cb1e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8200009870 tx=0x7f820002b980 comp rx=0 tx=0).stop 2024-03-12T17:08:06.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.399+0000 7f821ca00640 1 -- 172.21.15.52:0/1588191224 shutdown_connections 2024-03-12T17:08:06.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.399+0000 7f821ca00640 1 --2- 172.21.15.52:0/1588191224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82180cad70 0x7f82180cb1e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:06.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.399+0000 7f821ca00640 1 --2- 172.21.15.52:0/1588191224 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8218141410 0x7f8218141880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:06.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.399+0000 7f821ca00640 1 -- 172.21.15.52:0/1588191224 >> 172.21.15.52:0/1588191224 conn(0x7f8218065e20 msgr2=0x7f8218069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:06.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.400+0000 7f821ca00640 1 -- 172.21.15.52:0/1588191224 shutdown_connections 2024-03-12T17:08:06.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.400+0000 7f821ca00640 1 -- 172.21.15.52:0/1588191224 wait complete. 2024-03-12T17:08:06.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.402+0000 7f821ca00640 1 Processor -- start 2024-03-12T17:08:06.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.402+0000 7f821ca00640 1 -- start start 2024-03-12T17:08:06.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.403+0000 7f821ca00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82180cad70 0x7f821815bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:06.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.403+0000 7f821ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8218140cf0 0x7f8218135cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:06.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.403+0000 7f821ca00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8218141410 0x7f821815cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:06.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.404+0000 7f821ca00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8218136210 con 0x7f82180cad70 2024-03-12T17:08:06.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.404+0000 7f821ca00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82180ac8a0 con 0x7f8218140cf0 2024-03-12T17:08:06.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.404+0000 7f821ca00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82180abc90 con 0x7f8218141410 2024-03-12T17:08:06.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.404+0000 7f8217400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82180cad70 0x7f821815bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:06.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.404+0000 7f8217400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82180cad70 0x7f821815bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40606/0 (socket says 172.21.15.52:40606) 2024-03-12T17:08:06.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.404+0000 7f8217400640 1 -- 172.21.15.52:0/2065430377 learned_addr learned my addr 172.21.15.52:0/2065430377 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:06.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.404+0000 7f8217400640 1 -- 172.21.15.52:0/2065430377 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8218141410 msgr2=0x7f821815cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:06.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.404+0000 7f8217400640 1 --2- 172.21.15.52:0/2065430377 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8218141410 0x7f821815cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:06.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.404+0000 7f8217400640 1 -- 172.21.15.52:0/2065430377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8218140cf0 msgr2=0x7f8218135cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:06.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.404+0000 7f8217400640 1 --2- 172.21.15.52:0/2065430377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8218140cf0 0x7f8218135cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:06.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.404+0000 7f8217400640 1 -- 172.21.15.52:0/2065430377 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82000094d0 con 0x7f82180cad70 2024-03-12T17:08:06.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.404+0000 7f8217400640 1 --2- 172.21.15.52:0/2065430377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82180cad70 0x7f821815bc20 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f820800c400 tx=0x7f820800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:06.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.405+0000 7f820fe00640 1 -- 172.21.15.52:0/2065430377 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82080052d0 con 0x7f82180cad70 2024-03-12T17:08:06.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.405+0000 7f820fe00640 1 -- 172.21.15.52:0/2065430377 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8208005450 con 0x7f82180cad70 2024-03-12T17:08:06.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.405+0000 7f820fe00640 1 -- 172.21.15.52:0/2065430377 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8208007260 con 0x7f82180cad70 2024-03-12T17:08:06.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.405+0000 7f821ca00640 1 -- 172.21.15.52:0/2065430377 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82180abf70 con 0x7f82180cad70 2024-03-12T17:08:06.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.405+0000 7f821ca00640 1 -- 172.21.15.52:0/2065430377 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82180ac530 con 0x7f82180cad70 2024-03-12T17:08:06.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.406+0000 7f821ca00640 1 -- 172.21.15.52:0/2065430377 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82180ac6c0 con 0x7f82180cad70 2024-03-12T17:08:06.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.407+0000 7f820fe00640 1 -- 172.21.15.52:0/2065430377 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f82080073c0 con 0x7f82180cad70 2024-03-12T17:08:06.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.408+0000 7f820fe00640 1 --2- 172.21.15.52:0/2065430377 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f81f406ac30 0x7f81f406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:06.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.408+0000 7f820fe00640 1 -- 172.21.15.52:0/2065430377 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f82080885d0 con 0x7f82180cad70 2024-03-12T17:08:06.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.408+0000 7f820ea00640 1 --2- 172.21.15.52:0/2065430377 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f81f406ac30 0x7f81f406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:06.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.410+0000 7f820ea00640 1 --2- 172.21.15.52:0/2065430377 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f81f406ac30 0x7f81f406d0f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8200036000 tx=0x7f820003b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:06.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.412+0000 7f820fe00640 1 -- 172.21.15.52:0/2065430377 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f8208053920 con 0x7f82180cad70 2024-03-12T17:08:06.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.685+0000 7f821ca00640 1 -- 172.21.15.52:0/2065430377 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f821804e030 con 0x7f82180cad70 2024-03-12T17:08:06.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.694+0000 7f820fe00640 1 -- 172.21.15.52:0/2065430377 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8208056f40 con 0x7f82180cad70 2024-03-12T17:08:06.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.697+0000 7f821ca00640 1 -- 172.21.15.52:0/2065430377 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f81f406ac30 msgr2=0x7f81f406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:06.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.697+0000 7f821ca00640 1 --2- 172.21.15.52:0/2065430377 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f81f406ac30 0x7f81f406d0f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8200036000 tx=0x7f820003b040 comp rx=0 tx=0).stop 2024-03-12T17:08:06.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.697+0000 7f821ca00640 1 -- 172.21.15.52:0/2065430377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82180cad70 msgr2=0x7f821815bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:06.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.697+0000 7f821ca00640 1 --2- 172.21.15.52:0/2065430377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82180cad70 0x7f821815bc20 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f820800c400 tx=0x7f820800c8c0 comp rx=0 tx=0).stop 2024-03-12T17:08:06.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.697+0000 7f821ca00640 1 -- 172.21.15.52:0/2065430377 shutdown_connections 2024-03-12T17:08:06.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.697+0000 7f821ca00640 1 --2- 172.21.15.52:0/2065430377 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f81f406ac30 0x7f81f406d0f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:06.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.697+0000 7f821ca00640 1 --2- 172.21.15.52:0/2065430377 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8218141410 0x7f821815cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:06.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.697+0000 7f821ca00640 1 --2- 172.21.15.52:0/2065430377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8218140cf0 0x7f8218135cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:06.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.697+0000 7f821ca00640 1 --2- 172.21.15.52:0/2065430377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82180cad70 0x7f821815bc20 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:06.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.697+0000 7f821ca00640 1 -- 172.21.15.52:0/2065430377 >> 172.21.15.52:0/2065430377 conn(0x7f8218065e20 msgr2=0x7f8218141860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:06.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.697+0000 7f821ca00640 1 -- 172.21.15.52:0/2065430377 shutdown_connections 2024-03-12T17:08:06.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.697+0000 7f821ca00640 1 -- 172.21.15.52:0/2065430377 wait complete. 2024-03-12T17:08:06.709 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-03-12T17:08:06.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.930+0000 7f4d9fe00640 1 Processor -- start 2024-03-12T17:08:06.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.931+0000 7f4d9fe00640 1 -- start start 2024-03-12T17:08:06.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.932+0000 7f4d9fe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4da015b630 0x7f4da00700d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:06.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.932+0000 7f4d9fe00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4da0070610 0x7f4da006c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:06.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.932+0000 7f4d9fe00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da015b1a0 con 0x7f4da00ccd40 2024-03-12T17:08:06.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.932+0000 7f4d9fe00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4da0070aa0 con 0x7f4da015b630 2024-03-12T17:08:06.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.932+0000 7f4d9fe00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da006d940 con 0x7f4da0070610 2024-03-12T17:08:06.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.932+0000 7f4d97e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4da015b630 0x7f4da00700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:06.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.932+0000 7f4d97e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4da015b630 0x7f4da00700d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40614/0 (socket says 172.21.15.52:40614) 2024-03-12T17:08:06.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.932+0000 7f4d97e00640 1 -- 172.21.15.52:0/1384616114 learned_addr learned my addr 172.21.15.52:0/1384616114 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:06.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.933+0000 7f4d9f400640 1 --2- 172.21.15.52:0/1384616114 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4da0070610 0x7f4da006c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:06.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.933+0000 7f4d97e00640 1 -- 172.21.15.52:0/1384616114 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4da0070610 msgr2=0x7f4da006c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:06.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.933+0000 7f4d97e00640 1 --2- 172.21.15.52:0/1384616114 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4da0070610 0x7f4da006c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:06.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.933+0000 7f4d97e00640 1 -- 172.21.15.52:0/1384616114 >> v1:172.21.15.102:6789/0 conn(0x7f4da00ccd40 legacy=0x7f4da015e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:06.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.933+0000 7f4d97e00640 1 -- 172.21.15.52:0/1384616114 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da015eba0 con 0x7f4da015b630 2024-03-12T17:08:06.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.933+0000 7f4d9f400640 1 --2- 172.21.15.52:0/1384616114 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4da0070610 0x7f4da006c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:08:06.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.934+0000 7f4d97e00640 1 --2- 172.21.15.52:0/1384616114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4da015b630 0x7f4da00700d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4d88009870 tx=0x7f4d8802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d20dac53b72ad1f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:06.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.934+0000 7f4d9e000640 1 -- 172.21.15.52:0/1384616114 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d880317c0 con 0x7f4da015b630 2024-03-12T17:08:06.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.935+0000 7f4d9e000640 1 -- 172.21.15.52:0/1384616114 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d88031920 con 0x7f4da015b630 2024-03-12T17:08:06.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.935+0000 7f4d9e000640 1 -- 172.21.15.52:0/1384616114 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d88031cb0 con 0x7f4da015b630 2024-03-12T17:08:06.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.935+0000 7f4d9fe00640 1 -- 172.21.15.52:0/1384616114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4da015b630 msgr2=0x7f4da00700d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:06.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.935+0000 7f4d9fe00640 1 --2- 172.21.15.52:0/1384616114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4da015b630 0x7f4da00700d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4d88009870 tx=0x7f4d8802b860 comp rx=0 tx=0).stop 2024-03-12T17:08:06.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.935+0000 7f4d9fe00640 1 -- 172.21.15.52:0/1384616114 shutdown_connections 2024-03-12T17:08:06.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.935+0000 7f4d9fe00640 1 --2- 172.21.15.52:0/1384616114 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4da0070610 0x7f4da006c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:06.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.935+0000 7f4d9fe00640 1 --2- 172.21.15.52:0/1384616114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4da015b630 0x7f4da00700d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:06.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.935+0000 7f4d9fe00640 1 -- 172.21.15.52:0/1384616114 >> 172.21.15.52:0/1384616114 conn(0x7f4da0065df0 msgr2=0x7f4da0069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:06.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.935+0000 7f4d9fe00640 1 -- 172.21.15.52:0/1384616114 shutdown_connections 2024-03-12T17:08:06.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.935+0000 7f4d9fe00640 1 -- 172.21.15.52:0/1384616114 wait complete. 2024-03-12T17:08:06.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.937+0000 7f4d9fe00640 1 Processor -- start 2024-03-12T17:08:06.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.937+0000 7f4d9fe00640 1 -- start start 2024-03-12T17:08:06.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.938+0000 7f4d9fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4da0070610 0x7f4da0151240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:06.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.938+0000 7f4d9fe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4da00ccd40 0x7f4da00b1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:06.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.938+0000 7f4d9fe00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4da015b630 0x7f4da015cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:06.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.938+0000 7f4d9fe00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da0151780 con 0x7f4da00ccd40 2024-03-12T17:08:06.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.938+0000 7f4d9fe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da01f1cf0 con 0x7f4da0070610 2024-03-12T17:08:06.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.938+0000 7f4d9fe00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da01579a0 con 0x7f4da015b630 2024-03-12T17:08:06.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.938+0000 7f4d97e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4da00ccd40 0x7f4da00b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:06.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.938+0000 7f4d97e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4da00ccd40 0x7f4da00b1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40618/0 (socket says 172.21.15.52:40618) 2024-03-12T17:08:06.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.938+0000 7f4d97e00640 1 -- 172.21.15.52:0/3445596262 learned_addr learned my addr 172.21.15.52:0/3445596262 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:06.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.939+0000 7f4d9ea00640 1 --2- 172.21.15.52:0/3445596262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4da0070610 0x7f4da0151240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:06.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.939+0000 7f4d9f400640 1 --2- 172.21.15.52:0/3445596262 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4da015b630 0x7f4da015cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:06.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.939+0000 7f4d97e00640 1 -- 172.21.15.52:0/3445596262 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4da015b630 msgr2=0x7f4da015cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:06.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.939+0000 7f4d97e00640 1 --2- 172.21.15.52:0/3445596262 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4da015b630 0x7f4da015cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:06.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.939+0000 7f4d97e00640 1 -- 172.21.15.52:0/3445596262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4da0070610 msgr2=0x7f4da0151240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:06.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.939+0000 7f4d97e00640 1 --2- 172.21.15.52:0/3445596262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4da0070610 0x7f4da0151240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:06.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.939+0000 7f4d97e00640 1 -- 172.21.15.52:0/3445596262 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d880094d0 con 0x7f4da00ccd40 2024-03-12T17:08:06.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.939+0000 7f4d9ea00640 1 --2- 172.21.15.52:0/3445596262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4da0070610 0x7f4da0151240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:08:06.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.939+0000 7f4d97e00640 1 --2- 172.21.15.52:0/3445596262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4da00ccd40 0x7f4da00b1770 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4d880090b0 tx=0x7f4d880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:06.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.940+0000 7f4d96a00640 1 -- 172.21.15.52:0/3445596262 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d880329a0 con 0x7f4da00ccd40 2024-03-12T17:08:06.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.940+0000 7f4d96a00640 1 -- 172.21.15.52:0/3445596262 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d88032b40 con 0x7f4da00ccd40 2024-03-12T17:08:06.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.940+0000 7f4d96a00640 1 -- 172.21.15.52:0/3445596262 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d880453f0 con 0x7f4da00ccd40 2024-03-12T17:08:06.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.940+0000 7f4d9fe00640 1 -- 172.21.15.52:0/3445596262 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4da01f1f40 con 0x7f4da00ccd40 2024-03-12T17:08:06.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.940+0000 7f4d9fe00640 1 -- 172.21.15.52:0/3445596262 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da01f2500 con 0x7f4da00ccd40 2024-03-12T17:08:06.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.941+0000 7f4d9fe00640 1 -- 172.21.15.52:0/3445596262 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4da01f26f0 con 0x7f4da00ccd40 2024-03-12T17:08:06.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.942+0000 7f4d96a00640 1 -- 172.21.15.52:0/3445596262 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4d8804e450 con 0x7f4da00ccd40 2024-03-12T17:08:06.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.943+0000 7f4d96a00640 1 --2- 172.21.15.52:0/3445596262 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4d7c06ac00 0x7f4d7c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:06.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.943+0000 7f4d96a00640 1 -- 172.21.15.52:0/3445596262 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f4d880ab4b0 con 0x7f4da00ccd40 2024-03-12T17:08:06.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.943+0000 7f4d9ea00640 1 --2- 172.21.15.52:0/3445596262 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4d7c06ac00 0x7f4d7c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:06.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.945+0000 7f4d9ea00640 1 --2- 172.21.15.52:0/3445596262 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4d7c06ac00 0x7f4d7c06d0c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4da00b3830 tx=0x7f4d90009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:06.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:06.947+0000 7f4d96a00640 1 -- 172.21.15.52:0/3445596262 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f4d880768a0 con 0x7f4da00ccd40 2024-03-12T17:08:07.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.219+0000 7f4d9fe00640 1 -- 172.21.15.52:0/3445596262 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f4da015e6c0 con 0x7f4da00ccd40 2024-03-12T17:08:07.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.228+0000 7f4d96a00640 1 -- 172.21.15.52:0/3445596262 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f4d88079ec0 con 0x7f4da00ccd40 2024-03-12T17:08:07.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.230+0000 7f4d9fe00640 1 -- 172.21.15.52:0/3445596262 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4d7c06ac00 msgr2=0x7f4d7c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:07.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.230+0000 7f4d9fe00640 1 --2- 172.21.15.52:0/3445596262 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4d7c06ac00 0x7f4d7c06d0c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4da00b3830 tx=0x7f4d90009040 comp rx=0 tx=0).stop 2024-03-12T17:08:07.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.230+0000 7f4d9fe00640 1 -- 172.21.15.52:0/3445596262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4da00ccd40 msgr2=0x7f4da00b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:07.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.230+0000 7f4d9fe00640 1 --2- 172.21.15.52:0/3445596262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4da00ccd40 0x7f4da00b1770 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4d880090b0 tx=0x7f4d880046c0 comp rx=0 tx=0).stop 2024-03-12T17:08:07.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.230+0000 7f4d9fe00640 1 -- 172.21.15.52:0/3445596262 shutdown_connections 2024-03-12T17:08:07.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.230+0000 7f4d9fe00640 1 --2- 172.21.15.52:0/3445596262 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4d7c06ac00 0x7f4d7c06d0c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:07.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.230+0000 7f4d9fe00640 1 --2- 172.21.15.52:0/3445596262 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4da015b630 0x7f4da015cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:07.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.231+0000 7f4d9fe00640 1 --2- 172.21.15.52:0/3445596262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4da00ccd40 0x7f4da00b1770 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:07.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.231+0000 7f4d9fe00640 1 --2- 172.21.15.52:0/3445596262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4da0070610 0x7f4da0151240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:07.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.231+0000 7f4d9fe00640 1 -- 172.21.15.52:0/3445596262 >> 172.21.15.52:0/3445596262 conn(0x7f4da0065df0 msgr2=0x7f4da00af920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:07.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.231+0000 7f4d9fe00640 1 -- 172.21.15.52:0/3445596262 shutdown_connections 2024-03-12T17:08:07.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.231+0000 7f4d9fe00640 1 -- 172.21.15.52:0/3445596262 wait complete. 2024-03-12T17:08:07.244 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:08:07.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.463+0000 7f50a7200640 1 Processor -- start 2024-03-12T17:08:07.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.463+0000 7f50a7200640 1 -- start start 2024-03-12T17:08:07.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.465+0000 7f50a7200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f50a80b3310 0x7f50a80cae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:07.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.465+0000 7f50a7200640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50a80cc8c0 con 0x7f50a80cb370 2024-03-12T17:08:07.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.465+0000 7f50a7200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50a815ce60 con 0x7f50a8156e30 2024-03-12T17:08:07.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.465+0000 7f50a7200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a80cb7e0 con 0x7f50a80b3310 2024-03-12T17:08:07.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.466+0000 7f50a6800640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f50a80cb370 0x7f50a80b1d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:44424/0 (socket says 172.21.15.52:44424) 2024-03-12T17:08:07.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.466+0000 7f50a6800640 1 -- 172.21.15.52:0/2993011453 learned_addr learned my addr 172.21.15.52:0/2993011453 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:08:07.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.466+0000 7f509fe00640 1 --2- 172.21.15.52:0/2993011453 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f50a80b3310 0x7f50a80cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:07.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.466+0000 7f50a5400640 1 -- 172.21.15.52:0/2993011453 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2129416700 0 0) 0x7f50a80cc8c0 con 0x7f50a80cb370 2024-03-12T17:08:07.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.466+0000 7f50a5400640 1 -- 172.21.15.52:0/2993011453 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f508c003520 con 0x7f50a80cb370 2024-03-12T17:08:07.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.467+0000 7f50a5400640 1 -- 172.21.15.52:0/2993011453 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1113925162 0 0) 0x7f508c003520 con 0x7f50a80cb370 2024-03-12T17:08:07.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.467+0000 7f50a5400640 1 -- 172.21.15.52:0/2993011453 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f50a80b3310 msgr2=0x7f50a80cae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:07.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.467+0000 7f50a5400640 1 --2- 172.21.15.52:0/2993011453 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f50a80b3310 0x7f50a80cae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:07.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.467+0000 7f50a5400640 1 -- 172.21.15.52:0/2993011453 >> v1:172.21.15.102:6789/0 conn(0x7f50a8156e30 legacy=0x7f50a806d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:07.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.467+0000 7f50a5400640 1 -- 172.21.15.52:0/2993011453 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a815d120 con 0x7f50a80cb370 2024-03-12T17:08:07.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.467+0000 7f50a5400640 1 -- 172.21.15.52:0/2993011453 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f50a0001a10 con 0x7f50a80cb370 2024-03-12T17:08:07.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.467+0000 7f50a5400640 1 -- 172.21.15.52:0/2993011453 >> v1:172.21.15.52:6789/0 conn(0x7f50a80cb370 legacy=0x7f50a80b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:07.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.467+0000 7f50a5400640 1 --2- 172.21.15.52:0/2993011453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f508c0039a0 0x7f508c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:07.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.467+0000 7f50a5400640 1 --2- 172.21.15.52:0/2993011453 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f50a80b3310 0x7f508c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:07.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.467+0000 7f50a5400640 1 --2- 172.21.15.52:0/2993011453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f508c004d00 0x7f508c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:07.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.467+0000 7f50a5400640 1 -- 172.21.15.52:0/2993011453 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a80cb7e0 con 0x7f508c004d00 2024-03-12T17:08:07.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.467+0000 7f50a5400640 1 -- 172.21.15.52:0/2993011453 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c00ba30 con 0x7f508c0039a0 2024-03-12T17:08:07.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.467+0000 7f50a5400640 1 -- 172.21.15.52:0/2993011453 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f508c00bbf0 con 0x7f50a80b3310 2024-03-12T17:08:07.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.467+0000 7f50a6800640 1 --2- 172.21.15.52:0/2993011453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f508c004d00 0x7f508c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:07.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.467+0000 7f50a5e00640 1 --2- 172.21.15.52:0/2993011453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f508c0039a0 0x7f508c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:07.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.467+0000 7f509fe00640 1 --2- 172.21.15.52:0/2993011453 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f50a80b3310 0x7f508c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:07.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.468+0000 7f50a6800640 1 -- 172.21.15.52:0/2993011453 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f50a80b3310 msgr2=0x7f508c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:07.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.468+0000 7f50a6800640 1 --2- 172.21.15.52:0/2993011453 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f50a80b3310 0x7f508c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:07.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.468+0000 7f50a6800640 1 -- 172.21.15.52:0/2993011453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f508c0039a0 msgr2=0x7f508c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:07.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.468+0000 7f50a6800640 1 --2- 172.21.15.52:0/2993011453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f508c0039a0 0x7f508c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:07.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.468+0000 7f50a6800640 1 -- 172.21.15.52:0/2993011453 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508c003520 con 0x7f508c004d00 2024-03-12T17:08:07.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.469+0000 7f50a6800640 1 --2- 172.21.15.52:0/2993011453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f508c004d00 0x7f508c005180 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f50a0002230 tx=0x7f508c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71fca7c73d1aaa4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:07.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.469+0000 7f50a5400640 1 -- 172.21.15.52:0/2993011453 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a0034630 con 0x7f508c004d00 2024-03-12T17:08:07.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.469+0000 7f50a5400640 1 -- 172.21.15.52:0/2993011453 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a002f070 con 0x7f508c004d00 2024-03-12T17:08:07.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.469+0000 7f50a5400640 1 -- 172.21.15.52:0/2993011453 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a0034990 con 0x7f508c004d00 2024-03-12T17:08:07.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.469+0000 7f50a7200640 1 -- 172.21.15.52:0/2993011453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f508c004d00 msgr2=0x7f508c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:07.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.469+0000 7f50a7200640 1 --2- 172.21.15.52:0/2993011453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f508c004d00 0x7f508c005180 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f50a0002230 tx=0x7f508c0057f0 comp rx=0 tx=0).stop 2024-03-12T17:08:07.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.470+0000 7f50a6800640 1 -- 172.21.15.52:0/2993011453 reap_dead start 2024-03-12T17:08:07.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.470+0000 7f50a7200640 1 -- 172.21.15.52:0/2993011453 shutdown_connections 2024-03-12T17:08:07.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.470+0000 7f50a7200640 1 -- 172.21.15.52:0/2993011453 >> 172.21.15.52:0/2993011453 conn(0x7f50a8065df0 msgr2=0x7f50a8069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:07.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.470+0000 7f50a7200640 1 -- 172.21.15.52:0/2993011453 shutdown_connections 2024-03-12T17:08:07.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.470+0000 7f50a7200640 1 -- 172.21.15.52:0/2993011453 wait complete. 2024-03-12T17:08:07.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.472+0000 7f50a7200640 1 Processor -- start 2024-03-12T17:08:07.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.472+0000 7f50a7200640 1 -- start start 2024-03-12T17:08:07.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.473+0000 7f50a7200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f508c004d00 0x7f50a814bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:07.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.473+0000 7f50a7200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50a814cdd0 0x7f50a80afb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:07.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.473+0000 7f50a7200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50a806cb00 0x7f50a806cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:07.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.473+0000 7f50a7200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a806d4c0 con 0x7f50a806cb00 2024-03-12T17:08:07.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.473+0000 7f50a7200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a814d270 con 0x7f50a814cdd0 2024-03-12T17:08:07.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.473+0000 7f50a7200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a814d550 con 0x7f508c004d00 2024-03-12T17:08:07.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.473+0000 7f50a6800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50a806cb00 0x7f50a806cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:07.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.473+0000 7f50a6800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50a806cb00 0x7f50a806cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40636/0 (socket says 172.21.15.52:40636) 2024-03-12T17:08:07.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.473+0000 7f50a6800640 1 -- 172.21.15.52:0/2170404992 learned_addr learned my addr 172.21.15.52:0/2170404992 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:07.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.473+0000 7f50a5e00640 1 --2- 172.21.15.52:0/2170404992 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f508c004d00 0x7f50a814bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:07.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.473+0000 7f509fe00640 1 --2- 172.21.15.52:0/2170404992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50a814cdd0 0x7f50a80afb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:07.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.474+0000 7f50a6800640 1 -- 172.21.15.52:0/2170404992 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f508c004d00 msgr2=0x7f50a814bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:07.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.474+0000 7f50a6800640 1 --2- 172.21.15.52:0/2170404992 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f508c004d00 0x7f50a814bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:07.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.474+0000 7f50a6800640 1 -- 172.21.15.52:0/2170404992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50a814cdd0 msgr2=0x7f50a80afb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:07.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.474+0000 7f50a6800640 1 --2- 172.21.15.52:0/2170404992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50a814cdd0 0x7f50a80afb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:07.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.474+0000 7f50a6800640 1 -- 172.21.15.52:0/2170404992 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a814d830 con 0x7f50a806cb00 2024-03-12T17:08:07.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.474+0000 7f50a5e00640 1 --2- 172.21.15.52:0/2170404992 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f508c004d00 0x7f50a814bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:08:07.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.474+0000 7f50a6800640 1 --2- 172.21.15.52:0/2170404992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50a806cb00 0x7f50a806cf80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f50a0000f80 tx=0x7f50a0001da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:07.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.474+0000 7f509e000640 1 -- 172.21.15.52:0/2170404992 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a0044070 con 0x7f50a806cb00 2024-03-12T17:08:07.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.474+0000 7f509e000640 1 -- 172.21.15.52:0/2170404992 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a0001250 con 0x7f50a806cb00 2024-03-12T17:08:07.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.474+0000 7f509e000640 1 -- 172.21.15.52:0/2170404992 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a0034840 con 0x7f50a806cb00 2024-03-12T17:08:07.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.474+0000 7f50a7200640 1 -- 172.21.15.52:0/2170404992 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a814db10 con 0x7f50a806cb00 2024-03-12T17:08:07.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.474+0000 7f50a7200640 1 -- 172.21.15.52:0/2170404992 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a814c2a0 con 0x7f50a806cb00 2024-03-12T17:08:07.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.476+0000 7f50a7200640 1 -- 172.21.15.52:0/2170404992 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50a804e030 con 0x7f50a806cb00 2024-03-12T17:08:07.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.476+0000 7f509e000640 1 -- 172.21.15.52:0/2170404992 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f50a0031030 con 0x7f50a806cb00 2024-03-12T17:08:07.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.477+0000 7f509e000640 1 --2- 172.21.15.52:0/2170404992 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f508406ac00 0x7f508406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:07.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.477+0000 7f509e000640 1 -- 172.21.15.52:0/2170404992 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f50a00aabc0 con 0x7f50a806cb00 2024-03-12T17:08:07.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.478+0000 7f50a5e00640 1 --2- 172.21.15.52:0/2170404992 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f508406ac00 0x7f508406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:07.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.479+0000 7f50a5e00640 1 --2- 172.21.15.52:0/2170404992 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f508406ac00 0x7f508406d0c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f50980055d0 tx=0x7f5098009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:07.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.482+0000 7f509e000640 1 -- 172.21.15.52:0/2170404992 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f50a0075fa0 con 0x7f50a806cb00 2024-03-12T17:08:07.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.681+0000 7f50a7200640 1 -- 172.21.15.52:0/2170404992 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50a804e030 con 0x7f50a806cb00 2024-03-12T17:08:07.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.682+0000 7f509e000640 1 -- 172.21.15.52:0/2170404992 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+12594 (secure 0 0 0) 0x7f50a00795c0 con 0x7f50a806cb00 2024-03-12T17:08:07.677 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:08:07.677 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":36,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:08:03.484301+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T17:06:19.218543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.733031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.775178+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.591522+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.820838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.829252+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.582481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:08:07.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.685+0000 7f50a7200640 1 -- 172.21.15.52:0/2170404992 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f508406ac00 msgr2=0x7f508406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:07.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.685+0000 7f50a7200640 1 --2- 172.21.15.52:0/2170404992 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f508406ac00 0x7f508406d0c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f50980055d0 tx=0x7f5098009040 comp rx=0 tx=0).stop 2024-03-12T17:08:07.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.685+0000 7f50a7200640 1 -- 172.21.15.52:0/2170404992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50a806cb00 msgr2=0x7f50a806cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:07.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.685+0000 7f50a7200640 1 --2- 172.21.15.52:0/2170404992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50a806cb00 0x7f50a806cf80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f50a0000f80 tx=0x7f50a0001da0 comp rx=0 tx=0).stop 2024-03-12T17:08:07.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.686+0000 7f50a7200640 1 -- 172.21.15.52:0/2170404992 shutdown_connections 2024-03-12T17:08:07.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.686+0000 7f50a7200640 1 --2- 172.21.15.52:0/2170404992 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f508406ac00 0x7f508406d0c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:07.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.686+0000 7f50a7200640 1 --2- 172.21.15.52:0/2170404992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50a806cb00 0x7f50a806cf80 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:07.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.686+0000 7f50a7200640 1 --2- 172.21.15.52:0/2170404992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50a814cdd0 0x7f50a80afb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:07.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.686+0000 7f50a7200640 1 --2- 172.21.15.52:0/2170404992 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f508c004d00 0x7f50a814bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:07.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.686+0000 7f50a7200640 1 -- 172.21.15.52:0/2170404992 >> 172.21.15.52:0/2170404992 conn(0x7f50a8065df0 msgr2=0x7f50a80ab8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:07.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.686+0000 7f50a7200640 1 -- 172.21.15.52:0/2170404992 shutdown_connections 2024-03-12T17:08:07.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.686+0000 7f50a7200640 1 -- 172.21.15.52:0/2170404992 wait complete. 2024-03-12T17:08:07.700 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T17:06:19.218543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-12T17:08:07.701 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T17:08:07.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.922+0000 7f37e3000640 1 Processor -- start 2024-03-12T17:08:07.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.922+0000 7f37e3000640 1 -- start start 2024-03-12T17:08:07.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.924+0000 7f37e3000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f37dc159cd0 0x7f37dc0ad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:07.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.924+0000 7f37e3000640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f37dc0cc200 0x7f37dc0cab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:07.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.924+0000 7f37e3000640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37dc0c8c90 con 0x7f37dc0ad7a0 2024-03-12T17:08:07.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.924+0000 7f37e3000640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37dc0cc6c0 con 0x7f37dc0cc200 2024-03-12T17:08:07.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.924+0000 7f37e3000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37dc0cc9a0 con 0x7f37dc159cd0 2024-03-12T17:08:07.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.924+0000 7f37e1200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f37dc0cc200 0x7f37dc0cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:07.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.924+0000 7f37e2600640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f37dc0ad7a0 0x7f37dc15a0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:44428/0 (socket says 172.21.15.52:44428) 2024-03-12T17:08:07.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.924+0000 7f37e2600640 1 -- 172.21.15.52:0/3261110317 learned_addr learned my addr 172.21.15.52:0/3261110317 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:08:07.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.925+0000 7f37e1c00640 1 --2- 172.21.15.52:0/3261110317 >> v2:172.21.15.102:3300/0 conn(0x7f37dc159cd0 0x7f37dc0ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:07.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.925+0000 7f37e1200640 1 -- 172.21.15.52:0/3261110317 >> v2:172.21.15.102:3300/0 conn(0x7f37dc159cd0 msgr2=0x7f37dc0ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:07.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.925+0000 7f37e1200640 1 --2- 172.21.15.52:0/3261110317 >> v2:172.21.15.102:3300/0 conn(0x7f37dc159cd0 0x7f37dc0ad260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:07.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.925+0000 7f37e1200640 1 -- 172.21.15.52:0/3261110317 >> v1:172.21.15.52:6789/0 conn(0x7f37dc0ad7a0 legacy=0x7f37dc15a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:07.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.925+0000 7f37dbe00640 1 -- 172.21.15.52:0/3261110317 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1650827023 0 0) 0x7f37dc0c8c90 con 0x7f37dc0ad7a0 2024-03-12T17:08:07.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.925+0000 7f37e1200640 1 -- 172.21.15.52:0/3261110317 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37dc0ccc80 con 0x7f37dc0cc200 2024-03-12T17:08:07.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.926+0000 7f37e1200640 1 --2- 172.21.15.52:0/3261110317 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37dc0cc200 0x7f37dc0cab30 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f37dc06e4f0 tx=0x7f37c802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e39c0960a557ff server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:07.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.926+0000 7f37dbe00640 1 -- 172.21.15.52:0/3261110317 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c802e070 con 0x7f37dc0cc200 2024-03-12T17:08:07.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.926+0000 7f37dbe00640 1 -- 172.21.15.52:0/3261110317 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c80315a0 con 0x7f37dc0cc200 2024-03-12T17:08:07.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.926+0000 7f37dbe00640 1 -- 172.21.15.52:0/3261110317 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c8031930 con 0x7f37dc0cc200 2024-03-12T17:08:07.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.927+0000 7f37e3000640 1 -- 172.21.15.52:0/3261110317 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37dc0cc200 msgr2=0x7f37dc0cab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:07.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.927+0000 7f37e3000640 1 --2- 172.21.15.52:0/3261110317 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37dc0cc200 0x7f37dc0cab30 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f37dc06e4f0 tx=0x7f37c802b840 comp rx=0 tx=0).stop 2024-03-12T17:08:07.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.927+0000 7f37e3000640 1 -- 172.21.15.52:0/3261110317 shutdown_connections 2024-03-12T17:08:07.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.927+0000 7f37e3000640 1 --2- 172.21.15.52:0/3261110317 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37dc0cc200 0x7f37dc0cab30 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:07.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.927+0000 7f37e3000640 1 --2- 172.21.15.52:0/3261110317 >> v2:172.21.15.102:3300/0 conn(0x7f37dc159cd0 0x7f37dc0ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:07.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.927+0000 7f37e3000640 1 -- 172.21.15.52:0/3261110317 >> 172.21.15.52:0/3261110317 conn(0x7f37dc0b01b0 msgr2=0x7f37dc0b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:07.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.927+0000 7f37e3000640 1 -- 172.21.15.52:0/3261110317 shutdown_connections 2024-03-12T17:08:07.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.927+0000 7f37e3000640 1 -- 172.21.15.52:0/3261110317 wait complete. 2024-03-12T17:08:07.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.929+0000 7f37e3000640 1 Processor -- start 2024-03-12T17:08:07.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.929+0000 7f37e3000640 1 -- start start 2024-03-12T17:08:07.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.930+0000 7f37e3000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f37dc0ad7a0 0x7f37dc15c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:07.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.930+0000 7f37e3000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f37dc0cc200 0x7f37dc0adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:07.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.930+0000 7f37e3000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37dc159cd0 0x7f37dc158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:07.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.930+0000 7f37e3000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37dc1f24a0 con 0x7f37dc159cd0 2024-03-12T17:08:07.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.930+0000 7f37e3000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37dc1f2780 con 0x7f37dc0cc200 2024-03-12T17:08:07.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.930+0000 7f37e3000640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37dc1f2a60 con 0x7f37dc0ad7a0 2024-03-12T17:08:07.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.930+0000 7f37e2600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37dc159cd0 0x7f37dc158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:07.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.930+0000 7f37e2600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37dc159cd0 0x7f37dc158730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40652/0 (socket says 172.21.15.52:40652) 2024-03-12T17:08:07.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.930+0000 7f37e2600640 1 -- 172.21.15.52:0/3205988793 learned_addr learned my addr 172.21.15.52:0/3205988793 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:07.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.931+0000 7f37e1200640 1 --2- 172.21.15.52:0/3205988793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f37dc0cc200 0x7f37dc0adcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:07.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.931+0000 7f37e1c00640 1 --2- 172.21.15.52:0/3205988793 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f37dc0ad7a0 0x7f37dc15c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:07.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.931+0000 7f37e2600640 1 -- 172.21.15.52:0/3205988793 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f37dc0ad7a0 msgr2=0x7f37dc15c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:07.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.931+0000 7f37e2600640 1 --2- 172.21.15.52:0/3205988793 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f37dc0ad7a0 0x7f37dc15c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:07.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.931+0000 7f37e2600640 1 -- 172.21.15.52:0/3205988793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f37dc0cc200 msgr2=0x7f37dc0adcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:07.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.931+0000 7f37e2600640 1 --2- 172.21.15.52:0/3205988793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f37dc0cc200 0x7f37dc0adcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:07.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.931+0000 7f37e2600640 1 -- 172.21.15.52:0/3205988793 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c8009400 con 0x7f37dc159cd0 2024-03-12T17:08:07.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.931+0000 7f37e1200640 1 --2- 172.21.15.52:0/3205988793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f37dc0cc200 0x7f37dc0adcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:08:07.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.931+0000 7f37e2600640 1 --2- 172.21.15.52:0/3205988793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37dc159cd0 0x7f37dc158730 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f37d400c7a0 tx=0x7f37d400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:07.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.931+0000 7f37d9600640 1 -- 172.21.15.52:0/3205988793 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d400a640 con 0x7f37dc159cd0 2024-03-12T17:08:07.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.931+0000 7f37d9600640 1 -- 172.21.15.52:0/3205988793 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d400a7a0 con 0x7f37dc159cd0 2024-03-12T17:08:07.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.931+0000 7f37d9600640 1 -- 172.21.15.52:0/3205988793 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d400abd0 con 0x7f37dc159cd0 2024-03-12T17:08:07.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.931+0000 7f37e3000640 1 -- 172.21.15.52:0/3205988793 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37dc1f2d40 con 0x7f37dc159cd0 2024-03-12T17:08:07.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.932+0000 7f37e3000640 1 -- 172.21.15.52:0/3205988793 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37dc1f2150 con 0x7f37dc159cd0 2024-03-12T17:08:07.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.933+0000 7f37e3000640 1 -- 172.21.15.52:0/3205988793 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37dc0b2ed0 con 0x7f37dc159cd0 2024-03-12T17:08:07.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.934+0000 7f37d9600640 1 -- 172.21.15.52:0/3205988793 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f37d400ad30 con 0x7f37dc159cd0 2024-03-12T17:08:07.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.934+0000 7f37d9600640 1 --2- 172.21.15.52:0/3205988793 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f37bc06ac30 0x7f37bc06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:07.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.935+0000 7f37d9600640 1 -- 172.21.15.52:0/3205988793 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f37d40885d0 con 0x7f37dc159cd0 2024-03-12T17:08:07.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.935+0000 7f37e1c00640 1 --2- 172.21.15.52:0/3205988793 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f37bc06ac30 0x7f37bc06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:07.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.936+0000 7f37e1c00640 1 --2- 172.21.15.52:0/3205988793 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f37bc06ac30 0x7f37bc06d0f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f37d0006fd0 tx=0x7f37d0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:07.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:07.939+0000 7f37d9600640 1 -- 172.21.15.52:0/3205988793 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f37d40539c0 con 0x7f37dc159cd0 2024-03-12T17:08:08.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.141+0000 7f37e3000640 1 -- 172.21.15.52:0/3205988793 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f37dc04e030 con 0x7f37dc159cd0 2024-03-12T17:08:08.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.142+0000 7f37d9600640 1 -- 172.21.15.52:0/3205988793 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f37d40180a0 con 0x7f37dc159cd0 2024-03-12T17:08:08.136 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-03-12T17:08:08.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.144+0000 7f37e3000640 1 -- 172.21.15.52:0/3205988793 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f37bc06ac30 msgr2=0x7f37bc06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:08.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.144+0000 7f37e3000640 1 --2- 172.21.15.52:0/3205988793 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f37bc06ac30 0x7f37bc06d0f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f37d0006fd0 tx=0x7f37d0008040 comp rx=0 tx=0).stop 2024-03-12T17:08:08.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.144+0000 7f37e3000640 1 -- 172.21.15.52:0/3205988793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37dc159cd0 msgr2=0x7f37dc158730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:08.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.144+0000 7f37e3000640 1 --2- 172.21.15.52:0/3205988793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37dc159cd0 0x7f37dc158730 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f37d400c7a0 tx=0x7f37d400cc60 comp rx=0 tx=0).stop 2024-03-12T17:08:08.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.145+0000 7f37e3000640 1 -- 172.21.15.52:0/3205988793 shutdown_connections 2024-03-12T17:08:08.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.145+0000 7f37e3000640 1 --2- 172.21.15.52:0/3205988793 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f37bc06ac30 0x7f37bc06d0f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:08.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.145+0000 7f37e3000640 1 --2- 172.21.15.52:0/3205988793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37dc159cd0 0x7f37dc158730 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:08.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.145+0000 7f37e3000640 1 --2- 172.21.15.52:0/3205988793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f37dc0cc200 0x7f37dc0adcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:08.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.145+0000 7f37e3000640 1 --2- 172.21.15.52:0/3205988793 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f37dc0ad7a0 0x7f37dc15c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:08.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.145+0000 7f37e3000640 1 -- 172.21.15.52:0/3205988793 >> 172.21.15.52:0/3205988793 conn(0x7f37dc0b01b0 msgr2=0x7f37dc069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:08.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.145+0000 7f37e3000640 1 -- 172.21.15.52:0/3205988793 shutdown_connections 2024-03-12T17:08:08.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.145+0000 7f37e3000640 1 -- 172.21.15.52:0/3205988793 wait complete. 2024-03-12T17:08:08.157 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-12T17:08:08.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.380+0000 7f83b5800640 1 Processor -- start 2024-03-12T17:08:08.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.380+0000 7f83b5800640 1 -- start start 2024-03-12T17:08:08.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.382+0000 7f83b5800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f83b00707e0 0x7f83b00b2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:08.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.382+0000 7f83b5800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f83b015cd00 0x7f83b006cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:08.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.382+0000 7f83b5800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83b00b2f70 con 0x7f83b0146070 2024-03-12T17:08:08.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.382+0000 7f83b5800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83b006d130 con 0x7f83b00707e0 2024-03-12T17:08:08.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.382+0000 7f83b5800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b00b3250 con 0x7f83b015cd00 2024-03-12T17:08:08.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.383+0000 7f83a7400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f83b00707e0 0x7f83b00b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:08.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.383+0000 7f83b4e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f83b015cd00 0x7f83b006cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:08.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.383+0000 7f83b4e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f83b015cd00 0x7f83b006cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.52:34758/0 (socket says 172.21.15.52:34758) 2024-03-12T17:08:08.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.383+0000 7f83b4e00640 1 -- 172.21.15.52:0/218777608 learned_addr learned my addr 172.21.15.52:0/218777608 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:08.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.383+0000 7f83a7400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f83b00707e0 0x7f83b00b2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:59352/0 (socket says 172.21.15.52:59352) 2024-03-12T17:08:08.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.384+0000 7f83b4e00640 1 -- 172.21.15.52:0/218777608 >> v2:172.21.15.102:3300/0 conn(0x7f83b00707e0 msgr2=0x7f83b00b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:08.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.384+0000 7f83b4e00640 1 --2- 172.21.15.52:0/218777608 >> v2:172.21.15.102:3300/0 conn(0x7f83b00707e0 0x7f83b00b2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:08.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.384+0000 7f83b4e00640 1 -- 172.21.15.52:0/218777608 >> v1:172.21.15.102:6789/0 conn(0x7f83b0146070 legacy=0x7f83b00700d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:08.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.384+0000 7f83b4e00640 1 -- 172.21.15.52:0/218777608 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b00b3530 con 0x7f83b015cd00 2024-03-12T17:08:08.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.385+0000 7f83b4e00640 1 --2- 172.21.15.52:0/218777608 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f83b015cd00 0x7f83b006cbf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f83a8009e50 tx=0x7f83a802be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98e80b3970d41382 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:08.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.386+0000 7f83af400640 1 -- 172.21.15.52:0/218777608 <== mon.2 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a802c9e0 con 0x7f83b015cd00 2024-03-12T17:08:08.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.386+0000 7f83af400640 1 -- 172.21.15.52:0/218777608 <== mon.2 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a802cb40 con 0x7f83b015cd00 2024-03-12T17:08:08.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.386+0000 7f83af400640 1 -- 172.21.15.52:0/218777608 <== mon.2 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a80343f0 con 0x7f83b015cd00 2024-03-12T17:08:08.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.386+0000 7f83b5800640 1 -- 172.21.15.52:0/218777608 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f83b015cd00 msgr2=0x7f83b006cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:08.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.386+0000 7f83b5800640 1 --2- 172.21.15.52:0/218777608 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f83b015cd00 0x7f83b006cbf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f83a8009e50 tx=0x7f83a802be40 comp rx=0 tx=0).stop 2024-03-12T17:08:08.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.386+0000 7f83b5800640 1 -- 172.21.15.52:0/218777608 shutdown_connections 2024-03-12T17:08:08.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.386+0000 7f83b5800640 1 --2- 172.21.15.52:0/218777608 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f83b015cd00 0x7f83b006cbf0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:08.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.386+0000 7f83b5800640 1 --2- 172.21.15.52:0/218777608 >> v2:172.21.15.102:3300/0 conn(0x7f83b00707e0 0x7f83b00b2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:08.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.386+0000 7f83b5800640 1 -- 172.21.15.52:0/218777608 >> 172.21.15.52:0/218777608 conn(0x7f83b0065df0 msgr2=0x7f83b0069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:08.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.386+0000 7f83b5800640 1 -- 172.21.15.52:0/218777608 shutdown_connections 2024-03-12T17:08:08.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.387+0000 7f83b5800640 1 -- 172.21.15.52:0/218777608 wait complete. 2024-03-12T17:08:08.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.388+0000 7f83b5800640 1 Processor -- start 2024-03-12T17:08:08.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.388+0000 7f83b5800640 1 -- start start 2024-03-12T17:08:08.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.389+0000 7f83b5800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f83b00707e0 0x7f83b00ad6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:08.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.389+0000 7f83b5800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83b0146070 0x7f83b00adbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:08.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.389+0000 7f83b5800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f83b015cd00 0x7f83b013f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:08.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.389+0000 7f83b5800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b00ae120 con 0x7f83b0146070 2024-03-12T17:08:08.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.389+0000 7f83b5800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b00ae3b0 con 0x7f83b015cd00 2024-03-12T17:08:08.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.389+0000 7f83b5800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b00ae690 con 0x7f83b00707e0 2024-03-12T17:08:08.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.390+0000 7f83a7400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83b0146070 0x7f83b00adbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:08.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.390+0000 7f83a7400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83b0146070 0x7f83b00adbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40660/0 (socket says 172.21.15.52:40660) 2024-03-12T17:08:08.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.390+0000 7f83a7400640 1 -- 172.21.15.52:0/485961293 learned_addr learned my addr 172.21.15.52:0/485961293 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:08.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.390+0000 7f83afe00640 1 --2- 172.21.15.52:0/485961293 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f83b00707e0 0x7f83b00ad6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:08.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.390+0000 7f83b4e00640 1 --2- 172.21.15.52:0/485961293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f83b015cd00 0x7f83b013f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:08.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.390+0000 7f83a7400640 1 -- 172.21.15.52:0/485961293 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f83b00707e0 msgr2=0x7f83b00ad6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:08.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.390+0000 7f83a7400640 1 --2- 172.21.15.52:0/485961293 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f83b00707e0 0x7f83b00ad6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:08.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.390+0000 7f83a7400640 1 -- 172.21.15.52:0/485961293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f83b015cd00 msgr2=0x7f83b013f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:08.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.390+0000 7f83a7400640 1 --2- 172.21.15.52:0/485961293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f83b015cd00 0x7f83b013f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:08.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.390+0000 7f83a7400640 1 -- 172.21.15.52:0/485961293 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a8009b30 con 0x7f83b0146070 2024-03-12T17:08:08.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.390+0000 7f83afe00640 1 --2- 172.21.15.52:0/485961293 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f83b00707e0 0x7f83b00ad6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:08:08.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.391+0000 7f83a7400640 1 --2- 172.21.15.52:0/485961293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83b0146070 0x7f83b00adbe0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f839800e9b0 tx=0x7f839800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:08.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.391+0000 7f83acc00640 1 -- 172.21.15.52:0/485961293 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8398009670 con 0x7f83b0146070 2024-03-12T17:08:08.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.391+0000 7f83acc00640 1 -- 172.21.15.52:0/485961293 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839800c9a0 con 0x7f83b0146070 2024-03-12T17:08:08.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.391+0000 7f83acc00640 1 -- 172.21.15.52:0/485961293 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839800cd50 con 0x7f83b0146070 2024-03-12T17:08:08.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.391+0000 7f83b5800640 1 -- 172.21.15.52:0/485961293 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83b00ae970 con 0x7f83b0146070 2024-03-12T17:08:08.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.392+0000 7f83b5800640 1 -- 172.21.15.52:0/485961293 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b00aef30 con 0x7f83b0146070 2024-03-12T17:08:08.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.393+0000 7f83b5800640 1 -- 172.21.15.52:0/485961293 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83b0071b10 con 0x7f83b0146070 2024-03-12T17:08:08.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.394+0000 7f83acc00640 1 -- 172.21.15.52:0/485961293 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8398013470 con 0x7f83b0146070 2024-03-12T17:08:08.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.394+0000 7f83acc00640 1 --2- 172.21.15.52:0/485961293 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f838c06ac50 0x7f838c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:08.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.395+0000 7f83acc00640 1 -- 172.21.15.52:0/485961293 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f83980892d0 con 0x7f83b0146070 2024-03-12T17:08:08.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.395+0000 7f83afe00640 1 --2- 172.21.15.52:0/485961293 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f838c06ac50 0x7f838c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:08.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.396+0000 7f83afe00640 1 --2- 172.21.15.52:0/485961293 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f838c06ac50 0x7f838c06d110 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f83a0000f60 tx=0x7f83a0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:08.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.399+0000 7f83acc00640 1 -- 172.21.15.52:0/485961293 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f8398054830 con 0x7f83b0146070 2024-03-12T17:08:08.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.604+0000 7f83b5800640 1 -- 172.21.15.52:0/485961293 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f83b004e110 con 0x7f83b0146070 2024-03-12T17:08:08.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.605+0000 7f83acc00640 1 -- 172.21.15.52:0/485961293 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v36) v1 ==== 84+0+65 (secure 0 0 0) 0x7f8398057e50 con 0x7f83b0146070 2024-03-12T17:08:08.599 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:08:08.599 INFO:teuthology.orchestra.run.smithi052.stdout:[ 2024-03-12T17:08:08.599 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-12T17:08:08.599 INFO:teuthology.orchestra.run.smithi052.stdout: "poolnum": 1, 2024-03-12T17:08:08.599 INFO:teuthology.orchestra.run.smithi052.stdout: "poolname": ".mgr" 2024-03-12T17:08:08.599 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-12T17:08:08.599 INFO:teuthology.orchestra.run.smithi052.stdout:] 2024-03-12T17:08:08.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.607+0000 7f83b5800640 1 -- 172.21.15.52:0/485961293 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f838c06ac50 msgr2=0x7f838c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:08.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.607+0000 7f83b5800640 1 --2- 172.21.15.52:0/485961293 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f838c06ac50 0x7f838c06d110 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f83a0000f60 tx=0x7f83a0009040 comp rx=0 tx=0).stop 2024-03-12T17:08:08.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.607+0000 7f83b5800640 1 -- 172.21.15.52:0/485961293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83b0146070 msgr2=0x7f83b00adbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:08.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.607+0000 7f83b5800640 1 --2- 172.21.15.52:0/485961293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83b0146070 0x7f83b00adbe0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f839800e9b0 tx=0x7f839800ee70 comp rx=0 tx=0).stop 2024-03-12T17:08:08.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.608+0000 7f83b5800640 1 -- 172.21.15.52:0/485961293 shutdown_connections 2024-03-12T17:08:08.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.608+0000 7f83b5800640 1 --2- 172.21.15.52:0/485961293 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f838c06ac50 0x7f838c06d110 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:08.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.608+0000 7f83b5800640 1 --2- 172.21.15.52:0/485961293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f83b015cd00 0x7f83b013f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:08.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.608+0000 7f83b5800640 1 --2- 172.21.15.52:0/485961293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83b0146070 0x7f83b00adbe0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:08.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.608+0000 7f83b5800640 1 --2- 172.21.15.52:0/485961293 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f83b00707e0 0x7f83b00ad6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:08.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.608+0000 7f83b5800640 1 -- 172.21.15.52:0/485961293 >> 172.21.15.52:0/485961293 conn(0x7f83b0065df0 msgr2=0x7f83b01434d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:08.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.608+0000 7f83b5800640 1 -- 172.21.15.52:0/485961293 shutdown_connections 2024-03-12T17:08:08.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.608+0000 7f83b5800640 1 -- 172.21.15.52:0/485961293 wait complete. 2024-03-12T17:08:08.620 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-12T17:08:08.620 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-12T17:08:08.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.846+0000 7f68f7e00640 1 Processor -- start 2024-03-12T17:08:08.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.846+0000 7f68f7e00640 1 -- start start 2024-03-12T17:08:08.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.848+0000 7f68f7e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f68f80b22b0 0x7f68f815eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:08.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.848+0000 7f68f7e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f68f815f020 0x7f68f80cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:08.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.848+0000 7f68f7e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68f815e320 con 0x7f68f815a850 2024-03-12T17:08:08.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.848+0000 7f68f7e00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68f815ac30 con 0x7f68f80b22b0 2024-03-12T17:08:08.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.848+0000 7f68f7e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68f80cda20 con 0x7f68f815f020 2024-03-12T17:08:08.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.848+0000 7f68efe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f68f80b22b0 0x7f68f815eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:08.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.848+0000 7f68efe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f68f80b22b0 0x7f68f815eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40672/0 (socket says 172.21.15.52:40672) 2024-03-12T17:08:08.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.848+0000 7f68efe00640 1 -- 172.21.15.52:0/4191978406 learned_addr learned my addr 172.21.15.52:0/4191978406 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:08.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.848+0000 7f68f7400640 1 --2- 172.21.15.52:0/4191978406 >> v2:172.21.15.102:3300/0 conn(0x7f68f815f020 0x7f68f80cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:08.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.849+0000 7f68efe00640 1 -- 172.21.15.52:0/4191978406 >> v2:172.21.15.102:3300/0 conn(0x7f68f815f020 msgr2=0x7f68f80cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:08.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.849+0000 7f68efe00640 1 --2- 172.21.15.52:0/4191978406 >> v2:172.21.15.102:3300/0 conn(0x7f68f815f020 0x7f68f80cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:08.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.849+0000 7f68efe00640 1 -- 172.21.15.52:0/4191978406 >> v1:172.21.15.102:6789/0 conn(0x7f68f815a850 legacy=0x7f68f80cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:08.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.849+0000 7f68efe00640 1 -- 172.21.15.52:0/4191978406 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f80cdd00 con 0x7f68f80b22b0 2024-03-12T17:08:08.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.850+0000 7f68efe00640 1 --2- 172.21.15.52:0/4191978406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68f80b22b0 0x7f68f815eae0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f68f80addf0 tx=0x7f68e002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc31af26f300f837 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:08.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.850+0000 7f68f6000640 1 -- 172.21.15.52:0/4191978406 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e0031870 con 0x7f68f80b22b0 2024-03-12T17:08:08.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.850+0000 7f68f6000640 1 -- 172.21.15.52:0/4191978406 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e00319d0 con 0x7f68f80b22b0 2024-03-12T17:08:08.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.850+0000 7f68f6000640 1 -- 172.21.15.52:0/4191978406 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e0031d60 con 0x7f68f80b22b0 2024-03-12T17:08:08.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.851+0000 7f68f7e00640 1 -- 172.21.15.52:0/4191978406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68f80b22b0 msgr2=0x7f68f815eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:08.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.851+0000 7f68f7e00640 1 --2- 172.21.15.52:0/4191978406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68f80b22b0 0x7f68f815eae0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f68f80addf0 tx=0x7f68e002b910 comp rx=0 tx=0).stop 2024-03-12T17:08:08.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.851+0000 7f68f7e00640 1 -- 172.21.15.52:0/4191978406 shutdown_connections 2024-03-12T17:08:08.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.851+0000 7f68f7e00640 1 --2- 172.21.15.52:0/4191978406 >> v2:172.21.15.102:3300/0 conn(0x7f68f815f020 0x7f68f80cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:08.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.851+0000 7f68f7e00640 1 --2- 172.21.15.52:0/4191978406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68f80b22b0 0x7f68f815eae0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:08.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.851+0000 7f68f7e00640 1 -- 172.21.15.52:0/4191978406 >> 172.21.15.52:0/4191978406 conn(0x7f68f80b01b0 msgr2=0x7f68f80b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:08.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.851+0000 7f68f7e00640 1 -- 172.21.15.52:0/4191978406 shutdown_connections 2024-03-12T17:08:08.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.851+0000 7f68f7e00640 1 -- 172.21.15.52:0/4191978406 wait complete. 2024-03-12T17:08:08.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.853+0000 7f68f7e00640 1 Processor -- start 2024-03-12T17:08:08.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.853+0000 7f68f7e00640 1 -- start start 2024-03-12T17:08:08.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.854+0000 7f68f7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68f80b22b0 0x7f68f8149940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:08.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.854+0000 7f68f7e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f68f815a850 0x7f68f8070c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:08.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.854+0000 7f68f7e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68f815f020 0x7f68f80c9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:08.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.854+0000 7f68f7e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f806fee0 con 0x7f68f815f020 2024-03-12T17:08:08.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.854+0000 7f68f7e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f81f3110 con 0x7f68f80b22b0 2024-03-12T17:08:08.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.854+0000 7f68f7e00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f80701a0 con 0x7f68f815a850 2024-03-12T17:08:08.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.855+0000 7f68f7400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68f815f020 0x7f68f80c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:08.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.855+0000 7f68f7400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68f815f020 0x7f68f80c9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40688/0 (socket says 172.21.15.52:40688) 2024-03-12T17:08:08.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.855+0000 7f68f7400640 1 -- 172.21.15.52:0/3792825447 learned_addr learned my addr 172.21.15.52:0/3792825447 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:08.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.855+0000 7f68efe00640 1 --2- 172.21.15.52:0/3792825447 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f68f815a850 0x7f68f8070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:08.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.855+0000 7f68f6a00640 1 --2- 172.21.15.52:0/3792825447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68f80b22b0 0x7f68f8149940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:08.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.855+0000 7f68f7400640 1 -- 172.21.15.52:0/3792825447 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f68f815a850 msgr2=0x7f68f8070c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:08.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.855+0000 7f68f7400640 1 --2- 172.21.15.52:0/3792825447 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f68f815a850 0x7f68f8070c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:08.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.855+0000 7f68f7400640 1 -- 172.21.15.52:0/3792825447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68f80b22b0 msgr2=0x7f68f8149940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:08.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.855+0000 7f68f7400640 1 --2- 172.21.15.52:0/3792825447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68f80b22b0 0x7f68f8149940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:08.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.855+0000 7f68f7400640 1 -- 172.21.15.52:0/3792825447 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e00094d0 con 0x7f68f815f020 2024-03-12T17:08:08.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.855+0000 7f68f6a00640 1 --2- 172.21.15.52:0/3792825447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68f80b22b0 0x7f68f8149940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:08:08.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.855+0000 7f68f7400640 1 --2- 172.21.15.52:0/3792825447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68f815f020 0x7f68f80c9980 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f68f0007e60 tx=0x7f68f000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:08.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.856+0000 7f68eea00640 1 -- 172.21.15.52:0/3792825447 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f0014420 con 0x7f68f815f020 2024-03-12T17:08:08.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.856+0000 7f68eea00640 1 -- 172.21.15.52:0/3792825447 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f00145a0 con 0x7f68f815f020 2024-03-12T17:08:08.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.856+0000 7f68eea00640 1 -- 172.21.15.52:0/3792825447 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f0014950 con 0x7f68f815f020 2024-03-12T17:08:08.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.856+0000 7f68f7e00640 1 -- 172.21.15.52:0/3792825447 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68f8070480 con 0x7f68f815f020 2024-03-12T17:08:08.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.856+0000 7f68f7e00640 1 -- 172.21.15.52:0/3792825447 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f80c9ec0 con 0x7f68f815f020 2024-03-12T17:08:08.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.857+0000 7f68f7e00640 1 -- 172.21.15.52:0/3792825447 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68f80709a0 con 0x7f68f815f020 2024-03-12T17:08:08.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.858+0000 7f68eea00640 1 -- 172.21.15.52:0/3792825447 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f68f0014ab0 con 0x7f68f815f020 2024-03-12T17:08:08.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.859+0000 7f68eea00640 1 --2- 172.21.15.52:0/3792825447 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f68d406ac00 0x7f68d406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:08.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.859+0000 7f68eea00640 1 -- 172.21.15.52:0/3792825447 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f68f0089af0 con 0x7f68f815f020 2024-03-12T17:08:08.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.860+0000 7f68f6a00640 1 --2- 172.21.15.52:0/3792825447 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f68d406ac00 0x7f68d406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:08.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.861+0000 7f68f6a00640 1 --2- 172.21.15.52:0/3792825447 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f68d406ac00 0x7f68d406d0c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f68f80c8720 tx=0x7f68e8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:08.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:08.863+0000 7f68eea00640 1 -- 172.21.15.52:0/3792825447 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f68f80709a0 con 0x7f68f815f020 2024-03-12T17:08:09.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.060+0000 7f68f7e00640 1 -- 172.21.15.52:0/3792825447 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f68f80709a0 con 0x7f68f815f020 2024-03-12T17:08:09.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.173+0000 7f68eea00640 1 -- 172.21.15.52:0/3792825447 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v37) v1 ==== 138+0+0 (secure 0 0 0) 0x7f68f80709a0 con 0x7f68f815f020 2024-03-12T17:08:09.168 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_metadata' created 2024-03-12T17:08:09.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.178+0000 7f68f7e00640 1 -- 172.21.15.52:0/3792825447 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f68d406ac00 msgr2=0x7f68d406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:09.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.178+0000 7f68f7e00640 1 --2- 172.21.15.52:0/3792825447 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f68d406ac00 0x7f68d406d0c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f68f80c8720 tx=0x7f68e8009040 comp rx=0 tx=0).stop 2024-03-12T17:08:09.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.178+0000 7f68f7e00640 1 -- 172.21.15.52:0/3792825447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68f815f020 msgr2=0x7f68f80c9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:09.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.178+0000 7f68f7e00640 1 --2- 172.21.15.52:0/3792825447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68f815f020 0x7f68f80c9980 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f68f0007e60 tx=0x7f68f000e4b0 comp rx=0 tx=0).stop 2024-03-12T17:08:09.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.178+0000 7f68f7e00640 1 -- 172.21.15.52:0/3792825447 shutdown_connections 2024-03-12T17:08:09.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.178+0000 7f68f7e00640 1 --2- 172.21.15.52:0/3792825447 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f68d406ac00 0x7f68d406d0c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:09.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.178+0000 7f68f7e00640 1 --2- 172.21.15.52:0/3792825447 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f68f815a850 0x7f68f8070c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:09.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.178+0000 7f68f7e00640 1 --2- 172.21.15.52:0/3792825447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68f815f020 0x7f68f80c9980 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:09.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.178+0000 7f68f7e00640 1 --2- 172.21.15.52:0/3792825447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68f80b22b0 0x7f68f8149940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:09.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.178+0000 7f68f7e00640 1 -- 172.21.15.52:0/3792825447 >> 172.21.15.52:0/3792825447 conn(0x7f68f80b01b0 msgr2=0x7f68f806d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:09.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.178+0000 7f68f7e00640 1 -- 172.21.15.52:0/3792825447 shutdown_connections 2024-03-12T17:08:09.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.178+0000 7f68f7e00640 1 -- 172.21.15.52:0/3792825447 wait complete. 2024-03-12T17:08:09.198 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-12T17:08:09.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.418+0000 7f2153200640 1 Processor -- start 2024-03-12T17:08:09.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.418+0000 7f2153200640 1 -- start start 2024-03-12T17:08:09.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.419+0000 7f2153200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f215415cab0 0x7f21540b8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:09.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.419+0000 7f2153200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f21540d1c30 0x7f21540cf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:09.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.420+0000 7f2153200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21540cf850 con 0x7f21540ceeb0 2024-03-12T17:08:09.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.420+0000 7f2153200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f215415ce90 con 0x7f21540d1c30 2024-03-12T17:08:09.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.420+0000 7f2153200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21540cddf0 con 0x7f215415cab0 2024-03-12T17:08:09.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.420+0000 7f2152800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f21540d1c30 0x7f21540cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:09.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.420+0000 7f2151e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f215415cab0 0x7f21540b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:09.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.420+0000 7f2152800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f21540d1c30 0x7f21540cf310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:59394/0 (socket says 172.21.15.52:59394) 2024-03-12T17:08:09.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.420+0000 7f2152800640 1 -- 172.21.15.52:0/3277272295 learned_addr learned my addr 172.21.15.52:0/3277272295 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:09.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.420+0000 7f2151400640 1 --1- 172.21.15.52:0/3277272295 >> v1:172.21.15.102:6789/0 conn(0x7f21540ceeb0 0x7f21540b4120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.52:52290/0 (socket says 172.21.15.52:52290) 2024-03-12T17:08:09.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.421+0000 7f2152800640 1 -- 172.21.15.52:0/3277272295 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f215415cab0 msgr2=0x7f21540b8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:09.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.421+0000 7f2152800640 1 --2- 172.21.15.52:0/3277272295 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f215415cab0 0x7f21540b8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:09.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.421+0000 7f2152800640 1 -- 172.21.15.52:0/3277272295 >> v1:172.21.15.102:6789/0 conn(0x7f21540ceeb0 legacy=0x7f21540b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:09.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.421+0000 7f2152800640 1 -- 172.21.15.52:0/3277272295 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21540cd570 con 0x7f21540d1c30 2024-03-12T17:08:09.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.421+0000 7f2151e00640 1 --2- 172.21.15.52:0/3277272295 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f215415cab0 0x7f21540b8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T17:08:09.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.423+0000 7f2152800640 1 --2- 172.21.15.52:0/3277272295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21540d1c30 0x7f21540cf310 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f214c009e50 tx=0x7f214c02bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a57e717b0aa17580 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:09.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.423+0000 7f2150a00640 1 -- 172.21.15.52:0/3277272295 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c02cb00 con 0x7f21540d1c30 2024-03-12T17:08:09.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.423+0000 7f2150a00640 1 -- 172.21.15.52:0/3277272295 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214c02cc60 con 0x7f21540d1c30 2024-03-12T17:08:09.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.423+0000 7f2150a00640 1 -- 172.21.15.52:0/3277272295 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c0343f0 con 0x7f21540d1c30 2024-03-12T17:08:09.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.424+0000 7f2153200640 1 -- 172.21.15.52:0/3277272295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21540d1c30 msgr2=0x7f21540cf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:09.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.424+0000 7f2153200640 1 --2- 172.21.15.52:0/3277272295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21540d1c30 0x7f21540cf310 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f214c009e50 tx=0x7f214c02bf60 comp rx=0 tx=0).stop 2024-03-12T17:08:09.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.424+0000 7f2153200640 1 -- 172.21.15.52:0/3277272295 shutdown_connections 2024-03-12T17:08:09.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.424+0000 7f2153200640 1 --2- 172.21.15.52:0/3277272295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21540d1c30 0x7f21540cf310 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:09.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.424+0000 7f2153200640 1 --2- 172.21.15.52:0/3277272295 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f215415cab0 0x7f21540b8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:09.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.424+0000 7f2153200640 1 -- 172.21.15.52:0/3277272295 >> 172.21.15.52:0/3277272295 conn(0x7f215406ae90 msgr2=0x7f215406b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:09.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.424+0000 7f2153200640 1 -- 172.21.15.52:0/3277272295 shutdown_connections 2024-03-12T17:08:09.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.424+0000 7f2153200640 1 -- 172.21.15.52:0/3277272295 wait complete. 2024-03-12T17:08:09.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.426+0000 7f2153200640 1 Processor -- start 2024-03-12T17:08:09.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.427+0000 7f2153200640 1 -- start start 2024-03-12T17:08:09.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.428+0000 7f2153200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f21540ceeb0 0x7f215414aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:09.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.428+0000 7f2153200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21540d1c30 0x7f21540caa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:09.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.428+0000 7f2153200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f215415cab0 0x7f21540cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:09.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.428+0000 7f2153200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21540c9770 con 0x7f21540d1c30 2024-03-12T17:08:09.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.428+0000 7f2153200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21540ca5d0 con 0x7f215415cab0 2024-03-12T17:08:09.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.428+0000 7f2153200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f215414af90 con 0x7f21540ceeb0 2024-03-12T17:08:09.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.428+0000 7f2151400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21540d1c30 0x7f21540caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:09.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.428+0000 7f2151400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21540d1c30 0x7f21540caa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40700/0 (socket says 172.21.15.52:40700) 2024-03-12T17:08:09.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.428+0000 7f2151400640 1 -- 172.21.15.52:0/2255397182 learned_addr learned my addr 172.21.15.52:0/2255397182 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:09.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.429+0000 7f2151e00640 1 --2- 172.21.15.52:0/2255397182 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f21540ceeb0 0x7f215414aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:09.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.429+0000 7f2152800640 1 --2- 172.21.15.52:0/2255397182 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f215415cab0 0x7f21540cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:09.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.429+0000 7f2151400640 1 -- 172.21.15.52:0/2255397182 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f21540ceeb0 msgr2=0x7f215414aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:09.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.429+0000 7f2151400640 1 --2- 172.21.15.52:0/2255397182 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f21540ceeb0 0x7f215414aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:09.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.429+0000 7f2151400640 1 -- 172.21.15.52:0/2255397182 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f215415cab0 msgr2=0x7f21540cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:09.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.429+0000 7f2151400640 1 --2- 172.21.15.52:0/2255397182 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f215415cab0 0x7f21540cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:09.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.429+0000 7f2151400640 1 -- 172.21.15.52:0/2255397182 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2144009400 con 0x7f21540d1c30 2024-03-12T17:08:09.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.429+0000 7f2151400640 1 --2- 172.21.15.52:0/2255397182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21540d1c30 0x7f21540caa60 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f213c00c500 tx=0x7f213c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:09.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.429+0000 7f214a000640 1 -- 172.21.15.52:0/2255397182 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c007500 con 0x7f21540d1c30 2024-03-12T17:08:09.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.430+0000 7f214a000640 1 -- 172.21.15.52:0/2255397182 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213c007660 con 0x7f21540d1c30 2024-03-12T17:08:09.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.430+0000 7f2153200640 1 -- 172.21.15.52:0/2255397182 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f214c009b30 con 0x7f21540d1c30 2024-03-12T17:08:09.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.430+0000 7f214a000640 1 -- 172.21.15.52:0/2255397182 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c0049e0 con 0x7f21540d1c30 2024-03-12T17:08:09.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.430+0000 7f2153200640 1 -- 172.21.15.52:0/2255397182 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21540c9ca0 con 0x7f21540d1c30 2024-03-12T17:08:09.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.431+0000 7f2153200640 1 -- 172.21.15.52:0/2255397182 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21540cd020 con 0x7f21540d1c30 2024-03-12T17:08:09.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.432+0000 7f214a000640 1 -- 172.21.15.52:0/2255397182 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f213c004c00 con 0x7f21540d1c30 2024-03-12T17:08:09.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.433+0000 7f214a000640 1 --2- 172.21.15.52:0/2255397182 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f213006ac00 0x7f213006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:09.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.433+0000 7f214a000640 1 -- 172.21.15.52:0/2255397182 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f213c0888a0 con 0x7f21540d1c30 2024-03-12T17:08:09.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.433+0000 7f2151e00640 1 --2- 172.21.15.52:0/2255397182 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f213006ac00 0x7f213006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:09.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.435+0000 7f2151e00640 1 --2- 172.21.15.52:0/2255397182 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f213006ac00 0x7f213006d0c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2144009bc0 tx=0x7f2144008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:09.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.438+0000 7f214a000640 1 -- 172.21.15.52:0/2255397182 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f213c053b10 con 0x7f21540d1c30 2024-03-12T17:08:09.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:09.636+0000 7f2153200640 1 -- 172.21.15.52:0/2255397182 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f215404e030 con 0x7f21540d1c30 2024-03-12T17:08:10.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.185+0000 7f214a000640 1 -- 172.21.15.52:0/2255397182 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v38) v1 ==== 170+0+0 (secure 0 0 0) 0x7f213c0180d0 con 0x7f21540d1c30 2024-03-12T17:08:10.182 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_data' created 2024-03-12T17:08:10.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.192+0000 7f2153200640 1 -- 172.21.15.52:0/2255397182 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f213006ac00 msgr2=0x7f213006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:10.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.192+0000 7f2153200640 1 --2- 172.21.15.52:0/2255397182 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f213006ac00 0x7f213006d0c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2144009bc0 tx=0x7f2144008040 comp rx=0 tx=0).stop 2024-03-12T17:08:10.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.192+0000 7f2153200640 1 -- 172.21.15.52:0/2255397182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21540d1c30 msgr2=0x7f21540caa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:10.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.192+0000 7f2153200640 1 --2- 172.21.15.52:0/2255397182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21540d1c30 0x7f21540caa60 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f213c00c500 tx=0x7f213c00c9c0 comp rx=0 tx=0).stop 2024-03-12T17:08:10.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.193+0000 7f2153200640 1 -- 172.21.15.52:0/2255397182 shutdown_connections 2024-03-12T17:08:10.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.193+0000 7f2153200640 1 --2- 172.21.15.52:0/2255397182 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f213006ac00 0x7f213006d0c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:10.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.193+0000 7f2153200640 1 --2- 172.21.15.52:0/2255397182 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f215415cab0 0x7f21540cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:10.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.193+0000 7f2153200640 1 --2- 172.21.15.52:0/2255397182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21540d1c30 0x7f21540caa60 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:10.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.193+0000 7f2153200640 1 --2- 172.21.15.52:0/2255397182 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f21540ceeb0 0x7f215414aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:10.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.193+0000 7f2153200640 1 -- 172.21.15.52:0/2255397182 >> 172.21.15.52:0/2255397182 conn(0x7f215406ae90 msgr2=0x7f215406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:10.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.194+0000 7f2153200640 1 -- 172.21.15.52:0/2255397182 shutdown_connections 2024-03-12T17:08:10.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.194+0000 7f2153200640 1 -- 172.21.15.52:0/2255397182 wait complete. 2024-03-12T17:08:10.222 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-12T17:08:10.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.415+0000 7f6172400640 1 Processor -- start 2024-03-12T17:08:10.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.416+0000 7f6172400640 1 -- start start 2024-03-12T17:08:10.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.417+0000 7f6172400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f616c0cd740 0x7f616c0b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:10.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.417+0000 7f6172400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f616c0b5aa0 0x7f616c0b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:10.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.417+0000 7f6172400640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f616c0d20c0 con 0x7f616c0d0d40 2024-03-12T17:08:10.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.417+0000 7f6172400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f616c0d14b0 con 0x7f616c0cd740 2024-03-12T17:08:10.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.417+0000 7f6172400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f616c0d1790 con 0x7f616c0b5aa0 2024-03-12T17:08:10.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.418+0000 7f6162a00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f616c0d0d40 0x7f616c0cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:44438/0 (socket says 172.21.15.52:44438) 2024-03-12T17:08:10.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.418+0000 7f6162a00640 1 -- 172.21.15.52:0/3252359281 learned_addr learned my addr 172.21.15.52:0/3252359281 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:08:10.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.418+0000 7f616be00640 1 --2- 172.21.15.52:0/3252359281 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f616c0b5aa0 0x7f616c0b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:10.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.418+0000 7f616aa00640 1 -- 172.21.15.52:0/3252359281 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1278863307 0 0) 0x7f616c0d20c0 con 0x7f616c0d0d40 2024-03-12T17:08:10.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.418+0000 7f616b400640 1 --2- 172.21.15.52:0/3252359281 >> v2:172.21.15.102:3300/0 conn(0x7f616c0cd740 0x7f616c0b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:10.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.418+0000 7f616aa00640 1 -- 172.21.15.52:0/3252359281 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6150003520 con 0x7f616c0d0d40 2024-03-12T17:08:10.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.419+0000 7f616aa00640 1 -- 172.21.15.52:0/3252359281 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1049602791 0 0) 0x7f6150003520 con 0x7f616c0d0d40 2024-03-12T17:08:10.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.419+0000 7f616aa00640 1 -- 172.21.15.52:0/3252359281 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f616c0b5aa0 msgr2=0x7f616c0b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:10.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.419+0000 7f616aa00640 1 --2- 172.21.15.52:0/3252359281 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f616c0b5aa0 0x7f616c0b5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:10.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.419+0000 7f616aa00640 1 -- 172.21.15.52:0/3252359281 >> v2:172.21.15.102:3300/0 conn(0x7f616c0cd740 msgr2=0x7f616c0b84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:10.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.419+0000 7f616aa00640 1 --2- 172.21.15.52:0/3252359281 >> v2:172.21.15.102:3300/0 conn(0x7f616c0cd740 0x7f616c0b84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:10.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.419+0000 7f616aa00640 1 -- 172.21.15.52:0/3252359281 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f616c0d1a70 con 0x7f616c0d0d40 2024-03-12T17:08:10.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.419+0000 7f616aa00640 1 -- 172.21.15.52:0/3252359281 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f61540019f0 con 0x7f616c0d0d40 2024-03-12T17:08:10.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.419+0000 7f616aa00640 1 -- 172.21.15.52:0/3252359281 >> v1:172.21.15.52:6789/0 conn(0x7f616c0d0d40 legacy=0x7f616c0cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:10.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.419+0000 7f616aa00640 1 --2- 172.21.15.52:0/3252359281 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6150003970 0x7f6150003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:10.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.419+0000 7f616aa00640 1 --2- 172.21.15.52:0/3252359281 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61500047c0 0x7f6150004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:10.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.419+0000 7f616aa00640 1 --2- 172.21.15.52:0/3252359281 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f616c0b5aa0 0x7f6150005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:10.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.419+0000 7f616aa00640 1 -- 172.21.15.52:0/3252359281 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616c0d14b0 con 0x7f61500047c0 2024-03-12T17:08:10.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.419+0000 7f616aa00640 1 -- 172.21.15.52:0/3252359281 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616c0d1790 con 0x7f6150003970 2024-03-12T17:08:10.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.419+0000 7f616aa00640 1 -- 172.21.15.52:0/3252359281 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f615000b980 con 0x7f616c0b5aa0 2024-03-12T17:08:10.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.419+0000 7f6162a00640 1 --2- 172.21.15.52:0/3252359281 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61500047c0 0x7f6150004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:10.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.420+0000 7f616b400640 1 --2- 172.21.15.52:0/3252359281 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6150003970 0x7f6150003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:10.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.420+0000 7f6162a00640 1 -- 172.21.15.52:0/3252359281 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f616c0b5aa0 msgr2=0x7f6150005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:10.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.420+0000 7f6162a00640 1 --2- 172.21.15.52:0/3252359281 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f616c0b5aa0 0x7f6150005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:10.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.420+0000 7f6162a00640 1 -- 172.21.15.52:0/3252359281 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6150003970 msgr2=0x7f6150003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:10.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.420+0000 7f6162a00640 1 --2- 172.21.15.52:0/3252359281 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6150003970 0x7f6150003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:10.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.420+0000 7f6162a00640 1 -- 172.21.15.52:0/3252359281 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6150003520 con 0x7f61500047c0 2024-03-12T17:08:10.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.421+0000 7f6162a00640 1 --2- 172.21.15.52:0/3252359281 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61500047c0 0x7f6150004c40 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f615400a860 tx=0x7f61500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce9a66cdbb404450 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:10.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.421+0000 7f616aa00640 1 -- 172.21.15.52:0/3252359281 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6154034630 con 0x7f61500047c0 2024-03-12T17:08:10.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.421+0000 7f616aa00640 1 -- 172.21.15.52:0/3252359281 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615402f070 con 0x7f61500047c0 2024-03-12T17:08:10.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.421+0000 7f616aa00640 1 -- 172.21.15.52:0/3252359281 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61540349b0 con 0x7f61500047c0 2024-03-12T17:08:10.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.422+0000 7f6172400640 1 -- 172.21.15.52:0/3252359281 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61500047c0 msgr2=0x7f6150004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:10.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.422+0000 7f6172400640 1 --2- 172.21.15.52:0/3252359281 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61500047c0 0x7f6150004c40 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f615400a860 tx=0x7f61500057f0 comp rx=0 tx=0).stop 2024-03-12T17:08:10.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.422+0000 7f616be00640 1 -- 172.21.15.52:0/3252359281 reap_dead start 2024-03-12T17:08:10.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.422+0000 7f6172400640 1 -- 172.21.15.52:0/3252359281 shutdown_connections 2024-03-12T17:08:10.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.422+0000 7f6172400640 1 -- 172.21.15.52:0/3252359281 >> 172.21.15.52:0/3252359281 conn(0x7f616c0cf5d0 msgr2=0x7f616c0cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:10.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.422+0000 7f6172400640 1 -- 172.21.15.52:0/3252359281 shutdown_connections 2024-03-12T17:08:10.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.422+0000 7f6172400640 1 -- 172.21.15.52:0/3252359281 wait complete. 2024-03-12T17:08:10.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.424+0000 7f6172400640 1 Processor -- start 2024-03-12T17:08:10.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.424+0000 7f6172400640 1 -- start start 2024-03-12T17:08:10.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.425+0000 7f6172400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61500047c0 0x7f616c0cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:10.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.425+0000 7f6172400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f616c14fee0 0x7f616c1f54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:10.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.425+0000 7f6172400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f616c1f5a20 0x7f616c14c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:10.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.425+0000 7f6172400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616c153670 con 0x7f61500047c0 2024-03-12T17:08:10.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.425+0000 7f6172400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616c0ae560 con 0x7f616c1f5a20 2024-03-12T17:08:10.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.425+0000 7f6172400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f616c14fa50 con 0x7f616c14fee0 2024-03-12T17:08:10.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.425+0000 7f616b400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61500047c0 0x7f616c0cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:10.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.425+0000 7f616b400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61500047c0 0x7f616c0cbe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40728/0 (socket says 172.21.15.52:40728) 2024-03-12T17:08:10.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.425+0000 7f616b400640 1 -- 172.21.15.52:0/2843969416 learned_addr learned my addr 172.21.15.52:0/2843969416 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:10.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.426+0000 7f616be00640 1 --2- 172.21.15.52:0/2843969416 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f616c1f5a20 0x7f616c14c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:10.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.426+0000 7f616b400640 1 -- 172.21.15.52:0/2843969416 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f616c14fee0 msgr2=0x7f616c1f54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:10.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.426+0000 7f6162a00640 1 --2- 172.21.15.52:0/2843969416 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f616c14fee0 0x7f616c1f54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:10.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.426+0000 7f616b400640 1 --2- 172.21.15.52:0/2843969416 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f616c14fee0 0x7f616c1f54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:10.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.426+0000 7f616b400640 1 -- 172.21.15.52:0/2843969416 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f616c1f5a20 msgr2=0x7f616c14c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:10.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.426+0000 7f616b400640 1 --2- 172.21.15.52:0/2843969416 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f616c1f5a20 0x7f616c14c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:10.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.426+0000 7f616b400640 1 -- 172.21.15.52:0/2843969416 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f616c1f6410 con 0x7f61500047c0 2024-03-12T17:08:10.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.426+0000 7f616be00640 1 --2- 172.21.15.52:0/2843969416 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f616c1f5a20 0x7f616c14c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:08:10.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.426+0000 7f616b400640 1 --2- 172.21.15.52:0/2843969416 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61500047c0 0x7f616c0cbe80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f615c011660 tx=0x7f615c011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:10.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.426+0000 7f6163e00640 1 -- 172.21.15.52:0/2843969416 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c00f960 con 0x7f61500047c0 2024-03-12T17:08:10.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.427+0000 7f6163e00640 1 -- 172.21.15.52:0/2843969416 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615c00fac0 con 0x7f61500047c0 2024-03-12T17:08:10.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.427+0000 7f6163e00640 1 -- 172.21.15.52:0/2843969416 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c00fe30 con 0x7f61500047c0 2024-03-12T17:08:10.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.427+0000 7f6172400640 1 -- 172.21.15.52:0/2843969416 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f616c1f5e60 con 0x7f61500047c0 2024-03-12T17:08:10.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.427+0000 7f6172400640 1 -- 172.21.15.52:0/2843969416 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f616c14f400 con 0x7f61500047c0 2024-03-12T17:08:10.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.428+0000 7f6172400640 1 -- 172.21.15.52:0/2843969416 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f616c073390 con 0x7f61500047c0 2024-03-12T17:08:10.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.429+0000 7f6163e00640 1 -- 172.21.15.52:0/2843969416 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f615c015460 con 0x7f61500047c0 2024-03-12T17:08:10.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.430+0000 7f6163e00640 1 --2- 172.21.15.52:0/2843969416 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f614806ac00 0x7f614806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:10.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.430+0000 7f6163e00640 1 -- 172.21.15.52:0/2843969416 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f615c089850 con 0x7f61500047c0 2024-03-12T17:08:10.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.431+0000 7f6162a00640 1 --2- 172.21.15.52:0/2843969416 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f614806ac00 0x7f614806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:10.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.432+0000 7f6162a00640 1 --2- 172.21.15.52:0/2843969416 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f614806ac00 0x7f614806d0c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6154034000 tx=0x7f6154009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:10.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.436+0000 7f6163e00640 1 -- 172.21.15.52:0/2843969416 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f615c0549e0 con 0x7f61500047c0 2024-03-12T17:08:10.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.676+0000 7f6172400640 1 -- 172.21.15.52:0/2843969416 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f616c0b20b0 con 0x7f61500047c0 2024-03-12T17:08:10.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.688+0000 7f6163e00640 1 -- 172.21.15.52:0/2843969416 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f615c0220c0 con 0x7f61500047c0 2024-03-12T17:08:10.682 INFO:teuthology.orchestra.run.smithi052.stderr:new fs with metadata pool 5 and data pool 6 2024-03-12T17:08:10.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.693+0000 7f6172400640 1 -- 172.21.15.52:0/2843969416 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f614806ac00 msgr2=0x7f614806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:10.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.693+0000 7f6172400640 1 --2- 172.21.15.52:0/2843969416 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f614806ac00 0x7f614806d0c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6154034000 tx=0x7f6154009030 comp rx=0 tx=0).stop 2024-03-12T17:08:10.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.693+0000 7f6172400640 1 -- 172.21.15.52:0/2843969416 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61500047c0 msgr2=0x7f616c0cbe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:10.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.693+0000 7f6172400640 1 --2- 172.21.15.52:0/2843969416 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61500047c0 0x7f616c0cbe80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f615c011660 tx=0x7f615c011b20 comp rx=0 tx=0).stop 2024-03-12T17:08:10.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.693+0000 7f6172400640 1 -- 172.21.15.52:0/2843969416 shutdown_connections 2024-03-12T17:08:10.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.693+0000 7f6172400640 1 --2- 172.21.15.52:0/2843969416 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f614806ac00 0x7f614806d0c0 secure :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6154034000 tx=0x7f6154009030 comp rx=0 tx=0).stop 2024-03-12T17:08:10.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.693+0000 7f6172400640 1 --2- 172.21.15.52:0/2843969416 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f616c14fee0 0x7f616c1f54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:10.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.693+0000 7f6172400640 1 --2- 172.21.15.52:0/2843969416 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f616c1f5a20 0x7f616c14c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:10.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.693+0000 7f6172400640 1 --2- 172.21.15.52:0/2843969416 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61500047c0 0x7f616c0cbe80 secure :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f615c011660 tx=0x7f615c011b20 comp rx=0 tx=0).stop 2024-03-12T17:08:10.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.693+0000 7f6172400640 1 -- 172.21.15.52:0/2843969416 >> 172.21.15.52:0/2843969416 conn(0x7f616c0cf5d0 msgr2=0x7f616c0b77c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:10.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.695+0000 7f6172400640 1 -- 172.21.15.52:0/2843969416 shutdown_connections 2024-03-12T17:08:10.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.695+0000 7f6172400640 1 -- 172.21.15.52:0/2843969416 wait complete. 2024-03-12T17:08:10.720 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:08:10.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.907+0000 7f4696e00640 1 Processor -- start 2024-03-12T17:08:10.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.907+0000 7f4696e00640 1 -- start start 2024-03-12T17:08:10.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.909+0000 7f4696e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f46900b39b0 0x7f46900b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:10.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.909+0000 7f4696e00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f469006c6f0 con 0x7f469015b560 2024-03-12T17:08:10.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.909+0000 7f4696e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f469006c9d0 con 0x7f469015c400 2024-03-12T17:08:10.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.909+0000 7f4696e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46900b4370 con 0x7f46900b39b0 2024-03-12T17:08:10.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.909+0000 7f468fe00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f469015b560 0x7f469015b940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:44446/0 (socket says 172.21.15.52:44446) 2024-03-12T17:08:10.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.909+0000 7f468fe00640 1 -- 172.21.15.52:0/19607605 learned_addr learned my addr 172.21.15.52:0/19607605 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:08:10.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.910+0000 7f468f400640 1 --2- 172.21.15.52:0/19607605 >> v2:172.21.15.102:3300/0 conn(0x7f46900b39b0 0x7f46900b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:10.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.910+0000 7f468ea00640 1 -- 172.21.15.52:0/19607605 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1457555744 0 0) 0x7f469006c6f0 con 0x7f469015b560 2024-03-12T17:08:10.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.910+0000 7f468ea00640 1 -- 172.21.15.52:0/19607605 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4674003520 con 0x7f469015b560 2024-03-12T17:08:10.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.910+0000 7f468ea00640 1 -- 172.21.15.52:0/19607605 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3079005150 0 0) 0x7f4674003520 con 0x7f469015b560 2024-03-12T17:08:10.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.910+0000 7f468ea00640 1 -- 172.21.15.52:0/19607605 >> v2:172.21.15.102:3300/0 conn(0x7f46900b39b0 msgr2=0x7f46900b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:10.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.910+0000 7f468ea00640 1 --2- 172.21.15.52:0/19607605 >> v2:172.21.15.102:3300/0 conn(0x7f46900b39b0 0x7f46900b3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:10.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.910+0000 7f468ea00640 1 -- 172.21.15.52:0/19607605 >> v1:172.21.15.102:6789/0 conn(0x7f469015c400 legacy=0x7f469015c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:10.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.910+0000 7f468ea00640 1 -- 172.21.15.52:0/19607605 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469006ccb0 con 0x7f469015b560 2024-03-12T17:08:10.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.910+0000 7f468ea00640 1 -- 172.21.15.52:0/19607605 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f4678001670 con 0x7f469015b560 2024-03-12T17:08:10.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.910+0000 7f468ea00640 1 -- 172.21.15.52:0/19607605 >> v1:172.21.15.52:6789/0 conn(0x7f469015b560 legacy=0x7f469015b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:10.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.911+0000 7f468ea00640 1 --2- 172.21.15.52:0/19607605 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f46740039c0 0x7f4674003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:10.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.911+0000 7f468ea00640 1 --2- 172.21.15.52:0/19607605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46740047c0 0x7f4674004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:10.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.911+0000 7f468ea00640 1 --2- 172.21.15.52:0/19607605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4674005180 0x7f467400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:10.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.911+0000 7f468ea00640 1 -- 172.21.15.52:0/19607605 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46900b4370 con 0x7f46740047c0 2024-03-12T17:08:10.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.911+0000 7f468ea00640 1 -- 172.21.15.52:0/19607605 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467400c080 con 0x7f4674005180 2024-03-12T17:08:10.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.911+0000 7f468ea00640 1 -- 172.21.15.52:0/19607605 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467400c240 con 0x7f46740039c0 2024-03-12T17:08:10.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.911+0000 7f468f400640 1 --2- 172.21.15.52:0/19607605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46740047c0 0x7f4674004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:10.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.911+0000 7f468fe00640 1 --2- 172.21.15.52:0/19607605 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f46740039c0 0x7f4674003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:10.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.911+0000 7f4694e00640 1 --2- 172.21.15.52:0/19607605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4674005180 0x7f467400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:10.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.911+0000 7f468f400640 1 -- 172.21.15.52:0/19607605 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f46740039c0 msgr2=0x7f4674003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:10.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.911+0000 7f468f400640 1 --2- 172.21.15.52:0/19607605 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f46740039c0 0x7f4674003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:10.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.911+0000 7f468f400640 1 -- 172.21.15.52:0/19607605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4674005180 msgr2=0x7f467400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:10.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.911+0000 7f468f400640 1 --2- 172.21.15.52:0/19607605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4674005180 0x7f467400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:10.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.911+0000 7f468f400640 1 -- 172.21.15.52:0/19607605 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4674003520 con 0x7f46740047c0 2024-03-12T17:08:10.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.912+0000 7f4694e00640 1 --2- 172.21.15.52:0/19607605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4674005180 0x7f467400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:08:10.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.912+0000 7f4694e00640 1 -- 172.21.15.52:0/19607605 reap_dead start 2024-03-12T17:08:10.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.913+0000 7f468f400640 1 --2- 172.21.15.52:0/19607605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46740047c0 0x7f4674004c40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4680009ac0 tx=0x7f46740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31513db85411f6b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:10.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.914+0000 7f468ea00640 1 -- 172.21.15.52:0/19607605 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468002c6b0 con 0x7f46740047c0 2024-03-12T17:08:10.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.914+0000 7f468ea00640 1 -- 172.21.15.52:0/19607605 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468002c810 con 0x7f46740047c0 2024-03-12T17:08:10.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.914+0000 7f468ea00640 1 -- 172.21.15.52:0/19607605 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468002cba0 con 0x7f46740047c0 2024-03-12T17:08:10.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.914+0000 7f4696e00640 1 -- 172.21.15.52:0/19607605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46740047c0 msgr2=0x7f4674004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:10.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.914+0000 7f4696e00640 1 --2- 172.21.15.52:0/19607605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46740047c0 0x7f4674004c40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4680009ac0 tx=0x7f46740057f0 comp rx=0 tx=0).stop 2024-03-12T17:08:10.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.914+0000 7f4696e00640 1 -- 172.21.15.52:0/19607605 shutdown_connections 2024-03-12T17:08:10.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.914+0000 7f4696e00640 1 --2- 172.21.15.52:0/19607605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46740047c0 0x7f4674004c40 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:10.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.914+0000 7f4696e00640 1 -- 172.21.15.52:0/19607605 >> 172.21.15.52:0/19607605 conn(0x7f469006a3d0 msgr2=0x7f46900b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:10.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.914+0000 7f4696e00640 1 -- 172.21.15.52:0/19607605 shutdown_connections 2024-03-12T17:08:10.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.915+0000 7f4696e00640 1 -- 172.21.15.52:0/19607605 wait complete. 2024-03-12T17:08:10.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.916+0000 7f4696e00640 1 Processor -- start 2024-03-12T17:08:10.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.916+0000 7f4696e00640 1 -- start start 2024-03-12T17:08:10.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.917+0000 7f4696e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f46740047c0 0x7f4690158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:10.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.917+0000 7f4696e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4690143f80 0x7f4690153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:10.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.917+0000 7f4696e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f469014fa60 0x7f469014ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:10.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.917+0000 7f4696e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4690150450 con 0x7f469014fa60 2024-03-12T17:08:10.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.917+0000 7f4696e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4690153da0 con 0x7f4690143f80 2024-03-12T17:08:10.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.917+0000 7f4696e00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4690154000 con 0x7f46740047c0 2024-03-12T17:08:10.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.918+0000 7f4694e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f469014fa60 0x7f469014ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:10.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.918+0000 7f4694e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f469014fa60 0x7f469014ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40744/0 (socket says 172.21.15.52:40744) 2024-03-12T17:08:10.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.918+0000 7f4694e00640 1 -- 172.21.15.52:0/2394494592 learned_addr learned my addr 172.21.15.52:0/2394494592 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:10.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.918+0000 7f468f400640 1 --2- 172.21.15.52:0/2394494592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4690143f80 0x7f4690153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:10.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.918+0000 7f468fe00640 1 --2- 172.21.15.52:0/2394494592 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f46740047c0 0x7f4690158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:10.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.918+0000 7f4694e00640 1 -- 172.21.15.52:0/2394494592 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f46740047c0 msgr2=0x7f4690158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:10.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.918+0000 7f4694e00640 1 --2- 172.21.15.52:0/2394494592 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f46740047c0 0x7f4690158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:10.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.918+0000 7f4694e00640 1 -- 172.21.15.52:0/2394494592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4690143f80 msgr2=0x7f4690153860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:10.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.918+0000 7f4694e00640 1 --2- 172.21.15.52:0/2394494592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4690143f80 0x7f4690153860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:10.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.918+0000 7f4694e00640 1 -- 172.21.15.52:0/2394494592 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46901542e0 con 0x7f469014fa60 2024-03-12T17:08:10.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.918+0000 7f468fe00640 1 --2- 172.21.15.52:0/2394494592 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f46740047c0 0x7f4690158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:08:10.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.919+0000 7f4694e00640 1 --2- 172.21.15.52:0/2394494592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f469014fa60 0x7f469014ff10 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4690158a70 tx=0x7f468800d900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:10.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.919+0000 7f4687e00640 1 -- 172.21.15.52:0/2394494592 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468800b070 con 0x7f469014fa60 2024-03-12T17:08:10.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.919+0000 7f4687e00640 1 -- 172.21.15.52:0/2394494592 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4688003e60 con 0x7f469014fa60 2024-03-12T17:08:10.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.919+0000 7f4696e00640 1 -- 172.21.15.52:0/2394494592 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46901545c0 con 0x7f469014fa60 2024-03-12T17:08:10.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.919+0000 7f4687e00640 1 -- 172.21.15.52:0/2394494592 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4688006d90 con 0x7f469014fa60 2024-03-12T17:08:10.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.919+0000 7f4696e00640 1 -- 172.21.15.52:0/2394494592 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4690154b80 con 0x7f469014fa60 2024-03-12T17:08:10.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.920+0000 7f4696e00640 1 -- 172.21.15.52:0/2394494592 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f469006ac20 con 0x7f469014fa60 2024-03-12T17:08:10.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.921+0000 7f4687e00640 1 -- 172.21.15.52:0/2394494592 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4688005070 con 0x7f469014fa60 2024-03-12T17:08:10.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.923+0000 7f4687e00640 1 --2- 172.21.15.52:0/2394494592 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f466c06ac30 0x7f466c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:10.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.923+0000 7f4687e00640 1 -- 172.21.15.52:0/2394494592 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4688088380 con 0x7f469014fa60 2024-03-12T17:08:10.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.923+0000 7f468fe00640 1 --2- 172.21.15.52:0/2394494592 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f466c06ac30 0x7f466c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:10.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.925+0000 7f468fe00640 1 --2- 172.21.15.52:0/2394494592 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f466c06ac30 0x7f466c06d0f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4678001eb0 tx=0x7f4678000f10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:10.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:10.927+0000 7f4687e00640 1 -- 172.21.15.52:0/2394494592 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f46880534b0 con 0x7f469014fa60 2024-03-12T17:08:11.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.126+0000 7f4696e00640 1 -- 172.21.15.52:0/2394494592 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4690073390 con 0x7f469014fa60 2024-03-12T17:08:11.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.130+0000 7f4687e00640 1 -- 172.21.15.52:0/2394494592 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16319 (secure 0 0 0) 0x7f4688056ad0 con 0x7f469014fa60 2024-03-12T17:08:11.125 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:08:11.125 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":39,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:08:10.678661+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T17:06:19.218543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-12T17:08:09.114678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-12T17:08:09.691941+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.733031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.775178+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.591522+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.820838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.829252+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.582481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:08:11.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.133+0000 7f4696e00640 1 -- 172.21.15.52:0/2394494592 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f466c06ac30 msgr2=0x7f466c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:11.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.133+0000 7f4696e00640 1 --2- 172.21.15.52:0/2394494592 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f466c06ac30 0x7f466c06d0f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4678001eb0 tx=0x7f4678000f10 comp rx=0 tx=0).stop 2024-03-12T17:08:11.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.133+0000 7f4696e00640 1 -- 172.21.15.52:0/2394494592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f469014fa60 msgr2=0x7f469014ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:11.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.133+0000 7f4696e00640 1 --2- 172.21.15.52:0/2394494592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f469014fa60 0x7f469014ff10 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4690158a70 tx=0x7f468800d900 comp rx=0 tx=0).stop 2024-03-12T17:08:11.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.133+0000 7f4696e00640 1 -- 172.21.15.52:0/2394494592 shutdown_connections 2024-03-12T17:08:11.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.133+0000 7f4696e00640 1 --2- 172.21.15.52:0/2394494592 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f466c06ac30 0x7f466c06d0f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:11.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.133+0000 7f4696e00640 1 --2- 172.21.15.52:0/2394494592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f469014fa60 0x7f469014ff10 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:11.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.133+0000 7f4696e00640 1 --2- 172.21.15.52:0/2394494592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4690143f80 0x7f4690153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:11.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.133+0000 7f4696e00640 1 --2- 172.21.15.52:0/2394494592 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f46740047c0 0x7f4690158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:11.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.133+0000 7f4696e00640 1 -- 172.21.15.52:0/2394494592 >> 172.21.15.52:0/2394494592 conn(0x7f469006a3d0 msgr2=0x7f46900ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:11.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.133+0000 7f4696e00640 1 -- 172.21.15.52:0/2394494592 shutdown_connections 2024-03-12T17:08:11.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.133+0000 7f4696e00640 1 -- 172.21.15.52:0/2394494592 wait complete. 2024-03-12T17:08:11.145 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:08:11.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.364+0000 7f9f49c00640 1 Processor -- start 2024-03-12T17:08:11.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.364+0000 7f9f49c00640 1 -- start start 2024-03-12T17:08:11.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.365+0000 7f9f49c00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f9f440b5a30 0x7f9f440b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:11.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.365+0000 7f9f49c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9f440b4350 0x7f9f440b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:11.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.365+0000 7f9f49c00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f440d1830 con 0x7f9f4415cda0 2024-03-12T17:08:11.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.365+0000 7f9f49c00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f440b4d00 con 0x7f9f440b5a30 2024-03-12T17:08:11.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.365+0000 7f9f49c00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f440b1f20 con 0x7f9f440b4350 2024-03-12T17:08:11.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.366+0000 7f9f42a00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f9f4415cda0 0x7f9f440cdb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:44460/0 (socket says 172.21.15.52:44460) 2024-03-12T17:08:11.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.366+0000 7f9f42a00640 1 -- 172.21.15.52:0/3354999201 learned_addr learned my addr 172.21.15.52:0/3354999201 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:08:11.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.366+0000 7f9f3be00640 1 --2- 172.21.15.52:0/3354999201 >> v2:172.21.15.52:3300/0 conn(0x7f9f440b5a30 0x7f9f440b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:11.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.366+0000 7f9f43400640 1 --2- 172.21.15.52:0/3354999201 >> v2:172.21.15.102:3300/0 conn(0x7f9f440b4350 0x7f9f440b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:11.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.366+0000 7f9f42000640 1 -- 172.21.15.52:0/3354999201 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4232013719 0 0) 0x7f9f440d1830 con 0x7f9f4415cda0 2024-03-12T17:08:11.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.366+0000 7f9f42000640 1 -- 172.21.15.52:0/3354999201 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f28003520 con 0x7f9f4415cda0 2024-03-12T17:08:11.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.366+0000 7f9f3be00640 1 -- 172.21.15.52:0/3354999201 >> v2:172.21.15.102:3300/0 conn(0x7f9f440b4350 msgr2=0x7f9f440b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:11.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.366+0000 7f9f3be00640 1 --2- 172.21.15.52:0/3354999201 >> v2:172.21.15.102:3300/0 conn(0x7f9f440b4350 0x7f9f440b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:11.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.366+0000 7f9f3be00640 1 -- 172.21.15.52:0/3354999201 >> v1:172.21.15.52:6789/0 conn(0x7f9f4415cda0 legacy=0x7f9f440cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:11.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.366+0000 7f9f3be00640 1 -- 172.21.15.52:0/3354999201 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f440d0c20 con 0x7f9f440b5a30 2024-03-12T17:08:11.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.368+0000 7f9f3be00640 1 --2- 172.21.15.52:0/3354999201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f440b5a30 0x7f9f440b5e90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9f440b8460 tx=0x7f9f2c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cfbb7214f79c5a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:11.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.368+0000 7f9f42000640 1 -- 172.21.15.52:0/3354999201 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c031830 con 0x7f9f440b5a30 2024-03-12T17:08:11.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.368+0000 7f9f42000640 1 -- 172.21.15.52:0/3354999201 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2c031990 con 0x7f9f440b5a30 2024-03-12T17:08:11.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.368+0000 7f9f42000640 1 -- 172.21.15.52:0/3354999201 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c031d00 con 0x7f9f440b5a30 2024-03-12T17:08:11.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.368+0000 7f9f49c00640 1 -- 172.21.15.52:0/3354999201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f440b5a30 msgr2=0x7f9f440b5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:11.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.368+0000 7f9f49c00640 1 --2- 172.21.15.52:0/3354999201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f440b5a30 0x7f9f440b5e90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9f440b8460 tx=0x7f9f2c02b8d0 comp rx=0 tx=0).stop 2024-03-12T17:08:11.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.368+0000 7f9f49c00640 1 -- 172.21.15.52:0/3354999201 shutdown_connections 2024-03-12T17:08:11.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.369+0000 7f9f49c00640 1 --2- 172.21.15.52:0/3354999201 >> v2:172.21.15.102:3300/0 conn(0x7f9f440b4350 0x7f9f440b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:11.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.369+0000 7f9f49c00640 1 --2- 172.21.15.52:0/3354999201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f440b5a30 0x7f9f440b5e90 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:11.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.369+0000 7f9f49c00640 1 -- 172.21.15.52:0/3354999201 >> 172.21.15.52:0/3354999201 conn(0x7f9f440cf350 msgr2=0x7f9f440cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:11.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.369+0000 7f9f49c00640 1 -- 172.21.15.52:0/3354999201 shutdown_connections 2024-03-12T17:08:11.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.369+0000 7f9f49c00640 1 -- 172.21.15.52:0/3354999201 wait complete. 2024-03-12T17:08:11.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.370+0000 7f9f49c00640 1 Processor -- start 2024-03-12T17:08:11.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.371+0000 7f9f49c00640 1 -- start start 2024-03-12T17:08:11.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.372+0000 7f9f49c00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9f440b4350 0x7f9f441f1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:11.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.372+0000 7f9f49c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f440b5a30 0x7f9f441f5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:11.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.372+0000 7f9f49c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f4415cda0 0x7f9f441f5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:11.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.372+0000 7f9f49c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f440c8820 con 0x7f9f4415cda0 2024-03-12T17:08:11.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.372+0000 7f9f49c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f440c8b00 con 0x7f9f440b5a30 2024-03-12T17:08:11.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.372+0000 7f9f49c00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f440c8de0 con 0x7f9f440b4350 2024-03-12T17:08:11.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.372+0000 7f9f43400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f4415cda0 0x7f9f441f5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:11.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.372+0000 7f9f43400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f4415cda0 0x7f9f441f5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40772/0 (socket says 172.21.15.52:40772) 2024-03-12T17:08:11.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.372+0000 7f9f43400640 1 -- 172.21.15.52:0/4131813300 learned_addr learned my addr 172.21.15.52:0/4131813300 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:11.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.372+0000 7f9f42a00640 1 --2- 172.21.15.52:0/4131813300 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9f440b4350 0x7f9f441f1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:11.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.372+0000 7f9f3be00640 1 --2- 172.21.15.52:0/4131813300 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f440b5a30 0x7f9f441f5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:11.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.372+0000 7f9f43400640 1 -- 172.21.15.52:0/4131813300 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9f440b4350 msgr2=0x7f9f441f1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:11.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.372+0000 7f9f43400640 1 --2- 172.21.15.52:0/4131813300 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9f440b4350 0x7f9f441f1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:11.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.372+0000 7f9f43400640 1 -- 172.21.15.52:0/4131813300 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f440b5a30 msgr2=0x7f9f441f5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:11.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.372+0000 7f9f43400640 1 --2- 172.21.15.52:0/4131813300 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f440b5a30 0x7f9f441f5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:11.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.372+0000 7f9f43400640 1 -- 172.21.15.52:0/4131813300 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2c009400 con 0x7f9f4415cda0 2024-03-12T17:08:11.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.373+0000 7f9f43400640 1 --2- 172.21.15.52:0/4131813300 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f4415cda0 0x7f9f441f5d50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c007e40 tx=0x7f9f3c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:11.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.373+0000 7f9f3aa00640 1 -- 172.21.15.52:0/4131813300 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c014620 con 0x7f9f4415cda0 2024-03-12T17:08:11.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.373+0000 7f9f3aa00640 1 -- 172.21.15.52:0/4131813300 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f3c00ee90 con 0x7f9f4415cda0 2024-03-12T17:08:11.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.373+0000 7f9f3aa00640 1 -- 172.21.15.52:0/4131813300 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c0149e0 con 0x7f9f4415cda0 2024-03-12T17:08:11.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.373+0000 7f9f49c00640 1 -- 172.21.15.52:0/4131813300 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f440c90c0 con 0x7f9f4415cda0 2024-03-12T17:08:11.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.373+0000 7f9f49c00640 1 -- 172.21.15.52:0/4131813300 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f441f1f50 con 0x7f9f4415cda0 2024-03-12T17:08:11.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.374+0000 7f9f49c00640 1 -- 172.21.15.52:0/4131813300 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f4415e1d0 con 0x7f9f4415cda0 2024-03-12T17:08:11.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.375+0000 7f9f3aa00640 1 -- 172.21.15.52:0/4131813300 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9f3c014b40 con 0x7f9f4415cda0 2024-03-12T17:08:11.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.376+0000 7f9f3aa00640 1 --2- 172.21.15.52:0/4131813300 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f9f2006ac00 0x7f9f2006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:11.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.376+0000 7f9f3aa00640 1 -- 172.21.15.52:0/4131813300 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9f3c089e80 con 0x7f9f4415cda0 2024-03-12T17:08:11.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.377+0000 7f9f42a00640 1 --2- 172.21.15.52:0/4131813300 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f9f2006ac00 0x7f9f2006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:11.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.378+0000 7f9f42a00640 1 --2- 172.21.15.52:0/4131813300 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f9f2006ac00 0x7f9f2006d0c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9f34005880 tx=0x7f9f34009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:11.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.381+0000 7f9f3aa00640 1 -- 172.21.15.52:0/4131813300 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f9f4415e1d0 con 0x7f9f4415cda0 2024-03-12T17:08:11.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.581+0000 7f9f49c00640 1 -- 172.21.15.52:0/4131813300 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f4415e1d0 con 0x7f9f4415cda0 2024-03-12T17:08:11.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.584+0000 7f9f3aa00640 1 -- 172.21.15.52:0/4131813300 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16319 (secure 0 0 0) 0x7f9f4415e1d0 con 0x7f9f4415cda0 2024-03-12T17:08:11.578 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:08:11.578 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":39,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:08:10.678661+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T17:06:19.218543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-12T17:08:09.114678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-12T17:08:09.691941+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.733031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.775178+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.591522+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.820838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.829252+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.582481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:08:11.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.586+0000 7f9f49c00640 1 -- 172.21.15.52:0/4131813300 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f9f2006ac00 msgr2=0x7f9f2006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:11.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.586+0000 7f9f49c00640 1 --2- 172.21.15.52:0/4131813300 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f9f2006ac00 0x7f9f2006d0c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9f34005880 tx=0x7f9f34009040 comp rx=0 tx=0).stop 2024-03-12T17:08:11.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.586+0000 7f9f49c00640 1 -- 172.21.15.52:0/4131813300 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f4415cda0 msgr2=0x7f9f441f5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:11.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.586+0000 7f9f49c00640 1 --2- 172.21.15.52:0/4131813300 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f4415cda0 0x7f9f441f5d50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c007e40 tx=0x7f9f3c00e4b0 comp rx=0 tx=0).stop 2024-03-12T17:08:11.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.587+0000 7f9f49c00640 1 -- 172.21.15.52:0/4131813300 shutdown_connections 2024-03-12T17:08:11.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.587+0000 7f9f49c00640 1 --2- 172.21.15.52:0/4131813300 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f9f2006ac00 0x7f9f2006d0c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:11.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.587+0000 7f9f49c00640 1 --2- 172.21.15.52:0/4131813300 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f4415cda0 0x7f9f441f5d50 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:11.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.587+0000 7f9f49c00640 1 --2- 172.21.15.52:0/4131813300 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f440b5a30 0x7f9f441f5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:11.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.587+0000 7f9f49c00640 1 --2- 172.21.15.52:0/4131813300 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9f440b4350 0x7f9f441f1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:11.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.587+0000 7f9f49c00640 1 -- 172.21.15.52:0/4131813300 >> 172.21.15.52:0/4131813300 conn(0x7f9f440cf350 msgr2=0x7f9f4415c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:11.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.587+0000 7f9f49c00640 1 -- 172.21.15.52:0/4131813300 shutdown_connections 2024-03-12T17:08:11.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.587+0000 7f9f49c00640 1 -- 172.21.15.52:0/4131813300 wait complete. 2024-03-12T17:08:11.599 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-12T17:08:11.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.823+0000 7fef9e400640 1 Processor -- start 2024-03-12T17:08:11.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.824+0000 7fef9e400640 1 -- start start 2024-03-12T17:08:11.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.825+0000 7fef9e400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fef980ce930 0x7fef980add30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:11.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.825+0000 7fef9e400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fef9806ce90 0x7fef9806d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:11.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.825+0000 7fef9e400640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef980cdba0 con 0x7fef980cd3e0 2024-03-12T17:08:11.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.825+0000 7fef9e400640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef9806c760 con 0x7fef9806ce90 2024-03-12T17:08:11.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.825+0000 7fef9e400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef980b42d0 con 0x7fef980ce930 2024-03-12T17:08:11.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.825+0000 7fef8fe00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fef980cd3e0 0x7fef980ced10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:44466/0 (socket says 172.21.15.52:44466) 2024-03-12T17:08:11.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.825+0000 7fef8fe00640 1 -- 172.21.15.52:0/4920669 learned_addr learned my addr 172.21.15.52:0/4920669 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:08:11.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.825+0000 7fef97e00640 1 --2- 172.21.15.52:0/4920669 >> v2:172.21.15.52:3300/0 conn(0x7fef9806ce90 0x7fef9806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:11.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.826+0000 7fef97400640 1 --2- 172.21.15.52:0/4920669 >> v2:172.21.15.102:3300/0 conn(0x7fef980ce930 0x7fef980add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:11.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.826+0000 7fef96a00640 1 -- 172.21.15.52:0/4920669 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1214457629 0 0) 0x7fef980cdba0 con 0x7fef980cd3e0 2024-03-12T17:08:11.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.826+0000 7fef96a00640 1 -- 172.21.15.52:0/4920669 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef7c003520 con 0x7fef980cd3e0 2024-03-12T17:08:11.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.826+0000 7fef97e00640 1 -- 172.21.15.52:0/4920669 >> v2:172.21.15.102:3300/0 conn(0x7fef980ce930 msgr2=0x7fef980add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:11.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.826+0000 7fef97e00640 1 --2- 172.21.15.52:0/4920669 >> v2:172.21.15.102:3300/0 conn(0x7fef980ce930 0x7fef980add30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:11.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.826+0000 7fef97e00640 1 -- 172.21.15.52:0/4920669 >> v1:172.21.15.52:6789/0 conn(0x7fef980cd3e0 legacy=0x7fef980ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:11.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.826+0000 7fef97e00640 1 -- 172.21.15.52:0/4920669 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef980cde80 con 0x7fef9806ce90 2024-03-12T17:08:11.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.827+0000 7fef97e00640 1 --2- 172.21.15.52:0/4920669 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef9806ce90 0x7fef9806d300 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fef980cf540 tx=0x7fef9002bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7c6e6de67ae9ce server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:11.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.828+0000 7fef96a00640 1 -- 172.21.15.52:0/4920669 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9002cc70 con 0x7fef9806ce90 2024-03-12T17:08:11.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.828+0000 7fef96a00640 1 -- 172.21.15.52:0/4920669 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9002cdd0 con 0x7fef9806ce90 2024-03-12T17:08:11.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.828+0000 7fef96a00640 1 -- 172.21.15.52:0/4920669 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef90034640 con 0x7fef9806ce90 2024-03-12T17:08:11.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.828+0000 7fef9e400640 1 -- 172.21.15.52:0/4920669 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef9806ce90 msgr2=0x7fef9806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:11.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.828+0000 7fef9e400640 1 --2- 172.21.15.52:0/4920669 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef9806ce90 0x7fef9806d300 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fef980cf540 tx=0x7fef9002bec0 comp rx=0 tx=0).stop 2024-03-12T17:08:11.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.828+0000 7fef9e400640 1 -- 172.21.15.52:0/4920669 shutdown_connections 2024-03-12T17:08:11.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.828+0000 7fef9e400640 1 --2- 172.21.15.52:0/4920669 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef9806ce90 0x7fef9806d300 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:11.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.828+0000 7fef9e400640 1 --2- 172.21.15.52:0/4920669 >> v2:172.21.15.102:3300/0 conn(0x7fef980ce930 0x7fef980add30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:11.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.829+0000 7fef9e400640 1 -- 172.21.15.52:0/4920669 >> 172.21.15.52:0/4920669 conn(0x7fef9806a0b0 msgr2=0x7fef9806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:11.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.829+0000 7fef9e400640 1 -- 172.21.15.52:0/4920669 shutdown_connections 2024-03-12T17:08:11.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.829+0000 7fef9e400640 1 -- 172.21.15.52:0/4920669 wait complete. 2024-03-12T17:08:11.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.830+0000 7fef9e400640 1 Processor -- start 2024-03-12T17:08:11.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.830+0000 7fef9e400640 1 -- start start 2024-03-12T17:08:11.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.831+0000 7fef9e400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fef9806ce90 0x7fef980ae480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:11.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.831+0000 7fef9e400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef980cd3e0 0x7fef980ac270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:11.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.831+0000 7fef9e400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef980ce930 0x7fef981588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:11.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.831+0000 7fef9e400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef981f1fe0 con 0x7fef980cd3e0 2024-03-12T17:08:11.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.831+0000 7fef9e400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef980ac7b0 con 0x7fef980ce930 2024-03-12T17:08:11.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.831+0000 7fef9e400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef980b3080 con 0x7fef9806ce90 2024-03-12T17:08:11.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.832+0000 7fef8fe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef980cd3e0 0x7fef980ac270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:11.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.832+0000 7fef8fe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef980cd3e0 0x7fef980ac270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40776/0 (socket says 172.21.15.52:40776) 2024-03-12T17:08:11.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.832+0000 7fef8fe00640 1 -- 172.21.15.52:0/3361914674 learned_addr learned my addr 172.21.15.52:0/3361914674 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:11.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.832+0000 7fef8fe00640 1 -- 172.21.15.52:0/3361914674 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fef9806ce90 msgr2=0x7fef980ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:11.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.832+0000 7fef8fe00640 1 --2- 172.21.15.52:0/3361914674 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fef9806ce90 0x7fef980ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:11.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.832+0000 7fef8fe00640 1 -- 172.21.15.52:0/3361914674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef980ce930 msgr2=0x7fef981588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:11.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.832+0000 7fef8fe00640 1 --2- 172.21.15.52:0/3361914674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef980ce930 0x7fef981588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:11.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.832+0000 7fef8fe00640 1 -- 172.21.15.52:0/3361914674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef90009a70 con 0x7fef980cd3e0 2024-03-12T17:08:11.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.832+0000 7fef8fe00640 1 --2- 172.21.15.52:0/3361914674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef980cd3e0 0x7fef980ac270 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fef8000c430 tx=0x7fef8000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:11.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.833+0000 7fef8f400640 1 -- 172.21.15.52:0/3361914674 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef80005ce0 con 0x7fef980cd3e0 2024-03-12T17:08:11.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.833+0000 7fef8f400640 1 -- 172.21.15.52:0/3361914674 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef80005e40 con 0x7fef980cd3e0 2024-03-12T17:08:11.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.833+0000 7fef8f400640 1 -- 172.21.15.52:0/3361914674 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef800072f0 con 0x7fef980cd3e0 2024-03-12T17:08:11.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.833+0000 7fef9e400640 1 -- 172.21.15.52:0/3361914674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef980b3360 con 0x7fef980cd3e0 2024-03-12T17:08:11.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.833+0000 7fef9e400640 1 -- 172.21.15.52:0/3361914674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef981580a0 con 0x7fef980cd3e0 2024-03-12T17:08:11.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.834+0000 7fef9e400640 1 -- 172.21.15.52:0/3361914674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef98073390 con 0x7fef980cd3e0 2024-03-12T17:08:11.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.835+0000 7fef8f400640 1 -- 172.21.15.52:0/3361914674 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fef80007450 con 0x7fef980cd3e0 2024-03-12T17:08:11.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.836+0000 7fef8f400640 1 --2- 172.21.15.52:0/3361914674 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fef7406ac00 0x7fef7406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:11.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.836+0000 7fef8f400640 1 -- 172.21.15.52:0/3361914674 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fef80088b50 con 0x7fef980cd3e0 2024-03-12T17:08:11.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.837+0000 7fef97400640 1 --2- 172.21.15.52:0/3361914674 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fef7406ac00 0x7fef7406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:11.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.838+0000 7fef97400640 1 --2- 172.21.15.52:0/3361914674 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fef7406ac00 0x7fef7406d0c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fef88006fd0 tx=0x7fef88008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:11.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:11.841+0000 7fef8f400640 1 -- 172.21.15.52:0/3361914674 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fef80057670 con 0x7fef980cd3e0 2024-03-12T17:08:12.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.081+0000 7fef9e400640 1 -- 172.21.15.52:0/3361914674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fef9815ce70 con 0x7fef980cd3e0 2024-03-12T17:08:12.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.693+0000 7fef8f400640 1 -- 172.21.15.52:0/3361914674 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7fef80017020 con 0x7fef980cd3e0 2024-03-12T17:08:12.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.697+0000 7fef9e400640 1 -- 172.21.15.52:0/3361914674 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fef7406ac00 msgr2=0x7fef7406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:12.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.697+0000 7fef9e400640 1 --2- 172.21.15.52:0/3361914674 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fef7406ac00 0x7fef7406d0c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fef88006fd0 tx=0x7fef88008040 comp rx=0 tx=0).stop 2024-03-12T17:08:12.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.697+0000 7fef9e400640 1 -- 172.21.15.52:0/3361914674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef980cd3e0 msgr2=0x7fef980ac270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:12.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.697+0000 7fef9e400640 1 --2- 172.21.15.52:0/3361914674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef980cd3e0 0x7fef980ac270 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fef8000c430 tx=0x7fef8000c8f0 comp rx=0 tx=0).stop 2024-03-12T17:08:12.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.697+0000 7fef9e400640 1 -- 172.21.15.52:0/3361914674 shutdown_connections 2024-03-12T17:08:12.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.697+0000 7fef9e400640 1 --2- 172.21.15.52:0/3361914674 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fef7406ac00 0x7fef7406d0c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:12.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.697+0000 7fef9e400640 1 --2- 172.21.15.52:0/3361914674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef980ce930 0x7fef981588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:12.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.697+0000 7fef9e400640 1 --2- 172.21.15.52:0/3361914674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef980cd3e0 0x7fef980ac270 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:12.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.697+0000 7fef9e400640 1 --2- 172.21.15.52:0/3361914674 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fef9806ce90 0x7fef980ae480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:12.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.697+0000 7fef9e400640 1 -- 172.21.15.52:0/3361914674 >> 172.21.15.52:0/3361914674 conn(0x7fef9806a0b0 msgr2=0x7fef981542d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:12.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.697+0000 7fef9e400640 1 -- 172.21.15.52:0/3361914674 shutdown_connections 2024-03-12T17:08:12.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.697+0000 7fef9e400640 1 -- 172.21.15.52:0/3361914674 wait complete. 2024-03-12T17:08:12.712 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-12T17:08:12.712 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-12T17:08:12.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.949+0000 7f6d8b200640 1 Processor -- start 2024-03-12T17:08:12.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.949+0000 7f6d8b200640 1 -- start start 2024-03-12T17:08:12.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.950+0000 7f6d8b200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d8c0b22b0 0x7f6d8c15eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:12.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.950+0000 7f6d8b200640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d8c15e320 con 0x7f6d8c15f020 2024-03-12T17:08:12.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.950+0000 7f6d8b200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d8c0cda20 con 0x7f6d8c15a850 2024-03-12T17:08:12.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.951+0000 7f6d8b200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c15ac30 con 0x7f6d8c0b22b0 2024-03-12T17:08:12.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.951+0000 7f6d8a800640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f6d8c15f020 0x7f6d8c0cacf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:44468/0 (socket says 172.21.15.52:44468) 2024-03-12T17:08:12.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.951+0000 7f6d8a800640 1 -- 172.21.15.52:0/4101903345 learned_addr learned my addr 172.21.15.52:0/4101903345 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:08:12.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.951+0000 7f6d89400640 1 -- 172.21.15.52:0/4101903345 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2621982951 0 0) 0x7f6d8c15e320 con 0x7f6d8c15f020 2024-03-12T17:08:12.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.951+0000 7f6d89400640 1 -- 172.21.15.52:0/4101903345 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d70003520 con 0x7f6d8c15f020 2024-03-12T17:08:12.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.951+0000 7f6d83e00640 1 --2- 172.21.15.52:0/4101903345 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d8c0b22b0 0x7f6d8c15eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:12.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.952+0000 7f6d89400640 1 -- 172.21.15.52:0/4101903345 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3502462019 0 0) 0x7f6d70003520 con 0x7f6d8c15f020 2024-03-12T17:08:12.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.952+0000 7f6d89400640 1 -- 172.21.15.52:0/4101903345 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d8c0b22b0 msgr2=0x7f6d8c15eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:12.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.952+0000 7f6d89400640 1 --2- 172.21.15.52:0/4101903345 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d8c0b22b0 0x7f6d8c15eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:12.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.952+0000 7f6d89400640 1 -- 172.21.15.52:0/4101903345 >> v1:172.21.15.102:6789/0 conn(0x7f6d8c15a850 legacy=0x7f6d8c0cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:12.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.952+0000 7f6d89400640 1 -- 172.21.15.52:0/4101903345 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8c0cdd00 con 0x7f6d8c15f020 2024-03-12T17:08:12.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.952+0000 7f6d89400640 1 -- 172.21.15.52:0/4101903345 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f6d840019f0 con 0x7f6d8c15f020 2024-03-12T17:08:12.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.952+0000 7f6d89400640 1 -- 172.21.15.52:0/4101903345 >> v1:172.21.15.52:6789/0 conn(0x7f6d8c15f020 legacy=0x7f6d8c0cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:12.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.952+0000 7f6d89400640 1 --2- 172.21.15.52:0/4101903345 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d8c0b22b0 0x7f6d700039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:12.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.952+0000 7f6d89400640 1 --2- 172.21.15.52:0/4101903345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d70004020 0x7f6d700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:12.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.952+0000 7f6d89400640 1 --2- 172.21.15.52:0/4101903345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d70004d00 0x7f6d70005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:12.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.952+0000 7f6d89400640 1 -- 172.21.15.52:0/4101903345 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c15ac30 con 0x7f6d70004020 2024-03-12T17:08:12.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.952+0000 7f6d89400640 1 -- 172.21.15.52:0/4101903345 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7000bbc0 con 0x7f6d70004d00 2024-03-12T17:08:12.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.952+0000 7f6d89400640 1 -- 172.21.15.52:0/4101903345 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7000bd80 con 0x7f6d8c0b22b0 2024-03-12T17:08:12.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.952+0000 7f6d83e00640 1 --2- 172.21.15.52:0/4101903345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d70004020 0x7f6d700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:12.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.952+0000 7f6d89e00640 1 --2- 172.21.15.52:0/4101903345 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d8c0b22b0 0x7f6d700039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:12.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.953+0000 7f6d8a800640 1 --2- 172.21.15.52:0/4101903345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d70004d00 0x7f6d70005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:12.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.953+0000 7f6d83e00640 1 -- 172.21.15.52:0/4101903345 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d8c0b22b0 msgr2=0x7f6d700039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:12.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.953+0000 7f6d83e00640 1 --2- 172.21.15.52:0/4101903345 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d8c0b22b0 0x7f6d700039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:12.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.953+0000 7f6d83e00640 1 -- 172.21.15.52:0/4101903345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d70004d00 msgr2=0x7f6d70005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:12.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.953+0000 7f6d83e00640 1 --2- 172.21.15.52:0/4101903345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d70004d00 0x7f6d70005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:12.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.953+0000 7f6d83e00640 1 -- 172.21.15.52:0/4101903345 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d70003520 con 0x7f6d70004020 2024-03-12T17:08:12.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.954+0000 7f6d83e00640 1 --2- 172.21.15.52:0/4101903345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d70004020 0x7f6d700047c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6d7400ba30 tx=0x7f6d700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37aad2bf0139649c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:12.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.954+0000 7f6d89400640 1 -- 172.21.15.52:0/4101903345 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d74032640 con 0x7f6d70004020 2024-03-12T17:08:12.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.954+0000 7f6d89400640 1 -- 172.21.15.52:0/4101903345 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d740327a0 con 0x7f6d70004020 2024-03-12T17:08:12.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.954+0000 7f6d89400640 1 -- 172.21.15.52:0/4101903345 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d74032b30 con 0x7f6d70004020 2024-03-12T17:08:12.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.955+0000 7f6d8b200640 1 -- 172.21.15.52:0/4101903345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d70004020 msgr2=0x7f6d700047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:12.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.955+0000 7f6d8b200640 1 --2- 172.21.15.52:0/4101903345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d70004020 0x7f6d700047c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6d7400ba30 tx=0x7f6d700057f0 comp rx=0 tx=0).stop 2024-03-12T17:08:12.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.955+0000 7f6d8a800640 1 -- 172.21.15.52:0/4101903345 reap_dead start 2024-03-12T17:08:12.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.955+0000 7f6d8b200640 1 -- 172.21.15.52:0/4101903345 shutdown_connections 2024-03-12T17:08:12.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.955+0000 7f6d8b200640 1 -- 172.21.15.52:0/4101903345 >> 172.21.15.52:0/4101903345 conn(0x7f6d8c0b01b0 msgr2=0x7f6d8c0b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:12.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.955+0000 7f6d8b200640 1 -- 172.21.15.52:0/4101903345 shutdown_connections 2024-03-12T17:08:12.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.955+0000 7f6d8b200640 1 -- 172.21.15.52:0/4101903345 wait complete. 2024-03-12T17:08:12.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.957+0000 7f6d8b200640 1 Processor -- start 2024-03-12T17:08:12.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.957+0000 7f6d8b200640 1 -- start start 2024-03-12T17:08:12.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.958+0000 7f6d8b200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d70004020 0x7f6d8c14b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:12.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.958+0000 7f6d8b200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d8c156b20 0x7f6d8c15b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:12.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.958+0000 7f6d8b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d8c15bbf0 0x7f6d8c0ac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:12.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.958+0000 7f6d8b200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c0ac180 con 0x7f6d70004020 2024-03-12T17:08:12.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.958+0000 7f6d8b200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c0718d0 con 0x7f6d8c15bbf0 2024-03-12T17:08:12.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.958+0000 7f6d8b200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c15c070 con 0x7f6d8c156b20 2024-03-12T17:08:12.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.958+0000 7f6d89e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d70004020 0x7f6d8c14b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:12.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.958+0000 7f6d89e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d70004020 0x7f6d8c14b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40796/0 (socket says 172.21.15.52:40796) 2024-03-12T17:08:12.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.958+0000 7f6d89e00640 1 -- 172.21.15.52:0/2887922215 learned_addr learned my addr 172.21.15.52:0/2887922215 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:12.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.959+0000 7f6d83e00640 1 --2- 172.21.15.52:0/2887922215 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d8c156b20 0x7f6d8c15b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:12.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.959+0000 7f6d8a800640 1 --2- 172.21.15.52:0/2887922215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d8c15bbf0 0x7f6d8c0ac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:12.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.959+0000 7f6d89e00640 1 -- 172.21.15.52:0/2887922215 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d8c156b20 msgr2=0x7f6d8c15b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:12.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.959+0000 7f6d89e00640 1 --2- 172.21.15.52:0/2887922215 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d8c156b20 0x7f6d8c15b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:12.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.959+0000 7f6d89e00640 1 -- 172.21.15.52:0/2887922215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d8c15bbf0 msgr2=0x7f6d8c0ac5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:12.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.959+0000 7f6d89e00640 1 --2- 172.21.15.52:0/2887922215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d8c15bbf0 0x7f6d8c0ac5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:12.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.959+0000 7f6d89e00640 1 -- 172.21.15.52:0/2887922215 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8c15afe0 con 0x7f6d70004020 2024-03-12T17:08:12.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.959+0000 7f6d89e00640 1 --2- 172.21.15.52:0/2887922215 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d70004020 0x7f6d8c14b8f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c00f850 tx=0x7f6d7c00fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:12.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.959+0000 7f6d82000640 1 -- 172.21.15.52:0/2887922215 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c00fe40 con 0x7f6d70004020 2024-03-12T17:08:12.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.959+0000 7f6d82000640 1 -- 172.21.15.52:0/2887922215 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d7c0052e0 con 0x7f6d70004020 2024-03-12T17:08:12.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.959+0000 7f6d82000640 1 -- 172.21.15.52:0/2887922215 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c00d910 con 0x7f6d70004020 2024-03-12T17:08:12.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.959+0000 7f6d8b200640 1 -- 172.21.15.52:0/2887922215 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d8c0715f0 con 0x7f6d70004020 2024-03-12T17:08:12.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.960+0000 7f6d8b200640 1 -- 172.21.15.52:0/2887922215 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d8c0abda0 con 0x7f6d70004020 2024-03-12T17:08:12.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.961+0000 7f6d8b200640 1 -- 172.21.15.52:0/2887922215 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d8c0ac3f0 con 0x7f6d70004020 2024-03-12T17:08:12.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.962+0000 7f6d82000640 1 -- 172.21.15.52:0/2887922215 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6d7c00da70 con 0x7f6d70004020 2024-03-12T17:08:12.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.962+0000 7f6d82000640 1 --2- 172.21.15.52:0/2887922215 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f6d6806ac00 0x7f6d6806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:12.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.962+0000 7f6d82000640 1 -- 172.21.15.52:0/2887922215 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6d7c089390 con 0x7f6d70004020 2024-03-12T17:08:12.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.963+0000 7f6d83e00640 1 --2- 172.21.15.52:0/2887922215 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f6d6806ac00 0x7f6d6806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:12.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.964+0000 7f6d83e00640 1 --2- 172.21.15.52:0/2887922215 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f6d6806ac00 0x7f6d6806d0c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6d7400ba00 tx=0x7f6d74030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:12.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:12.967+0000 7f6d82000640 1 -- 172.21.15.52:0/2887922215 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f6d7c054430 con 0x7f6d70004020 2024-03-12T17:08:12.994 INFO:tasks.ceph.mds.a.smithi052.stderr:2024-03-12T17:08:13.000+0000 7f36b0e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-12T17:08:13.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:13.211+0000 7f6d8b200640 1 -- 172.21.15.52:0/2887922215 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6d8c0ab920 con 0x7f6d70004020 2024-03-12T17:08:14.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.154+0000 7f6d82000640 1 -- 172.21.15.52:0/2887922215 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6d8c0ab920 con 0x7f6d70004020 2024-03-12T17:08:14.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.157+0000 7f6d8b200640 1 -- 172.21.15.52:0/2887922215 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f6d6806ac00 msgr2=0x7f6d6806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:14.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.157+0000 7f6d8b200640 1 --2- 172.21.15.52:0/2887922215 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f6d6806ac00 0x7f6d6806d0c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6d7400ba00 tx=0x7f6d74030000 comp rx=0 tx=0).stop 2024-03-12T17:08:14.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.157+0000 7f6d8b200640 1 -- 172.21.15.52:0/2887922215 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d70004020 msgr2=0x7f6d8c14b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:14.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.157+0000 7f6d8b200640 1 --2- 172.21.15.52:0/2887922215 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d70004020 0x7f6d8c14b8f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c00f850 tx=0x7f6d7c00fd10 comp rx=0 tx=0).stop 2024-03-12T17:08:14.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.157+0000 7f6d8b200640 1 -- 172.21.15.52:0/2887922215 shutdown_connections 2024-03-12T17:08:14.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.157+0000 7f6d8b200640 1 --2- 172.21.15.52:0/2887922215 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f6d6806ac00 0x7f6d6806d0c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:14.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.157+0000 7f6d8b200640 1 --2- 172.21.15.52:0/2887922215 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d8c156b20 0x7f6d8c15b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:14.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.157+0000 7f6d8b200640 1 --2- 172.21.15.52:0/2887922215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d8c15bbf0 0x7f6d8c0ac5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:14.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.157+0000 7f6d8b200640 1 --2- 172.21.15.52:0/2887922215 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d70004020 0x7f6d8c14b8f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:14.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.157+0000 7f6d8b200640 1 -- 172.21.15.52:0/2887922215 >> 172.21.15.52:0/2887922215 conn(0x7f6d8c0b01b0 msgr2=0x7f6d8c0b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:14.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.158+0000 7f6d8b200640 1 -- 172.21.15.52:0/2887922215 shutdown_connections 2024-03-12T17:08:14.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.158+0000 7f6d8b200640 1 -- 172.21.15.52:0/2887922215 wait complete. 2024-03-12T17:08:14.174 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-12T17:08:14.174 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T17:08:14.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.397+0000 7f5b8ec00640 1 Processor -- start 2024-03-12T17:08:14.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.398+0000 7f5b8ec00640 1 -- start start 2024-03-12T17:08:14.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.399+0000 7f5b8ec00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5b880b3310 0x7f5b880cae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:14.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.399+0000 7f5b8ec00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f5b880cb370 0x7f5b880b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:14.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.399+0000 7f5b8ec00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b880cc8c0 con 0x7f5b88156e30 2024-03-12T17:08:14.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.400+0000 7f5b8ec00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b880cb7e0 con 0x7f5b880cb370 2024-03-12T17:08:14.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.400+0000 7f5b8ec00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8815ce60 con 0x7f5b880b3310 2024-03-12T17:08:14.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.400+0000 7f5b8e200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f5b880cb370 0x7f5b880b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:14.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.400+0000 7f5b8e200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f5b880cb370 0x7f5b880b1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40798/0 (socket says 172.21.15.52:40798) 2024-03-12T17:08:14.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.400+0000 7f5b8e200640 1 -- 172.21.15.52:0/3045834755 learned_addr learned my addr 172.21.15.52:0/3045834755 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:14.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.400+0000 7f5b8ce00640 1 --2- 172.21.15.52:0/3045834755 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5b880b3310 0x7f5b880cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:14.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.400+0000 7f5b8e200640 1 -- 172.21.15.52:0/3045834755 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5b880b3310 msgr2=0x7f5b880cae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:14.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.400+0000 7f5b8e200640 1 --2- 172.21.15.52:0/3045834755 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5b880b3310 0x7f5b880cae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:14.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.400+0000 7f5b8e200640 1 -- 172.21.15.52:0/3045834755 >> v1:172.21.15.102:6789/0 conn(0x7f5b88156e30 legacy=0x7f5b8806d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:14.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.400+0000 7f5b8e200640 1 -- 172.21.15.52:0/3045834755 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b8815d120 con 0x7f5b880cb370 2024-03-12T17:08:14.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.402+0000 7f5b8e200640 1 --2- 172.21.15.52:0/3045834755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b880cb370 0x7f5b880b1d70 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5b8806a9b0 tx=0x7f5b8002be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492420dd2143e881 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:14.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.402+0000 7f5b87e00640 1 -- 172.21.15.52:0/3045834755 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8002ca30 con 0x7f5b880cb370 2024-03-12T17:08:14.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.402+0000 7f5b87e00640 1 -- 172.21.15.52:0/3045834755 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b8002cb90 con 0x7f5b880cb370 2024-03-12T17:08:14.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.402+0000 7f5b87e00640 1 -- 172.21.15.52:0/3045834755 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b800343f0 con 0x7f5b880cb370 2024-03-12T17:08:14.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.402+0000 7f5b8ec00640 1 -- 172.21.15.52:0/3045834755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b880cb370 msgr2=0x7f5b880b1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:14.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.402+0000 7f5b8ec00640 1 --2- 172.21.15.52:0/3045834755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b880cb370 0x7f5b880b1d70 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5b8806a9b0 tx=0x7f5b8002be90 comp rx=0 tx=0).stop 2024-03-12T17:08:14.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.403+0000 7f5b8ec00640 1 -- 172.21.15.52:0/3045834755 shutdown_connections 2024-03-12T17:08:14.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.403+0000 7f5b8ec00640 1 --2- 172.21.15.52:0/3045834755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b880cb370 0x7f5b880b1d70 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:14.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.403+0000 7f5b8ec00640 1 --2- 172.21.15.52:0/3045834755 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5b880b3310 0x7f5b880cae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:14.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.403+0000 7f5b8ec00640 1 -- 172.21.15.52:0/3045834755 >> 172.21.15.52:0/3045834755 conn(0x7f5b88065df0 msgr2=0x7f5b88069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:14.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.403+0000 7f5b8ec00640 1 -- 172.21.15.52:0/3045834755 shutdown_connections 2024-03-12T17:08:14.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.403+0000 7f5b8ec00640 1 -- 172.21.15.52:0/3045834755 wait complete. 2024-03-12T17:08:14.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.405+0000 7f5b8ec00640 1 Processor -- start 2024-03-12T17:08:14.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.405+0000 7f5b8ec00640 1 -- start start 2024-03-12T17:08:14.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.406+0000 7f5b8ec00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b880b3310 0x7f5b880af100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:14.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.406+0000 7f5b8ec00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5b880cb370 0x7f5b8813ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:14.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.406+0000 7f5b8ec00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b88156e30 0x7f5b8813f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:14.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.406+0000 7f5b8ec00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b880cb170 con 0x7f5b880b3310 2024-03-12T17:08:14.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.406+0000 7f5b8ec00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b880af750 con 0x7f5b88156e30 2024-03-12T17:08:14.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.406+0000 7f5b8ec00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b880afa30 con 0x7f5b880cb370 2024-03-12T17:08:14.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.406+0000 7f5b8d800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b880b3310 0x7f5b880af100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:14.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.406+0000 7f5b8d800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b880b3310 0x7f5b880af100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40810/0 (socket says 172.21.15.52:40810) 2024-03-12T17:08:14.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.406+0000 7f5b8d800640 1 -- 172.21.15.52:0/2298935498 learned_addr learned my addr 172.21.15.52:0/2298935498 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:14.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.406+0000 7f5b8ce00640 1 --2- 172.21.15.52:0/2298935498 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5b880cb370 0x7f5b8813ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:14.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.406+0000 7f5b8d800640 1 -- 172.21.15.52:0/2298935498 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5b880cb370 msgr2=0x7f5b8813ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:14.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.406+0000 7f5b8d800640 1 --2- 172.21.15.52:0/2298935498 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5b880cb370 0x7f5b8813ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:14.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.406+0000 7f5b8e200640 1 --2- 172.21.15.52:0/2298935498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b88156e30 0x7f5b8813f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:14.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.406+0000 7f5b8d800640 1 -- 172.21.15.52:0/2298935498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b88156e30 msgr2=0x7f5b8813f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:14.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.406+0000 7f5b8d800640 1 --2- 172.21.15.52:0/2298935498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b88156e30 0x7f5b8813f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:14.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.406+0000 7f5b8d800640 1 -- 172.21.15.52:0/2298935498 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b80009b30 con 0x7f5b880b3310 2024-03-12T17:08:14.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.407+0000 7f5b8d800640 1 --2- 172.21.15.52:0/2298935498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b880b3310 0x7f5b880af100 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c00b3f0 tx=0x7f5b7c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:14.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.407+0000 7f5b85600640 1 -- 172.21.15.52:0/2298935498 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c011830 con 0x7f5b880b3310 2024-03-12T17:08:14.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.407+0000 7f5b85600640 1 -- 172.21.15.52:0/2298935498 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b7c011990 con 0x7f5b880b3310 2024-03-12T17:08:14.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.407+0000 7f5b85600640 1 -- 172.21.15.52:0/2298935498 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c011d40 con 0x7f5b880b3310 2024-03-12T17:08:14.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.407+0000 7f5b8ec00640 1 -- 172.21.15.52:0/2298935498 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b880afd10 con 0x7f5b880b3310 2024-03-12T17:08:14.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.407+0000 7f5b8ec00640 1 -- 172.21.15.52:0/2298935498 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b880b02d0 con 0x7f5b880b3310 2024-03-12T17:08:14.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.409+0000 7f5b8ec00640 1 -- 172.21.15.52:0/2298935498 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b880b04f0 con 0x7f5b880b3310 2024-03-12T17:08:14.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.410+0000 7f5b85600640 1 -- 172.21.15.52:0/2298935498 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5b7c01a400 con 0x7f5b880b3310 2024-03-12T17:08:14.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.410+0000 7f5b85600640 1 --2- 172.21.15.52:0/2298935498 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f5b6806ac30 0x7f5b6806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:14.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.411+0000 7f5b85600640 1 -- 172.21.15.52:0/2298935498 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5b7c089300 con 0x7f5b880b3310 2024-03-12T17:08:14.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.411+0000 7f5b8ce00640 1 --2- 172.21.15.52:0/2298935498 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f5b6806ac30 0x7f5b6806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:14.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.412+0000 7f5b8ce00640 1 --2- 172.21.15.52:0/2298935498 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f5b6806ac30 0x7f5b6806d0f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5b8815fd40 tx=0x7f5b74008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:14.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.415+0000 7f5b85600640 1 -- 172.21.15.52:0/2298935498 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f5b7c054300 con 0x7f5b880b3310 2024-03-12T17:08:14.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.658+0000 7f5b8ec00640 1 -- 172.21.15.52:0/2298935498 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b8804e030 con 0x7f5b880b3310 2024-03-12T17:08:14.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.660+0000 7f5b85600640 1 -- 172.21.15.52:0/2298935498 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4734 (secure 0 0 0) 0x7f5b7c057920 con 0x7f5b880b3310 2024-03-12T17:08:14.654 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:08:14.654 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6836/285561331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":285561331},{"type":"v1","addr":"172.21.15.102:6836","nonce":285561331}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4909,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1312095655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6835","nonce":1312095655},{"type":"v1","addr":"172.21.15.102:6837","nonce":1312095655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5337,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2451989690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2451989690},{"type":"v1","addr":"172.21.15.52:6833","nonce":2451989690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-12T17:08:10.679436+0000","modified":"2024-03-12T17:08:14.140830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5343},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5343":{"gid":5343,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.52:6835/543567396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":543567396},{"type":"v1","addr":"172.21.15.52:6835","nonce":543567396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-12T17:08:14.654 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 17 2024-03-12T17:08:14.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.662+0000 7f5b8ec00640 1 -- 172.21.15.52:0/2298935498 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f5b6806ac30 msgr2=0x7f5b6806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:14.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.662+0000 7f5b8ec00640 1 --2- 172.21.15.52:0/2298935498 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f5b6806ac30 0x7f5b6806d0f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5b8815fd40 tx=0x7f5b74008040 comp rx=0 tx=0).stop 2024-03-12T17:08:14.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.662+0000 7f5b8ec00640 1 -- 172.21.15.52:0/2298935498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b880b3310 msgr2=0x7f5b880af100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:14.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.662+0000 7f5b8ec00640 1 --2- 172.21.15.52:0/2298935498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b880b3310 0x7f5b880af100 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c00b3f0 tx=0x7f5b7c00b8b0 comp rx=0 tx=0).stop 2024-03-12T17:08:14.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.663+0000 7f5b8ec00640 1 -- 172.21.15.52:0/2298935498 shutdown_connections 2024-03-12T17:08:14.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.663+0000 7f5b8ec00640 1 --2- 172.21.15.52:0/2298935498 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f5b6806ac30 0x7f5b6806d0f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:14.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.663+0000 7f5b8ec00640 1 --2- 172.21.15.52:0/2298935498 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5b880cb370 0x7f5b8813ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:14.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.663+0000 7f5b8ec00640 1 --2- 172.21.15.52:0/2298935498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b88156e30 0x7f5b8813f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:14.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.663+0000 7f5b8ec00640 1 --2- 172.21.15.52:0/2298935498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b880b3310 0x7f5b880af100 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:14.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.663+0000 7f5b8ec00640 1 -- 172.21.15.52:0/2298935498 >> 172.21.15.52:0/2298935498 conn(0x7f5b88065df0 msgr2=0x7f5b880aca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:14.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.663+0000 7f5b8ec00640 1 -- 172.21.15.52:0/2298935498 shutdown_connections 2024-03-12T17:08:14.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.663+0000 7f5b8ec00640 1 -- 172.21.15.52:0/2298935498 wait complete. 2024-03-12T17:08:14.677 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:08:14.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.904+0000 7fae78200640 1 Processor -- start 2024-03-12T17:08:14.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.905+0000 7fae78200640 1 -- start start 2024-03-12T17:08:14.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.906+0000 7fae78200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fae700b5a30 0x7fae700b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:14.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.907+0000 7fae78200640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae700d1830 con 0x7fae7015cda0 2024-03-12T17:08:14.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.907+0000 7fae78200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae700b1f20 con 0x7fae700b4350 2024-03-12T17:08:14.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.907+0000 7fae78200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae700d0c20 con 0x7fae700b5a30 2024-03-12T17:08:14.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.907+0000 7fae75800640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fae7015cda0 0x7fae700cdb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:34638/0 (socket says 172.21.15.52:34638) 2024-03-12T17:08:14.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.907+0000 7fae75800640 1 -- 172.21.15.52:0/2642425751 learned_addr learned my addr 172.21.15.52:0/2642425751 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:08:14.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.907+0000 7fae74e00640 1 --2- 172.21.15.52:0/2642425751 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fae700b5a30 0x7fae700b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:14.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.907+0000 7fae6fe00640 1 -- 172.21.15.52:0/2642425751 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3666320170 0 0) 0x7fae700d1830 con 0x7fae7015cda0 2024-03-12T17:08:14.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.907+0000 7fae6fe00640 1 -- 172.21.15.52:0/2642425751 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae58003520 con 0x7fae7015cda0 2024-03-12T17:08:14.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.908+0000 7fae6fe00640 1 -- 172.21.15.52:0/2642425751 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1714730752 0 0) 0x7fae58003520 con 0x7fae7015cda0 2024-03-12T17:08:14.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.908+0000 7fae6fe00640 1 -- 172.21.15.52:0/2642425751 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fae700b5a30 msgr2=0x7fae700b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:14.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.908+0000 7fae6fe00640 1 --2- 172.21.15.52:0/2642425751 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fae700b5a30 0x7fae700b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:14.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.908+0000 7fae6fe00640 1 -- 172.21.15.52:0/2642425751 >> v1:172.21.15.102:6789/0 conn(0x7fae700b4350 legacy=0x7fae700d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:14.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.908+0000 7fae6fe00640 1 -- 172.21.15.52:0/2642425751 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae700d0f00 con 0x7fae7015cda0 2024-03-12T17:08:14.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.908+0000 7fae6fe00640 1 -- 172.21.15.52:0/2642425751 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7fae64001670 con 0x7fae7015cda0 2024-03-12T17:08:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.908+0000 7fae6fe00640 1 -- 172.21.15.52:0/2642425751 >> v1:172.21.15.52:6789/0 conn(0x7fae7015cda0 legacy=0x7fae700cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.908+0000 7fae6fe00640 1 --2- 172.21.15.52:0/2642425751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae580039a0 0x7fae58003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.908+0000 7fae6fe00640 1 --2- 172.21.15.52:0/2642425751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae580047c0 0x7fae58004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.908+0000 7fae6fe00640 1 --2- 172.21.15.52:0/2642425751 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fae700b5a30 0x7fae58005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.908+0000 7fae6fe00640 1 -- 172.21.15.52:0/2642425751 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae700d0c20 con 0x7fae580047c0 2024-03-12T17:08:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.908+0000 7fae6fe00640 1 -- 172.21.15.52:0/2642425751 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5800ba60 con 0x7fae580039a0 2024-03-12T17:08:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.908+0000 7fae6fe00640 1 -- 172.21.15.52:0/2642425751 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae5800bc20 con 0x7fae700b5a30 2024-03-12T17:08:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.908+0000 7fae74e00640 1 --2- 172.21.15.52:0/2642425751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae580047c0 0x7fae58004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.909+0000 7fae75800640 1 --2- 172.21.15.52:0/2642425751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae580039a0 0x7fae58003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.909+0000 7fae76200640 1 --2- 172.21.15.52:0/2642425751 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fae700b5a30 0x7fae58005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.909+0000 7fae74e00640 1 -- 172.21.15.52:0/2642425751 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fae700b5a30 msgr2=0x7fae58005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.909+0000 7fae74e00640 1 --2- 172.21.15.52:0/2642425751 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fae700b5a30 0x7fae58005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.909+0000 7fae74e00640 1 -- 172.21.15.52:0/2642425751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae580039a0 msgr2=0x7fae58003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.909+0000 7fae74e00640 1 --2- 172.21.15.52:0/2642425751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae580039a0 0x7fae58003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:14.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.909+0000 7fae74e00640 1 -- 172.21.15.52:0/2642425751 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae58003520 con 0x7fae580047c0 2024-03-12T17:08:14.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.910+0000 7fae74e00640 1 --2- 172.21.15.52:0/2642425751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae580047c0 0x7fae58004c40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fae5c009a90 tx=0x7fae580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=160704b8f57260b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:14.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.910+0000 7fae6fe00640 1 -- 172.21.15.52:0/2642425751 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5c02c680 con 0x7fae580047c0 2024-03-12T17:08:14.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.910+0000 7fae6fe00640 1 -- 172.21.15.52:0/2642425751 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae5c02c7e0 con 0x7fae580047c0 2024-03-12T17:08:14.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.910+0000 7fae6fe00640 1 -- 172.21.15.52:0/2642425751 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5c02cb70 con 0x7fae580047c0 2024-03-12T17:08:14.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.911+0000 7fae78200640 1 -- 172.21.15.52:0/2642425751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae580047c0 msgr2=0x7fae58004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:14.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.911+0000 7fae78200640 1 --2- 172.21.15.52:0/2642425751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae580047c0 0x7fae58004c40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fae5c009a90 tx=0x7fae580057f0 comp rx=0 tx=0).stop 2024-03-12T17:08:14.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.911+0000 7fae76200640 1 -- 172.21.15.52:0/2642425751 reap_dead start 2024-03-12T17:08:14.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.911+0000 7fae78200640 1 -- 172.21.15.52:0/2642425751 shutdown_connections 2024-03-12T17:08:14.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.911+0000 7fae78200640 1 -- 172.21.15.52:0/2642425751 >> 172.21.15.52:0/2642425751 conn(0x7fae700cf350 msgr2=0x7fae700cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:14.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.911+0000 7fae78200640 1 -- 172.21.15.52:0/2642425751 shutdown_connections 2024-03-12T17:08:14.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.911+0000 7fae78200640 1 -- 172.21.15.52:0/2642425751 wait complete. 2024-03-12T17:08:14.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.913+0000 7fae78200640 1 Processor -- start 2024-03-12T17:08:14.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.913+0000 7fae78200640 1 -- start start 2024-03-12T17:08:14.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.914+0000 7fae78200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae580047c0 0x7fae700b1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:14.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.914+0000 7fae78200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fae700c8af0 0x7fae701f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:14.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.914+0000 7fae78200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae700ca9f0 0x7fae700caea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:14.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.914+0000 7fae78200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae700c9810 con 0x7fae580047c0 2024-03-12T17:08:14.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.914+0000 7fae78200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae700cb3e0 con 0x7fae700ca9f0 2024-03-12T17:08:14.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.914+0000 7fae78200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae700b1550 con 0x7fae700c8af0 2024-03-12T17:08:14.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.914+0000 7fae75800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae580047c0 0x7fae700b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:14.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.914+0000 7fae75800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae580047c0 0x7fae700b1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36892/0 (socket says 172.21.15.52:36892) 2024-03-12T17:08:14.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.914+0000 7fae75800640 1 -- 172.21.15.52:0/1394037090 learned_addr learned my addr 172.21.15.52:0/1394037090 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:14.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.914+0000 7fae74e00640 1 --2- 172.21.15.52:0/1394037090 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fae700c8af0 0x7fae701f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:14.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.914+0000 7fae76200640 1 --2- 172.21.15.52:0/1394037090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae700ca9f0 0x7fae700caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:14.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.915+0000 7fae75800640 1 -- 172.21.15.52:0/1394037090 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fae700c8af0 msgr2=0x7fae701f1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:14.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.915+0000 7fae75800640 1 --2- 172.21.15.52:0/1394037090 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fae700c8af0 0x7fae701f1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:14.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.915+0000 7fae75800640 1 -- 172.21.15.52:0/1394037090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae700ca9f0 msgr2=0x7fae700caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:14.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.915+0000 7fae75800640 1 --2- 172.21.15.52:0/1394037090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae700ca9f0 0x7fae700caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:14.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.915+0000 7fae75800640 1 -- 172.21.15.52:0/1394037090 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae70156cd0 con 0x7fae580047c0 2024-03-12T17:08:14.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.915+0000 7fae76200640 1 --2- 172.21.15.52:0/1394037090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae700ca9f0 0x7fae700caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:08:14.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.915+0000 7fae75800640 1 --2- 172.21.15.52:0/1394037090 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae580047c0 0x7fae700b1010 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fae6400e830 tx=0x7fae6400ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:14.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.915+0000 7fae6d600640 1 -- 172.21.15.52:0/1394037090 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6400eee0 con 0x7fae580047c0 2024-03-12T17:08:14.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.915+0000 7fae6d600640 1 -- 172.21.15.52:0/1394037090 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae6400c870 con 0x7fae580047c0 2024-03-12T17:08:14.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.915+0000 7fae6d600640 1 -- 172.21.15.52:0/1394037090 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6400cc20 con 0x7fae580047c0 2024-03-12T17:08:14.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.915+0000 7fae78200640 1 -- 172.21.15.52:0/1394037090 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae701f20c0 con 0x7fae580047c0 2024-03-12T17:08:14.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.915+0000 7fae78200640 1 -- 172.21.15.52:0/1394037090 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae700c9250 con 0x7fae580047c0 2024-03-12T17:08:14.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.917+0000 7fae78200640 1 -- 172.21.15.52:0/1394037090 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae70156aa0 con 0x7fae580047c0 2024-03-12T17:08:14.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.917+0000 7fae6d600640 1 -- 172.21.15.52:0/1394037090 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fae64013420 con 0x7fae580047c0 2024-03-12T17:08:14.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.918+0000 7fae6d600640 1 --2- 172.21.15.52:0/1394037090 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fae5006ac30 0x7fae5006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:14.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.918+0000 7fae6d600640 1 -- 172.21.15.52:0/1394037090 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fae640891d0 con 0x7fae580047c0 2024-03-12T17:08:14.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.919+0000 7fae74e00640 1 --2- 172.21.15.52:0/1394037090 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fae5006ac30 0x7fae5006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:14.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.920+0000 7fae74e00640 1 --2- 172.21.15.52:0/1394037090 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fae5006ac30 0x7fae5006d0f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fae5c036000 tx=0x7fae5c030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:14.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:14.923+0000 7fae6d600640 1 -- 172.21.15.52:0/1394037090 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fae70156aa0 con 0x7fae580047c0 2024-03-12T17:08:15.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.118+0000 7fae78200640 1 -- 172.21.15.52:0/1394037090 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae70156aa0 con 0x7fae580047c0 2024-03-12T17:08:15.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.119+0000 7fae6d600640 1 -- 172.21.15.52:0/1394037090 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16306 (secure 0 0 0) 0x7fae70156aa0 con 0x7fae580047c0 2024-03-12T17:08:15.114 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:08:15.114 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":40,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:08:11.683597+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T17:06:19.218543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-12T17:08:09.114678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-12T17:08:09.691941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.733031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.775178+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.591522+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.820838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.829252+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.582481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:08:15.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.122+0000 7fae78200640 1 -- 172.21.15.52:0/1394037090 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fae5006ac30 msgr2=0x7fae5006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:15.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.122+0000 7fae78200640 1 --2- 172.21.15.52:0/1394037090 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fae5006ac30 0x7fae5006d0f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fae5c036000 tx=0x7fae5c030000 comp rx=0 tx=0).stop 2024-03-12T17:08:15.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.122+0000 7fae78200640 1 -- 172.21.15.52:0/1394037090 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae580047c0 msgr2=0x7fae700b1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:15.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.122+0000 7fae78200640 1 --2- 172.21.15.52:0/1394037090 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae580047c0 0x7fae700b1010 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fae6400e830 tx=0x7fae6400ecf0 comp rx=0 tx=0).stop 2024-03-12T17:08:15.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.122+0000 7fae78200640 1 -- 172.21.15.52:0/1394037090 shutdown_connections 2024-03-12T17:08:15.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.122+0000 7fae78200640 1 --2- 172.21.15.52:0/1394037090 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fae5006ac30 0x7fae5006d0f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:15.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.122+0000 7fae78200640 1 --2- 172.21.15.52:0/1394037090 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fae700c8af0 0x7fae701f1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:15.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.122+0000 7fae78200640 1 --2- 172.21.15.52:0/1394037090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae700ca9f0 0x7fae700caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:15.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.122+0000 7fae78200640 1 --2- 172.21.15.52:0/1394037090 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae580047c0 0x7fae700b1010 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:15.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.122+0000 7fae78200640 1 -- 172.21.15.52:0/1394037090 >> 172.21.15.52:0/1394037090 conn(0x7fae700cf350 msgr2=0x7fae700b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:15.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.122+0000 7fae78200640 1 -- 172.21.15.52:0/1394037090 shutdown_connections 2024-03-12T17:08:15.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.122+0000 7fae78200640 1 -- 172.21.15.52:0/1394037090 wait complete. 2024-03-12T17:08:15.134 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-12T17:08:15.134 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-12T17:08:15.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.355+0000 7f542b600640 1 Processor -- start 2024-03-12T17:08:15.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.355+0000 7f542b600640 1 -- start start 2024-03-12T17:08:15.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.357+0000 7f542b600640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f54240b37f0 0x7f54240cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:15.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.357+0000 7f542b600640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f54240ccec0 0x7f542406c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:15.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.357+0000 7f542b600640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f542406da70 con 0x7f542414aa40 2024-03-12T17:08:15.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.357+0000 7f542b600640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f542406d7d0 con 0x7f54240b37f0 2024-03-12T17:08:15.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.357+0000 7f542b600640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54240b1d00 con 0x7f54240ccec0 2024-03-12T17:08:15.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.357+0000 7f5423e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f54240b37f0 0x7f54240cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:15.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.357+0000 7f5423e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f54240b37f0 0x7f54240cd7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36904/0 (socket says 172.21.15.52:36904) 2024-03-12T17:08:15.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.357+0000 7f5423e00640 1 -- 172.21.15.52:0/3424539021 learned_addr learned my addr 172.21.15.52:0/3424539021 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:15.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.358+0000 7f5429600640 1 --2- 172.21.15.52:0/3424539021 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f54240ccec0 0x7f542406c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:15.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.358+0000 7f5423e00640 1 -- 172.21.15.52:0/3424539021 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f54240ccec0 msgr2=0x7f542406c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:15.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.358+0000 7f5423e00640 1 --2- 172.21.15.52:0/3424539021 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f54240ccec0 0x7f542406c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:15.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.358+0000 7f5423e00640 1 -- 172.21.15.52:0/3424539021 >> v1:172.21.15.102:6789/0 conn(0x7f542414aa40 legacy=0x7f54240cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:15.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.358+0000 7f5423e00640 1 -- 172.21.15.52:0/3424539021 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54240b1fe0 con 0x7f54240b37f0 2024-03-12T17:08:15.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.359+0000 7f5423e00640 1 --2- 172.21.15.52:0/3424539021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54240b37f0 0x7f54240cd7a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5410009870 tx=0x7f541002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf639946ea3d08be server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:15.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.359+0000 7f5423400640 1 -- 172.21.15.52:0/3424539021 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5410031640 con 0x7f54240b37f0 2024-03-12T17:08:15.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.360+0000 7f5423400640 1 -- 172.21.15.52:0/3424539021 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54100317a0 con 0x7f54240b37f0 2024-03-12T17:08:15.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.360+0000 7f5423400640 1 -- 172.21.15.52:0/3424539021 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5410031b30 con 0x7f54240b37f0 2024-03-12T17:08:15.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.360+0000 7f542b600640 1 -- 172.21.15.52:0/3424539021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54240b37f0 msgr2=0x7f54240cd7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:15.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.360+0000 7f542b600640 1 --2- 172.21.15.52:0/3424539021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54240b37f0 0x7f54240cd7a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5410009870 tx=0x7f541002b8f0 comp rx=0 tx=0).stop 2024-03-12T17:08:15.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.360+0000 7f542b600640 1 -- 172.21.15.52:0/3424539021 shutdown_connections 2024-03-12T17:08:15.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.360+0000 7f542b600640 1 --2- 172.21.15.52:0/3424539021 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f54240ccec0 0x7f542406c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:15.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.360+0000 7f542b600640 1 --2- 172.21.15.52:0/3424539021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54240b37f0 0x7f54240cd7a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:15.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.360+0000 7f542b600640 1 -- 172.21.15.52:0/3424539021 >> 172.21.15.52:0/3424539021 conn(0x7f542406a3d0 msgr2=0x7f54240b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:15.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.361+0000 7f542b600640 1 -- 172.21.15.52:0/3424539021 shutdown_connections 2024-03-12T17:08:15.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.361+0000 7f542b600640 1 -- 172.21.15.52:0/3424539021 wait complete. 2024-03-12T17:08:15.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.362+0000 7f542b600640 1 Processor -- start 2024-03-12T17:08:15.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.363+0000 7f542b600640 1 -- start start 2024-03-12T17:08:15.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.364+0000 7f542b600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54240b37f0 0x7f54241535f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:15.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.364+0000 7f542b600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54240ccec0 0x7f5424153b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:15.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.364+0000 7f542b600640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f542414aa40 0x7f5424154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:15.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.364+0000 7f542b600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54241545b0 con 0x7f54240b37f0 2024-03-12T17:08:15.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.364+0000 7f542b600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5424142e40 con 0x7f54240ccec0 2024-03-12T17:08:15.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.364+0000 7f542b600640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542413eea0 con 0x7f542414aa40 2024-03-12T17:08:15.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.364+0000 7f5428c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54240b37f0 0x7f54241535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:15.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.364+0000 7f5428c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54240b37f0 0x7f54241535f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36914/0 (socket says 172.21.15.52:36914) 2024-03-12T17:08:15.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.364+0000 7f5428c00640 1 -- 172.21.15.52:0/1762993925 learned_addr learned my addr 172.21.15.52:0/1762993925 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:15.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.364+0000 7f5429600640 1 --2- 172.21.15.52:0/1762993925 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f542414aa40 0x7f5424154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:15.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.364+0000 7f5423e00640 1 --2- 172.21.15.52:0/1762993925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54240ccec0 0x7f5424153b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:15.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.364+0000 7f5428c00640 1 -- 172.21.15.52:0/1762993925 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f542414aa40 msgr2=0x7f5424154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:15.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.364+0000 7f5428c00640 1 --2- 172.21.15.52:0/1762993925 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f542414aa40 0x7f5424154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:15.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.364+0000 7f5428c00640 1 -- 172.21.15.52:0/1762993925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54240ccec0 msgr2=0x7f5424153b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:15.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.364+0000 7f5428c00640 1 --2- 172.21.15.52:0/1762993925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54240ccec0 0x7f5424153b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:15.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.364+0000 7f5428c00640 1 -- 172.21.15.52:0/1762993925 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54100094d0 con 0x7f54240b37f0 2024-03-12T17:08:15.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.364+0000 7f5429600640 1 --2- 172.21.15.52:0/1762993925 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f542414aa40 0x7f5424154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:08:15.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.364+0000 7f5423e00640 1 --2- 172.21.15.52:0/1762993925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54240ccec0 0x7f5424153b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:08:15.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.365+0000 7f5428c00640 1 --2- 172.21.15.52:0/1762993925 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54240b37f0 0x7f54241535f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f541800b3f0 tx=0x7f541800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:15.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.365+0000 7f5420c00640 1 -- 172.21.15.52:0/1762993925 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5418011830 con 0x7f54240b37f0 2024-03-12T17:08:15.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.365+0000 7f5420c00640 1 -- 172.21.15.52:0/1762993925 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5418011990 con 0x7f54240b37f0 2024-03-12T17:08:15.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.365+0000 7f542b600640 1 -- 172.21.15.52:0/1762993925 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f542413f180 con 0x7f54240b37f0 2024-03-12T17:08:15.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.365+0000 7f5420c00640 1 -- 172.21.15.52:0/1762993925 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5418011d40 con 0x7f54240b37f0 2024-03-12T17:08:15.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.365+0000 7f542b600640 1 -- 172.21.15.52:0/1762993925 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f542413f460 con 0x7f54240b37f0 2024-03-12T17:08:15.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.367+0000 7f542b600640 1 -- 172.21.15.52:0/1762993925 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5424141e10 con 0x7f54240b37f0 2024-03-12T17:08:15.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.367+0000 7f5420c00640 1 -- 172.21.15.52:0/1762993925 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f541801a400 con 0x7f54240b37f0 2024-03-12T17:08:15.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.368+0000 7f5420c00640 1 --2- 172.21.15.52:0/1762993925 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f540406ac30 0x7f540406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:15.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.368+0000 7f5420c00640 1 -- 172.21.15.52:0/1762993925 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5418089430 con 0x7f54240b37f0 2024-03-12T17:08:15.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.369+0000 7f5423e00640 1 --2- 172.21.15.52:0/1762993925 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f540406ac30 0x7f540406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:15.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.370+0000 7f5423e00640 1 --2- 172.21.15.52:0/1762993925 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f540406ac30 0x7f540406d0f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5410036000 tx=0x7f541003c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:15.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.372+0000 7f5420c00640 1 -- 172.21.15.52:0/1762993925 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f5424141e10 con 0x7f54240b37f0 2024-03-12T17:08:15.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.562+0000 7f542b600640 1 -- 172.21.15.52:0/1762993925 --> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5424141e10 con 0x7f540406ac30 2024-03-12T17:08:15.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.588+0000 7f5420c00640 1 -- 172.21.15.52:0/1762993925 <== mgr.4109 v2:172.21.15.102:6832/36622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287636 (secure 0 0 0) 0x7f5424141e10 con 0x7f540406ac30 2024-03-12T17:08:15.584 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:08:15.588 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-03-12T17:08:15.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.595+0000 7f542b600640 1 -- 172.21.15.52:0/1762993925 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f540406ac30 msgr2=0x7f540406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:15.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.595+0000 7f542b600640 1 --2- 172.21.15.52:0/1762993925 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f540406ac30 0x7f540406d0f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5410036000 tx=0x7f541003c040 comp rx=0 tx=0).stop 2024-03-12T17:08:15.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.595+0000 7f542b600640 1 -- 172.21.15.52:0/1762993925 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54240b37f0 msgr2=0x7f54241535f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:15.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.595+0000 7f542b600640 1 --2- 172.21.15.52:0/1762993925 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54240b37f0 0x7f54241535f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f541800b3f0 tx=0x7f541800b8b0 comp rx=0 tx=0).stop 2024-03-12T17:08:15.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.596+0000 7f542b600640 1 -- 172.21.15.52:0/1762993925 shutdown_connections 2024-03-12T17:08:15.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.596+0000 7f542b600640 1 --2- 172.21.15.52:0/1762993925 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f540406ac30 0x7f540406d0f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:15.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.596+0000 7f542b600640 1 --2- 172.21.15.52:0/1762993925 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f542414aa40 0x7f5424154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:15.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.596+0000 7f542b600640 1 --2- 172.21.15.52:0/1762993925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54240ccec0 0x7f5424153b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:15.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.596+0000 7f542b600640 1 --2- 172.21.15.52:0/1762993925 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54240b37f0 0x7f54241535f0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:15.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.596+0000 7f542b600640 1 -- 172.21.15.52:0/1762993925 >> 172.21.15.52:0/1762993925 conn(0x7f542406a3d0 msgr2=0x7f54240cc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:15.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.596+0000 7f542b600640 1 -- 172.21.15.52:0/1762993925 shutdown_connections 2024-03-12T17:08:15.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.596+0000 7f542b600640 1 -- 172.21.15.52:0/1762993925 wait complete. 2024-03-12T17:08:15.609 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2024-03-12T17:08:15.189782+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":121,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237424,"kb_used_data":4816,"kb_used_omap":13,"kb_used_meta":232434,"kb_avail":754737296,"statfs":{"total":773094113280,"available":772850991104,"internally_reserved":0,"allocated":4931584,"data_stored":2762444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":238012504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001867"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.691460+0000","last_change":"2024-03-12T17:08:10.705329+0000","last_active":"2024-03-12T17:08:11.691460+0000","last_peered":"2024-03-12T17:08:11.691460+0000","last_clean":"2024-03-12T17:08:11.691460+0000","last_became_active":"2024-03-12T17:08:10.704448+0000","last_became_peered":"2024-03-12T17:08:10.704448+0000","last_unstale":"2024-03-12T17:08:11.691460+0000","last_undegraded":"2024-03-12T17:08:11.691460+0000","last_fullsized":"2024-03-12T17:08:11.691460+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:26:50.546066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.690621+0000","last_change":"2024-03-12T17:08:10.706543+0000","last_active":"2024-03-12T17:08:11.690621+0000","last_peered":"2024-03-12T17:08:11.690621+0000","last_clean":"2024-03-12T17:08:11.690621+0000","last_became_active":"2024-03-12T17:08:10.706231+0000","last_became_peered":"2024-03-12T17:08:10.706231+0000","last_unstale":"2024-03-12T17:08:11.690621+0000","last_undegraded":"2024-03-12T17:08:11.690621+0000","last_fullsized":"2024-03-12T17:08:11.690621+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:51:16.461818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.898295+0000","last_change":"2024-03-12T17:08:10.708889+0000","last_active":"2024-03-12T17:08:11.898295+0000","last_peered":"2024-03-12T17:08:11.898295+0000","last_clean":"2024-03-12T17:08:11.898295+0000","last_became_active":"2024-03-12T17:08:10.708618+0000","last_became_peered":"2024-03-12T17:08:10.708618+0000","last_unstale":"2024-03-12T17:08:11.898295+0000","last_undegraded":"2024-03-12T17:08:11.898295+0000","last_fullsized":"2024-03-12T17:08:11.898295+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:17:24.644009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.479806+0000","last_change":"2024-03-12T17:08:10.730011+0000","last_active":"2024-03-12T17:08:12.479806+0000","last_peered":"2024-03-12T17:08:12.479806+0000","last_clean":"2024-03-12T17:08:12.479806+0000","last_became_active":"2024-03-12T17:08:10.729713+0000","last_became_peered":"2024-03-12T17:08:10.729713+0000","last_unstale":"2024-03-12T17:08:12.479806+0000","last_undegraded":"2024-03-12T17:08:12.479806+0000","last_fullsized":"2024-03-12T17:08:12.479806+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:06:32.038325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.479156+0000","last_change":"2024-03-12T17:08:10.730105+0000","last_active":"2024-03-12T17:08:12.479156+0000","last_peered":"2024-03-12T17:08:12.479156+0000","last_clean":"2024-03-12T17:08:12.479156+0000","last_became_active":"2024-03-12T17:08:10.729706+0000","last_became_peered":"2024-03-12T17:08:10.729706+0000","last_unstale":"2024-03-12T17:08:12.479156+0000","last_undegraded":"2024-03-12T17:08:12.479156+0000","last_fullsized":"2024-03-12T17:08:12.479156+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T02:54:29.108633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.997336+0000","last_change":"2024-03-12T17:08:10.705015+0000","last_active":"2024-03-12T17:08:12.997336+0000","last_peered":"2024-03-12T17:08:12.997336+0000","last_clean":"2024-03-12T17:08:12.997336+0000","last_became_active":"2024-03-12T17:08:10.703748+0000","last_became_peered":"2024-03-12T17:08:10.703748+0000","last_unstale":"2024-03-12T17:08:12.997336+0000","last_undegraded":"2024-03-12T17:08:12.997336+0000","last_fullsized":"2024-03-12T17:08:12.997336+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:23:31.014856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.695191+0000","last_change":"2024-03-12T17:08:10.711451+0000","last_active":"2024-03-12T17:08:11.695191+0000","last_peered":"2024-03-12T17:08:11.695191+0000","last_clean":"2024-03-12T17:08:11.695191+0000","last_became_active":"2024-03-12T17:08:10.711089+0000","last_became_peered":"2024-03-12T17:08:10.711089+0000","last_unstale":"2024-03-12T17:08:11.695191+0000","last_undegraded":"2024-03-12T17:08:11.695191+0000","last_fullsized":"2024-03-12T17:08:11.695191+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:16:52.065023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.690284+0000","last_change":"2024-03-12T17:08:10.705241+0000","last_active":"2024-03-12T17:08:11.690284+0000","last_peered":"2024-03-12T17:08:11.690284+0000","last_clean":"2024-03-12T17:08:11.690284+0000","last_became_active":"2024-03-12T17:08:10.704955+0000","last_became_peered":"2024-03-12T17:08:10.704955+0000","last_unstale":"2024-03-12T17:08:11.690284+0000","last_undegraded":"2024-03-12T17:08:11.690284+0000","last_fullsized":"2024-03-12T17:08:11.690284+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:49:09.840848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.898509+0000","last_change":"2024-03-12T17:08:10.719672+0000","last_active":"2024-03-12T17:08:11.898509+0000","last_peered":"2024-03-12T17:08:11.898509+0000","last_clean":"2024-03-12T17:08:11.898509+0000","last_became_active":"2024-03-12T17:08:10.719435+0000","last_became_peered":"2024-03-12T17:08:10.719435+0000","last_unstale":"2024-03-12T17:08:11.898509+0000","last_undegraded":"2024-03-12T17:08:11.898509+0000","last_fullsized":"2024-03-12T17:08:11.898509+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:31:56.609569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.696347+0000","last_change":"2024-03-12T17:08:10.737543+0000","last_active":"2024-03-12T17:08:11.696347+0000","last_peered":"2024-03-12T17:08:11.696347+0000","last_clean":"2024-03-12T17:08:11.696347+0000","last_became_active":"2024-03-12T17:08:10.737287+0000","last_became_peered":"2024-03-12T17:08:10.737287+0000","last_unstale":"2024-03-12T17:08:11.696347+0000","last_undegraded":"2024-03-12T17:08:11.696347+0000","last_fullsized":"2024-03-12T17:08:11.696347+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:23:32.915382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.695546+0000","last_change":"2024-03-12T17:08:10.732923+0000","last_active":"2024-03-12T17:08:11.695546+0000","last_peered":"2024-03-12T17:08:11.695546+0000","last_clean":"2024-03-12T17:08:11.695546+0000","last_became_active":"2024-03-12T17:08:10.732654+0000","last_became_peered":"2024-03-12T17:08:10.732654+0000","last_unstale":"2024-03-12T17:08:11.695546+0000","last_undegraded":"2024-03-12T17:08:11.695546+0000","last_fullsized":"2024-03-12T17:08:11.695546+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:31:56.609569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.898085+0000","last_change":"2024-03-12T17:08:10.715031+0000","last_active":"2024-03-12T17:08:11.898085+0000","last_peered":"2024-03-12T17:08:11.898085+0000","last_clean":"2024-03-12T17:08:11.898085+0000","last_became_active":"2024-03-12T17:08:10.714743+0000","last_became_peered":"2024-03-12T17:08:10.714743+0000","last_unstale":"2024-03-12T17:08:11.898085+0000","last_undegraded":"2024-03-12T17:08:11.898085+0000","last_fullsized":"2024-03-12T17:08:11.898085+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:23:31.014856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.694233+0000","last_change":"2024-03-12T17:08:10.730025+0000","last_active":"2024-03-12T17:08:11.694233+0000","last_peered":"2024-03-12T17:08:11.694233+0000","last_clean":"2024-03-12T17:08:11.694233+0000","last_became_active":"2024-03-12T17:08:10.729354+0000","last_became_peered":"2024-03-12T17:08:10.729354+0000","last_unstale":"2024-03-12T17:08:11.694233+0000","last_undegraded":"2024-03-12T17:08:11.694233+0000","last_fullsized":"2024-03-12T17:08:11.694233+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:13:11.256122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.694079+0000","last_change":"2024-03-12T17:08:10.722267+0000","last_active":"2024-03-12T17:08:11.694079+0000","last_peered":"2024-03-12T17:08:11.694079+0000","last_clean":"2024-03-12T17:08:11.694079+0000","last_became_active":"2024-03-12T17:08:10.721988+0000","last_became_peered":"2024-03-12T17:08:10.721988+0000","last_unstale":"2024-03-12T17:08:11.694079+0000","last_undegraded":"2024-03-12T17:08:11.694079+0000","last_fullsized":"2024-03-12T17:08:11.694079+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:51:16.461818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.696151+0000","last_change":"2024-03-12T17:08:10.713034+0000","last_active":"2024-03-12T17:08:11.696151+0000","last_peered":"2024-03-12T17:08:11.696151+0000","last_clean":"2024-03-12T17:08:11.696151+0000","last_became_active":"2024-03-12T17:08:10.712704+0000","last_became_peered":"2024-03-12T17:08:10.712704+0000","last_unstale":"2024-03-12T17:08:11.696151+0000","last_undegraded":"2024-03-12T17:08:11.696151+0000","last_fullsized":"2024-03-12T17:08:11.696151+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T02:41:04.330660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.998095+0000","last_change":"2024-03-12T17:08:10.728808+0000","last_active":"2024-03-12T17:08:12.998095+0000","last_peered":"2024-03-12T17:08:12.998095+0000","last_clean":"2024-03-12T17:08:12.998095+0000","last_became_active":"2024-03-12T17:08:10.728334+0000","last_became_peered":"2024-03-12T17:08:10.728334+0000","last_unstale":"2024-03-12T17:08:12.998095+0000","last_undegraded":"2024-03-12T17:08:12.998095+0000","last_fullsized":"2024-03-12T17:08:12.998095+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:23:31.014856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.998803+0000","last_change":"2024-03-12T17:08:10.734394+0000","last_active":"2024-03-12T17:08:12.998803+0000","last_peered":"2024-03-12T17:08:12.998803+0000","last_clean":"2024-03-12T17:08:12.998803+0000","last_became_active":"2024-03-12T17:08:10.733323+0000","last_became_peered":"2024-03-12T17:08:10.733323+0000","last_unstale":"2024-03-12T17:08:12.998803+0000","last_undegraded":"2024-03-12T17:08:12.998803+0000","last_fullsized":"2024-03-12T17:08:12.998803+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T02:41:04.330660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.834163+0000","last_change":"2024-03-12T17:08:10.730315+0000","last_active":"2024-03-12T17:08:11.834163+0000","last_peered":"2024-03-12T17:08:11.834163+0000","last_clean":"2024-03-12T17:08:11.834163+0000","last_became_active":"2024-03-12T17:08:10.729938+0000","last_became_peered":"2024-03-12T17:08:10.729938+0000","last_unstale":"2024-03-12T17:08:11.834163+0000","last_undegraded":"2024-03-12T17:08:11.834163+0000","last_fullsized":"2024-03-12T17:08:11.834163+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:17:24.644009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.479981+0000","last_change":"2024-03-12T17:08:10.707305+0000","last_active":"2024-03-12T17:08:12.479981+0000","last_peered":"2024-03-12T17:08:12.479981+0000","last_clean":"2024-03-12T17:08:12.479981+0000","last_became_active":"2024-03-12T17:08:10.707029+0000","last_became_peered":"2024-03-12T17:08:10.707029+0000","last_unstale":"2024-03-12T17:08:12.479981+0000","last_undegraded":"2024-03-12T17:08:12.479981+0000","last_fullsized":"2024-03-12T17:08:12.479981+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:22:08.884650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.833904+0000","last_change":"2024-03-12T17:08:10.731158+0000","last_active":"2024-03-12T17:08:11.833904+0000","last_peered":"2024-03-12T17:08:11.833904+0000","last_clean":"2024-03-12T17:08:11.833904+0000","last_became_active":"2024-03-12T17:08:10.730829+0000","last_became_peered":"2024-03-12T17:08:10.730829+0000","last_unstale":"2024-03-12T17:08:11.833904+0000","last_undegraded":"2024-03-12T17:08:11.833904+0000","last_fullsized":"2024-03-12T17:08:11.833904+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:31:56.609569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.694512+0000","last_change":"2024-03-12T17:08:10.714510+0000","last_active":"2024-03-12T17:08:11.694512+0000","last_peered":"2024-03-12T17:08:11.694512+0000","last_clean":"2024-03-12T17:08:11.694512+0000","last_became_active":"2024-03-12T17:08:10.713860+0000","last_became_peered":"2024-03-12T17:08:10.713860+0000","last_unstale":"2024-03-12T17:08:11.694512+0000","last_undegraded":"2024-03-12T17:08:11.694512+0000","last_fullsized":"2024-03-12T17:08:11.694512+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:33:01.223233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.695741+0000","last_change":"2024-03-12T17:08:10.722283+0000","last_active":"2024-03-12T17:08:11.695741+0000","last_peered":"2024-03-12T17:08:11.695741+0000","last_clean":"2024-03-12T17:08:11.695741+0000","last_became_active":"2024-03-12T17:08:10.721775+0000","last_became_peered":"2024-03-12T17:08:10.721775+0000","last_unstale":"2024-03-12T17:08:11.695741+0000","last_undegraded":"2024-03-12T17:08:11.695741+0000","last_fullsized":"2024-03-12T17:08:11.695741+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T02:41:04.330660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.998275+0000","last_change":"2024-03-12T17:08:10.735342+0000","last_active":"2024-03-12T17:08:12.998275+0000","last_peered":"2024-03-12T17:08:12.998275+0000","last_clean":"2024-03-12T17:08:12.998275+0000","last_became_active":"2024-03-12T17:08:10.732932+0000","last_became_peered":"2024-03-12T17:08:10.732932+0000","last_unstale":"2024-03-12T17:08:12.998275+0000","last_undegraded":"2024-03-12T17:08:12.998275+0000","last_fullsized":"2024-03-12T17:08:12.998275+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:18:33.858639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.479369+0000","last_change":"2024-03-12T17:08:10.735946+0000","last_active":"2024-03-12T17:08:12.479369+0000","last_peered":"2024-03-12T17:08:12.479369+0000","last_clean":"2024-03-12T17:08:12.479369+0000","last_became_active":"2024-03-12T17:08:10.735517+0000","last_became_peered":"2024-03-12T17:08:10.735517+0000","last_unstale":"2024-03-12T17:08:12.479369+0000","last_undegraded":"2024-03-12T17:08:12.479369+0000","last_fullsized":"2024-03-12T17:08:12.479369+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:29:59.176028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.834617+0000","last_change":"2024-03-12T17:08:10.714628+0000","last_active":"2024-03-12T17:08:11.834617+0000","last_peered":"2024-03-12T17:08:11.834617+0000","last_clean":"2024-03-12T17:08:11.834617+0000","last_became_active":"2024-03-12T17:08:10.714287+0000","last_became_peered":"2024-03-12T17:08:10.714287+0000","last_unstale":"2024-03-12T17:08:11.834617+0000","last_undegraded":"2024-03-12T17:08:11.834617+0000","last_fullsized":"2024-03-12T17:08:11.834617+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:23:31.014856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.997652+0000","last_change":"2024-03-12T17:08:10.721517+0000","last_active":"2024-03-12T17:08:12.997652+0000","last_peered":"2024-03-12T17:08:12.997652+0000","last_clean":"2024-03-12T17:08:12.997652+0000","last_became_active":"2024-03-12T17:08:10.721204+0000","last_became_peered":"2024-03-12T17:08:10.721204+0000","last_unstale":"2024-03-12T17:08:12.997652+0000","last_undegraded":"2024-03-12T17:08:12.997652+0000","last_fullsized":"2024-03-12T17:08:12.997652+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:17:24.644009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.834154+0000","last_change":"2024-03-12T17:08:10.712942+0000","last_active":"2024-03-12T17:08:11.834154+0000","last_peered":"2024-03-12T17:08:11.834154+0000","last_clean":"2024-03-12T17:08:11.834154+0000","last_became_active":"2024-03-12T17:08:10.711213+0000","last_became_peered":"2024-03-12T17:08:10.711213+0000","last_unstale":"2024-03-12T17:08:11.834154+0000","last_undegraded":"2024-03-12T17:08:11.834154+0000","last_fullsized":"2024-03-12T17:08:11.834154+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:18:33.858639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.694802+0000","last_change":"2024-03-12T17:08:10.721650+0000","last_active":"2024-03-12T17:08:11.694802+0000","last_peered":"2024-03-12T17:08:11.694802+0000","last_clean":"2024-03-12T17:08:11.694802+0000","last_became_active":"2024-03-12T17:08:10.721313+0000","last_became_peered":"2024-03-12T17:08:10.721313+0000","last_unstale":"2024-03-12T17:08:11.694802+0000","last_undegraded":"2024-03-12T17:08:11.694802+0000","last_fullsized":"2024-03-12T17:08:11.694802+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:39:34.128499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.833498+0000","last_change":"2024-03-12T17:08:10.710889+0000","last_active":"2024-03-12T17:08:11.833498+0000","last_peered":"2024-03-12T17:08:11.833498+0000","last_clean":"2024-03-12T17:08:11.833498+0000","last_became_active":"2024-03-12T17:08:10.710575+0000","last_became_peered":"2024-03-12T17:08:10.710575+0000","last_unstale":"2024-03-12T17:08:11.833498+0000","last_undegraded":"2024-03-12T17:08:11.833498+0000","last_fullsized":"2024-03-12T17:08:11.833498+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:13:11.256122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.833690+0000","last_change":"2024-03-12T17:08:10.704928+0000","last_active":"2024-03-12T17:08:11.833690+0000","last_peered":"2024-03-12T17:08:11.833690+0000","last_clean":"2024-03-12T17:08:11.833690+0000","last_became_active":"2024-03-12T17:08:10.704232+0000","last_became_peered":"2024-03-12T17:08:10.704232+0000","last_unstale":"2024-03-12T17:08:11.833690+0000","last_undegraded":"2024-03-12T17:08:11.833690+0000","last_fullsized":"2024-03-12T17:08:11.833690+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:04:26.911580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.998494+0000","last_change":"2024-03-12T17:08:10.737536+0000","last_active":"2024-03-12T17:08:12.998494+0000","last_peered":"2024-03-12T17:08:12.998494+0000","last_clean":"2024-03-12T17:08:12.998494+0000","last_became_active":"2024-03-12T17:08:10.733851+0000","last_became_peered":"2024-03-12T17:08:10.733851+0000","last_unstale":"2024-03-12T17:08:12.998494+0000","last_undegraded":"2024-03-12T17:08:12.998494+0000","last_fullsized":"2024-03-12T17:08:12.998494+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:09:44.012820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.997267+0000","last_change":"2024-03-12T17:08:10.710798+0000","last_active":"2024-03-12T17:08:12.997267+0000","last_peered":"2024-03-12T17:08:12.997267+0000","last_clean":"2024-03-12T17:08:12.997267+0000","last_became_active":"2024-03-12T17:08:10.710188+0000","last_became_peered":"2024-03-12T17:08:10.710188+0000","last_unstale":"2024-03-12T17:08:12.997267+0000","last_undegraded":"2024-03-12T17:08:12.997267+0000","last_fullsized":"2024-03-12T17:08:12.997267+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:09:44.012820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.998919+0000","last_change":"2024-03-12T17:08:10.739282+0000","last_active":"2024-03-12T17:08:12.998919+0000","last_peered":"2024-03-12T17:08:12.998919+0000","last_clean":"2024-03-12T17:08:12.998919+0000","last_became_active":"2024-03-12T17:08:10.738996+0000","last_became_peered":"2024-03-12T17:08:10.738996+0000","last_unstale":"2024-03-12T17:08:12.998919+0000","last_undegraded":"2024-03-12T17:08:12.998919+0000","last_fullsized":"2024-03-12T17:08:12.998919+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:04:26.911580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.834663+0000","last_change":"2024-03-12T17:08:10.218826+0000","last_active":"2024-03-12T17:08:11.834663+0000","last_peered":"2024-03-12T17:08:11.834663+0000","last_clean":"2024-03-12T17:08:11.834663+0000","last_became_active":"2024-03-12T17:08:10.218560+0000","last_became_peered":"2024-03-12T17:08:10.218560+0000","last_unstale":"2024-03-12T17:08:11.834663+0000","last_undegraded":"2024-03-12T17:08:11.834663+0000","last_fullsized":"2024-03-12T17:08:11.834663+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:25:24.925580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.997750+0000","last_change":"2024-03-12T17:08:10.712062+0000","last_active":"2024-03-12T17:08:12.997750+0000","last_peered":"2024-03-12T17:08:12.997750+0000","last_clean":"2024-03-12T17:08:12.997750+0000","last_became_active":"2024-03-12T17:08:10.710957+0000","last_became_peered":"2024-03-12T17:08:10.710957+0000","last_unstale":"2024-03-12T17:08:12.997750+0000","last_undegraded":"2024-03-12T17:08:12.997750+0000","last_fullsized":"2024-03-12T17:08:12.997750+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:13:11.256122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.898284+0000","last_change":"2024-03-12T17:08:10.209162+0000","last_active":"2024-03-12T17:08:11.898284+0000","last_peered":"2024-03-12T17:08:11.898284+0000","last_clean":"2024-03-12T17:08:11.898284+0000","last_became_active":"2024-03-12T17:08:10.208682+0000","last_became_peered":"2024-03-12T17:08:10.208682+0000","last_unstale":"2024-03-12T17:08:11.898284+0000","last_undegraded":"2024-03-12T17:08:11.898284+0000","last_fullsized":"2024-03-12T17:08:11.898284+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:29:17.220320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.695170+0000","last_change":"2024-03-12T17:08:10.731462+0000","last_active":"2024-03-12T17:08:11.695170+0000","last_peered":"2024-03-12T17:08:11.695170+0000","last_clean":"2024-03-12T17:08:11.695170+0000","last_became_active":"2024-03-12T17:08:10.728086+0000","last_became_peered":"2024-03-12T17:08:10.728086+0000","last_unstale":"2024-03-12T17:08:11.695170+0000","last_undegraded":"2024-03-12T17:08:11.695170+0000","last_fullsized":"2024-03-12T17:08:11.695170+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:56:08.915410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.997826+0000","last_change":"2024-03-12T17:08:10.220415+0000","last_active":"2024-03-12T17:08:12.997826+0000","last_peered":"2024-03-12T17:08:12.997826+0000","last_clean":"2024-03-12T17:08:12.997826+0000","last_became_active":"2024-03-12T17:08:10.219966+0000","last_became_peered":"2024-03-12T17:08:10.219966+0000","last_unstale":"2024-03-12T17:08:12.997826+0000","last_undegraded":"2024-03-12T17:08:12.997826+0000","last_fullsized":"2024-03-12T17:08:12.997826+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:19:39.777696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.691461+0000","last_change":"2024-03-12T17:08:10.719305+0000","last_active":"2024-03-12T17:08:11.691461+0000","last_peered":"2024-03-12T17:08:11.691461+0000","last_clean":"2024-03-12T17:08:11.691461+0000","last_became_active":"2024-03-12T17:08:10.716803+0000","last_became_peered":"2024-03-12T17:08:10.716803+0000","last_unstale":"2024-03-12T17:08:11.691461+0000","last_undegraded":"2024-03-12T17:08:11.691461+0000","last_fullsized":"2024-03-12T17:08:11.691461+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:21:26.723972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.695310+0000","last_change":"2024-03-12T17:08:10.220844+0000","last_active":"2024-03-12T17:08:11.695310+0000","last_peered":"2024-03-12T17:08:11.695310+0000","last_clean":"2024-03-12T17:08:11.695310+0000","last_became_active":"2024-03-12T17:08:10.218208+0000","last_became_peered":"2024-03-12T17:08:10.218208+0000","last_unstale":"2024-03-12T17:08:11.695310+0000","last_undegraded":"2024-03-12T17:08:11.695310+0000","last_fullsized":"2024-03-12T17:08:11.695310+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:06:31.032998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.897669+0000","last_change":"2024-03-12T17:08:10.700350+0000","last_active":"2024-03-12T17:08:11.897669+0000","last_peered":"2024-03-12T17:08:11.897669+0000","last_clean":"2024-03-12T17:08:11.897669+0000","last_became_active":"2024-03-12T17:08:10.700059+0000","last_became_peered":"2024-03-12T17:08:10.700059+0000","last_unstale":"2024-03-12T17:08:11.897669+0000","last_undegraded":"2024-03-12T17:08:11.897669+0000","last_fullsized":"2024-03-12T17:08:11.897669+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:13:11.256122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.694737+0000","last_change":"2024-03-12T17:08:10.214240+0000","last_active":"2024-03-12T17:08:11.694737+0000","last_peered":"2024-03-12T17:08:11.694737+0000","last_clean":"2024-03-12T17:08:11.694737+0000","last_became_active":"2024-03-12T17:08:10.213955+0000","last_became_peered":"2024-03-12T17:08:10.213955+0000","last_unstale":"2024-03-12T17:08:11.694737+0000","last_undegraded":"2024-03-12T17:08:11.694737+0000","last_fullsized":"2024-03-12T17:08:11.694737+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:19:39.777696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.695504+0000","last_change":"2024-03-12T17:08:10.710702+0000","last_active":"2024-03-12T17:08:11.695504+0000","last_peered":"2024-03-12T17:08:11.695504+0000","last_clean":"2024-03-12T17:08:11.695504+0000","last_became_active":"2024-03-12T17:08:10.705875+0000","last_became_peered":"2024-03-12T17:08:10.705875+0000","last_unstale":"2024-03-12T17:08:11.695504+0000","last_undegraded":"2024-03-12T17:08:11.695504+0000","last_fullsized":"2024-03-12T17:08:11.695504+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:27:10.880424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.998824+0000","last_change":"2024-03-12T17:08:10.218076+0000","last_active":"2024-03-12T17:08:12.998824+0000","last_peered":"2024-03-12T17:08:12.998824+0000","last_clean":"2024-03-12T17:08:12.998824+0000","last_became_active":"2024-03-12T17:08:10.217784+0000","last_became_peered":"2024-03-12T17:08:10.217784+0000","last_unstale":"2024-03-12T17:08:12.998824+0000","last_undegraded":"2024-03-12T17:08:12.998824+0000","last_fullsized":"2024-03-12T17:08:12.998824+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:09:20.404327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.696005+0000","last_change":"2024-03-12T17:08:10.723419+0000","last_active":"2024-03-12T17:08:11.696005+0000","last_peered":"2024-03-12T17:08:11.696005+0000","last_clean":"2024-03-12T17:08:11.696005+0000","last_became_active":"2024-03-12T17:08:10.723156+0000","last_became_peered":"2024-03-12T17:08:10.723156+0000","last_unstale":"2024-03-12T17:08:11.696005+0000","last_undegraded":"2024-03-12T17:08:11.696005+0000","last_fullsized":"2024-03-12T17:08:11.696005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:26:50.546066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.997460+0000","last_change":"2024-03-12T17:08:10.220396+0000","last_active":"2024-03-12T17:08:12.997460+0000","last_peered":"2024-03-12T17:08:12.997460+0000","last_clean":"2024-03-12T17:08:12.997460+0000","last_became_active":"2024-03-12T17:08:10.220068+0000","last_became_peered":"2024-03-12T17:08:10.220068+0000","last_unstale":"2024-03-12T17:08:12.997460+0000","last_undegraded":"2024-03-12T17:08:12.997460+0000","last_fullsized":"2024-03-12T17:08:12.997460+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T20:40:15.944188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.690410+0000","last_change":"2024-03-12T17:08:10.728470+0000","last_active":"2024-03-12T17:08:11.690410+0000","last_peered":"2024-03-12T17:08:11.690410+0000","last_clean":"2024-03-12T17:08:11.690410+0000","last_became_active":"2024-03-12T17:08:10.728172+0000","last_became_peered":"2024-03-12T17:08:10.728172+0000","last_unstale":"2024-03-12T17:08:11.690410+0000","last_undegraded":"2024-03-12T17:08:11.690410+0000","last_fullsized":"2024-03-12T17:08:11.690410+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:39:34.128499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.693989+0000","last_change":"2024-03-12T17:08:10.217263+0000","last_active":"2024-03-12T17:08:11.693989+0000","last_peered":"2024-03-12T17:08:11.693989+0000","last_clean":"2024-03-12T17:08:11.693989+0000","last_became_active":"2024-03-12T17:08:10.215937+0000","last_became_peered":"2024-03-12T17:08:10.215937+0000","last_unstale":"2024-03-12T17:08:11.693989+0000","last_undegraded":"2024-03-12T17:08:11.693989+0000","last_fullsized":"2024-03-12T17:08:11.693989+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T02:54:28.103306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.479883+0000","last_change":"2024-03-12T17:08:10.729889+0000","last_active":"2024-03-12T17:08:12.479883+0000","last_peered":"2024-03-12T17:08:12.479883+0000","last_clean":"2024-03-12T17:08:12.479883+0000","last_became_active":"2024-03-12T17:08:10.729600+0000","last_became_peered":"2024-03-12T17:08:10.729600+0000","last_unstale":"2024-03-12T17:08:12.479883+0000","last_undegraded":"2024-03-12T17:08:12.479883+0000","last_fullsized":"2024-03-12T17:08:12.479883+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:34:31.313582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.691641+0000","last_change":"2024-03-12T17:08:10.214927+0000","last_active":"2024-03-12T17:08:11.691641+0000","last_peered":"2024-03-12T17:08:11.691641+0000","last_clean":"2024-03-12T17:08:11.691641+0000","last_became_active":"2024-03-12T17:08:10.214638+0000","last_became_peered":"2024-03-12T17:08:10.214638+0000","last_unstale":"2024-03-12T17:08:11.691641+0000","last_undegraded":"2024-03-12T17:08:11.691641+0000","last_fullsized":"2024-03-12T17:08:11.691641+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:39:57.420626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.999058+0000","last_change":"2024-03-12T17:08:10.734809+0000","last_active":"2024-03-12T17:08:12.999058+0000","last_peered":"2024-03-12T17:08:12.999058+0000","last_clean":"2024-03-12T17:08:12.999058+0000","last_became_active":"2024-03-12T17:08:10.734568+0000","last_became_peered":"2024-03-12T17:08:10.734568+0000","last_unstale":"2024-03-12T17:08:12.999058+0000","last_undegraded":"2024-03-12T17:08:12.999058+0000","last_fullsized":"2024-03-12T17:08:12.999058+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:31:56.609569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.696405+0000","last_change":"2024-03-12T17:08:10.224618+0000","last_active":"2024-03-12T17:08:11.696405+0000","last_peered":"2024-03-12T17:08:11.696405+0000","last_clean":"2024-03-12T17:08:11.696405+0000","last_became_active":"2024-03-12T17:08:10.224322+0000","last_became_peered":"2024-03-12T17:08:10.224322+0000","last_unstale":"2024-03-12T17:08:11.696405+0000","last_undegraded":"2024-03-12T17:08:11.696405+0000","last_fullsized":"2024-03-12T17:08:11.696405+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:21:28.144885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.898032+0000","last_change":"2024-03-12T17:08:10.737489+0000","last_active":"2024-03-12T17:08:11.898032+0000","last_peered":"2024-03-12T17:08:11.898032+0000","last_clean":"2024-03-12T17:08:11.898032+0000","last_became_active":"2024-03-12T17:08:10.737097+0000","last_became_peered":"2024-03-12T17:08:10.737097+0000","last_unstale":"2024-03-12T17:08:11.898032+0000","last_undegraded":"2024-03-12T17:08:11.898032+0000","last_fullsized":"2024-03-12T17:08:11.898032+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:44:36.258866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.695783+0000","last_change":"2024-03-12T17:08:10.223815+0000","last_active":"2024-03-12T17:08:11.695783+0000","last_peered":"2024-03-12T17:08:11.695783+0000","last_clean":"2024-03-12T17:08:11.695783+0000","last_became_active":"2024-03-12T17:08:10.223501+0000","last_became_peered":"2024-03-12T17:08:10.223501+0000","last_unstale":"2024-03-12T17:08:11.695783+0000","last_undegraded":"2024-03-12T17:08:11.695783+0000","last_fullsized":"2024-03-12T17:08:11.695783+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:29:17.220320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.695156+0000","last_change":"2024-03-12T17:08:10.731336+0000","last_active":"2024-03-12T17:08:11.695156+0000","last_peered":"2024-03-12T17:08:11.695156+0000","last_clean":"2024-03-12T17:08:11.695156+0000","last_became_active":"2024-03-12T17:08:10.730543+0000","last_became_peered":"2024-03-12T17:08:10.730543+0000","last_unstale":"2024-03-12T17:08:11.695156+0000","last_undegraded":"2024-03-12T17:08:11.695156+0000","last_fullsized":"2024-03-12T17:08:11.695156+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:27:10.880424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.691196+0000","last_change":"2024-03-12T17:08:10.212100+0000","last_active":"2024-03-12T17:08:11.691196+0000","last_peered":"2024-03-12T17:08:11.691196+0000","last_clean":"2024-03-12T17:08:11.691196+0000","last_became_active":"2024-03-12T17:08:10.209780+0000","last_became_peered":"2024-03-12T17:08:10.209780+0000","last_unstale":"2024-03-12T17:08:11.691196+0000","last_undegraded":"2024-03-12T17:08:11.691196+0000","last_fullsized":"2024-03-12T17:08:11.691196+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:25:31.957988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.897713+0000","last_change":"2024-03-12T17:08:10.708692+0000","last_active":"2024-03-12T17:08:11.897713+0000","last_peered":"2024-03-12T17:08:11.897713+0000","last_clean":"2024-03-12T17:08:11.897713+0000","last_became_active":"2024-03-12T17:08:10.708377+0000","last_became_peered":"2024-03-12T17:08:10.708377+0000","last_unstale":"2024-03-12T17:08:11.897713+0000","last_undegraded":"2024-03-12T17:08:11.897713+0000","last_fullsized":"2024-03-12T17:08:11.897713+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:18:33.858639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.690931+0000","last_change":"2024-03-12T17:08:10.207696+0000","last_active":"2024-03-12T17:08:11.690931+0000","last_peered":"2024-03-12T17:08:11.690931+0000","last_clean":"2024-03-12T17:08:11.690931+0000","last_became_active":"2024-03-12T17:08:10.207049+0000","last_became_peered":"2024-03-12T17:08:10.207049+0000","last_unstale":"2024-03-12T17:08:11.690931+0000","last_undegraded":"2024-03-12T17:08:11.690931+0000","last_fullsized":"2024-03-12T17:08:11.690931+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T02:54:28.103306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.690401+0000","last_change":"2024-03-12T17:08:10.702871+0000","last_active":"2024-03-12T17:08:11.690401+0000","last_peered":"2024-03-12T17:08:11.690401+0000","last_clean":"2024-03-12T17:08:11.690401+0000","last_became_active":"2024-03-12T17:08:10.702594+0000","last_became_peered":"2024-03-12T17:08:10.702594+0000","last_unstale":"2024-03-12T17:08:11.690401+0000","last_undegraded":"2024-03-12T17:08:11.690401+0000","last_fullsized":"2024-03-12T17:08:11.690401+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:27:10.880424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.997569+0000","last_change":"2024-03-12T17:08:10.225233+0000","last_active":"2024-03-12T17:08:12.997569+0000","last_peered":"2024-03-12T17:08:12.997569+0000","last_clean":"2024-03-12T17:08:12.997569+0000","last_became_active":"2024-03-12T17:08:10.224937+0000","last_became_peered":"2024-03-12T17:08:10.224937+0000","last_unstale":"2024-03-12T17:08:12.997569+0000","last_undegraded":"2024-03-12T17:08:12.997569+0000","last_fullsized":"2024-03-12T17:08:12.997569+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:29:17.220320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.998497+0000","last_change":"2024-03-12T17:08:10.728427+0000","last_active":"2024-03-12T17:08:12.998497+0000","last_peered":"2024-03-12T17:08:12.998497+0000","last_clean":"2024-03-12T17:08:12.998497+0000","last_became_active":"2024-03-12T17:08:10.726613+0000","last_became_peered":"2024-03-12T17:08:10.726613+0000","last_unstale":"2024-03-12T17:08:12.998497+0000","last_undegraded":"2024-03-12T17:08:12.998497+0000","last_fullsized":"2024-03-12T17:08:12.998497+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:13:11.256122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.833793+0000","last_change":"2024-03-12T17:08:10.212226+0000","last_active":"2024-03-12T17:08:11.833793+0000","last_peered":"2024-03-12T17:08:11.833793+0000","last_clean":"2024-03-12T17:08:11.833793+0000","last_became_active":"2024-03-12T17:08:10.211929+0000","last_became_peered":"2024-03-12T17:08:10.211929+0000","last_unstale":"2024-03-12T17:08:11.833793+0000","last_undegraded":"2024-03-12T17:08:11.833793+0000","last_fullsized":"2024-03-12T17:08:11.833793+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:22:07.879324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.690710+0000","last_change":"2024-03-12T17:08:10.708949+0000","last_active":"2024-03-12T17:08:11.690710+0000","last_peered":"2024-03-12T17:08:11.690710+0000","last_clean":"2024-03-12T17:08:11.690710+0000","last_became_active":"2024-03-12T17:08:10.708684+0000","last_became_peered":"2024-03-12T17:08:10.708684+0000","last_unstale":"2024-03-12T17:08:11.690710+0000","last_undegraded":"2024-03-12T17:08:11.690710+0000","last_fullsized":"2024-03-12T17:08:11.690710+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:33:01.223233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":19,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.998405+0000","last_change":"2024-03-12T17:08:10.212353+0000","last_active":"2024-03-12T17:08:12.998405+0000","last_peered":"2024-03-12T17:08:12.998405+0000","last_clean":"2024-03-12T17:08:12.998405+0000","last_became_active":"2024-03-12T17:08:10.212069+0000","last_became_peered":"2024-03-12T17:08:10.212069+0000","last_unstale":"2024-03-12T17:08:12.998405+0000","last_undegraded":"2024-03-12T17:08:12.998405+0000","last_fullsized":"2024-03-12T17:08:12.998405+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:52:00.006743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.691140+0000","last_change":"2024-03-12T17:08:10.714647+0000","last_active":"2024-03-12T17:08:11.691140+0000","last_peered":"2024-03-12T17:08:11.691140+0000","last_clean":"2024-03-12T17:08:11.691140+0000","last_became_active":"2024-03-12T17:08:10.713543+0000","last_became_peered":"2024-03-12T17:08:10.713543+0000","last_unstale":"2024-03-12T17:08:11.691140+0000","last_undegraded":"2024-03-12T17:08:11.691140+0000","last_fullsized":"2024-03-12T17:08:11.691140+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:28:01.785461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.696335+0000","last_change":"2024-03-12T17:08:10.222123+0000","last_active":"2024-03-12T17:08:11.696335+0000","last_peered":"2024-03-12T17:08:11.696335+0000","last_clean":"2024-03-12T17:08:11.696335+0000","last_became_active":"2024-03-12T17:08:10.221168+0000","last_became_peered":"2024-03-12T17:08:10.221168+0000","last_unstale":"2024-03-12T17:08:11.696335+0000","last_undegraded":"2024-03-12T17:08:11.696335+0000","last_fullsized":"2024-03-12T17:08:11.696335+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:29:58.170701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.695665+0000","last_change":"2024-03-12T17:08:10.704614+0000","last_active":"2024-03-12T17:08:11.695665+0000","last_peered":"2024-03-12T17:08:11.695665+0000","last_clean":"2024-03-12T17:08:11.695665+0000","last_became_active":"2024-03-12T17:08:10.704243+0000","last_became_peered":"2024-03-12T17:08:10.704243+0000","last_unstale":"2024-03-12T17:08:11.695665+0000","last_undegraded":"2024-03-12T17:08:11.695665+0000","last_fullsized":"2024-03-12T17:08:11.695665+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:51:16.461818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.695431+0000","last_change":"2024-03-12T17:08:10.220809+0000","last_active":"2024-03-12T17:08:11.695431+0000","last_peered":"2024-03-12T17:08:11.695431+0000","last_clean":"2024-03-12T17:08:11.695431+0000","last_became_active":"2024-03-12T17:08:10.220495+0000","last_became_peered":"2024-03-12T17:08:10.220495+0000","last_unstale":"2024-03-12T17:08:11.695431+0000","last_undegraded":"2024-03-12T17:08:11.695431+0000","last_fullsized":"2024-03-12T17:08:11.695431+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:25:31.957988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.694940+0000","last_change":"2024-03-12T17:08:10.732113+0000","last_active":"2024-03-12T17:08:11.694940+0000","last_peered":"2024-03-12T17:08:11.694940+0000","last_clean":"2024-03-12T17:08:11.694940+0000","last_became_active":"2024-03-12T17:08:10.731823+0000","last_became_peered":"2024-03-12T17:08:10.731823+0000","last_unstale":"2024-03-12T17:08:11.694940+0000","last_undegraded":"2024-03-12T17:08:11.694940+0000","last_fullsized":"2024-03-12T17:08:11.694940+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:23:32.915382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.480191+0000","last_change":"2024-03-12T17:08:10.208921+0000","last_active":"2024-03-12T17:08:12.480191+0000","last_peered":"2024-03-12T17:08:12.480191+0000","last_clean":"2024-03-12T17:08:12.480191+0000","last_became_active":"2024-03-12T17:08:10.208638+0000","last_became_peered":"2024-03-12T17:08:10.208638+0000","last_unstale":"2024-03-12T17:08:12.480191+0000","last_undegraded":"2024-03-12T17:08:12.480191+0000","last_fullsized":"2024-03-12T17:08:12.480191+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:39:11.129967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.479915+0000","last_change":"2024-03-12T17:08:10.709839+0000","last_active":"2024-03-12T17:08:12.479915+0000","last_peered":"2024-03-12T17:08:12.479915+0000","last_clean":"2024-03-12T17:08:12.479915+0000","last_became_active":"2024-03-12T17:08:10.709514+0000","last_became_peered":"2024-03-12T17:08:10.709514+0000","last_unstale":"2024-03-12T17:08:12.479915+0000","last_undegraded":"2024-03-12T17:08:12.479915+0000","last_fullsized":"2024-03-12T17:08:12.479915+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:25:25.930906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.997786+0000","last_change":"2024-03-12T17:08:10.219600+0000","last_active":"2024-03-12T17:08:12.997786+0000","last_peered":"2024-03-12T17:08:12.997786+0000","last_clean":"2024-03-12T17:08:12.997786+0000","last_became_active":"2024-03-12T17:08:10.219314+0000","last_became_peered":"2024-03-12T17:08:10.219314+0000","last_unstale":"2024-03-12T17:08:12.997786+0000","last_undegraded":"2024-03-12T17:08:12.997786+0000","last_fullsized":"2024-03-12T17:08:12.997786+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:29:58.170701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.998422+0000","last_change":"2024-03-12T17:08:10.728976+0000","last_active":"2024-03-12T17:08:12.998422+0000","last_peered":"2024-03-12T17:08:12.998422+0000","last_clean":"2024-03-12T17:08:12.998422+0000","last_became_active":"2024-03-12T17:08:10.728612+0000","last_became_peered":"2024-03-12T17:08:10.728612+0000","last_unstale":"2024-03-12T17:08:12.998422+0000","last_undegraded":"2024-03-12T17:08:12.998422+0000","last_fullsized":"2024-03-12T17:08:12.998422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:44:36.258866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.479528+0000","last_change":"2024-03-12T17:08:10.204338+0000","last_active":"2024-03-12T17:08:12.479528+0000","last_peered":"2024-03-12T17:08:12.479528+0000","last_clean":"2024-03-12T17:08:12.479528+0000","last_became_active":"2024-03-12T17:08:10.203258+0000","last_became_peered":"2024-03-12T17:08:10.203258+0000","last_unstale":"2024-03-12T17:08:12.479528+0000","last_undegraded":"2024-03-12T17:08:12.479528+0000","last_fullsized":"2024-03-12T17:08:12.479528+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:43:43.249201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.897868+0000","last_change":"2024-03-12T17:08:10.727663+0000","last_active":"2024-03-12T17:08:11.897868+0000","last_peered":"2024-03-12T17:08:11.897868+0000","last_clean":"2024-03-12T17:08:11.897868+0000","last_became_active":"2024-03-12T17:08:10.727257+0000","last_became_peered":"2024-03-12T17:08:10.727257+0000","last_unstale":"2024-03-12T17:08:11.897868+0000","last_undegraded":"2024-03-12T17:08:11.897868+0000","last_fullsized":"2024-03-12T17:08:11.897868+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:09:44.012820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.479312+0000","last_change":"2024-03-12T17:08:10.217052+0000","last_active":"2024-03-12T17:08:12.479312+0000","last_peered":"2024-03-12T17:08:12.479312+0000","last_clean":"2024-03-12T17:08:12.479312+0000","last_became_active":"2024-03-12T17:08:10.215880+0000","last_became_peered":"2024-03-12T17:08:10.215880+0000","last_unstale":"2024-03-12T17:08:12.479312+0000","last_undegraded":"2024-03-12T17:08:12.479312+0000","last_fullsized":"2024-03-12T17:08:12.479312+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:52:00.006743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.695166+0000","last_change":"2024-03-12T17:08:10.718800+0000","last_active":"2024-03-12T17:08:11.695166+0000","last_peered":"2024-03-12T17:08:11.695166+0000","last_clean":"2024-03-12T17:08:11.695166+0000","last_became_active":"2024-03-12T17:08:10.713494+0000","last_became_peered":"2024-03-12T17:08:10.713494+0000","last_unstale":"2024-03-12T17:08:11.695166+0000","last_undegraded":"2024-03-12T17:08:11.695166+0000","last_fullsized":"2024-03-12T17:08:11.695166+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:56:08.915410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.998534+0000","last_change":"2024-03-12T17:08:10.217017+0000","last_active":"2024-03-12T17:08:12.998534+0000","last_peered":"2024-03-12T17:08:12.998534+0000","last_clean":"2024-03-12T17:08:12.998534+0000","last_became_active":"2024-03-12T17:08:10.215244+0000","last_became_peered":"2024-03-12T17:08:10.215244+0000","last_unstale":"2024-03-12T17:08:12.998534+0000","last_undegraded":"2024-03-12T17:08:12.998534+0000","last_fullsized":"2024-03-12T17:08:12.998534+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:00:06.057527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.690683+0000","last_change":"2024-03-12T17:08:10.728322+0000","last_active":"2024-03-12T17:08:11.690683+0000","last_peered":"2024-03-12T17:08:11.690683+0000","last_clean":"2024-03-12T17:08:11.690683+0000","last_became_active":"2024-03-12T17:08:10.727874+0000","last_became_peered":"2024-03-12T17:08:10.727874+0000","last_unstale":"2024-03-12T17:08:11.690683+0000","last_undegraded":"2024-03-12T17:08:11.690683+0000","last_fullsized":"2024-03-12T17:08:11.690683+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:16:52.065023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.479396+0000","last_change":"2024-03-12T17:08:10.203965+0000","last_active":"2024-03-12T17:08:12.479396+0000","last_peered":"2024-03-12T17:08:12.479396+0000","last_clean":"2024-03-12T17:08:12.479396+0000","last_became_active":"2024-03-12T17:08:10.203610+0000","last_became_peered":"2024-03-12T17:08:10.203610+0000","last_unstale":"2024-03-12T17:08:12.479396+0000","last_undegraded":"2024-03-12T17:08:12.479396+0000","last_fullsized":"2024-03-12T17:08:12.479396+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:00:06.057527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.997665+0000","last_change":"2024-03-12T17:08:10.711425+0000","last_active":"2024-03-12T17:08:12.997665+0000","last_peered":"2024-03-12T17:08:12.997665+0000","last_clean":"2024-03-12T17:08:12.997665+0000","last_became_active":"2024-03-12T17:08:10.711171+0000","last_became_peered":"2024-03-12T17:08:10.711171+0000","last_unstale":"2024-03-12T17:08:12.997665+0000","last_undegraded":"2024-03-12T17:08:12.997665+0000","last_fullsized":"2024-03-12T17:08:12.997665+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:18:33.858639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.696402+0000","last_change":"2024-03-12T17:08:10.222524+0000","last_active":"2024-03-12T17:08:11.696402+0000","last_peered":"2024-03-12T17:08:11.696402+0000","last_clean":"2024-03-12T17:08:11.696402+0000","last_became_active":"2024-03-12T17:08:10.221941+0000","last_became_peered":"2024-03-12T17:08:10.221941+0000","last_unstale":"2024-03-12T17:08:11.696402+0000","last_undegraded":"2024-03-12T17:08:11.696402+0000","last_fullsized":"2024-03-12T17:08:11.696402+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T20:40:15.944188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.999156+0000","last_change":"2024-03-12T17:08:10.734848+0000","last_active":"2024-03-12T17:08:12.999156+0000","last_peered":"2024-03-12T17:08:12.999156+0000","last_clean":"2024-03-12T17:08:12.999156+0000","last_became_active":"2024-03-12T17:08:10.734623+0000","last_became_peered":"2024-03-12T17:08:10.734623+0000","last_unstale":"2024-03-12T17:08:12.999156+0000","last_undegraded":"2024-03-12T17:08:12.999156+0000","last_fullsized":"2024-03-12T17:08:12.999156+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:17:24.644009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"40'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.483833+0000","last_change":"2024-03-12T17:08:10.224510+0000","last_active":"2024-03-12T17:08:12.483833+0000","last_peered":"2024-03-12T17:08:12.483833+0000","last_clean":"2024-03-12T17:08:12.483833+0000","last_became_active":"2024-03-12T17:08:10.224192+0000","last_became_peered":"2024-03-12T17:08:10.224192+0000","last_unstale":"2024-03-12T17:08:12.483833+0000","last_undegraded":"2024-03-12T17:08:12.483833+0000","last_fullsized":"2024-03-12T17:08:12.483833+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:35:52.711369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.690765+0000","last_change":"2024-03-12T17:08:10.730592+0000","last_active":"2024-03-12T17:08:11.690765+0000","last_peered":"2024-03-12T17:08:11.690765+0000","last_clean":"2024-03-12T17:08:11.690765+0000","last_became_active":"2024-03-12T17:08:10.729663+0000","last_became_peered":"2024-03-12T17:08:10.729663+0000","last_unstale":"2024-03-12T17:08:11.690765+0000","last_undegraded":"2024-03-12T17:08:11.690765+0000","last_fullsized":"2024-03-12T17:08:11.690765+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:52:53.108451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.898348+0000","last_change":"2024-03-12T17:08:10.212568+0000","last_active":"2024-03-12T17:08:11.898348+0000","last_peered":"2024-03-12T17:08:11.898348+0000","last_clean":"2024-03-12T17:08:11.898348+0000","last_became_active":"2024-03-12T17:08:10.212285+0000","last_became_peered":"2024-03-12T17:08:10.212285+0000","last_unstale":"2024-03-12T17:08:11.898348+0000","last_undegraded":"2024-03-12T17:08:11.898348+0000","last_fullsized":"2024-03-12T17:08:11.898348+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:21:28.144885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.897924+0000","last_change":"2024-03-12T17:08:10.702162+0000","last_active":"2024-03-12T17:08:11.897924+0000","last_peered":"2024-03-12T17:08:11.897924+0000","last_clean":"2024-03-12T17:08:11.897924+0000","last_became_active":"2024-03-12T17:08:10.701911+0000","last_became_peered":"2024-03-12T17:08:10.701911+0000","last_unstale":"2024-03-12T17:08:11.897924+0000","last_undegraded":"2024-03-12T17:08:11.897924+0000","last_fullsized":"2024-03-12T17:08:11.897924+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:25:32.963315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.898163+0000","last_change":"2024-03-12T17:08:10.214326+0000","last_active":"2024-03-12T17:08:11.898163+0000","last_peered":"2024-03-12T17:08:11.898163+0000","last_clean":"2024-03-12T17:08:11.898163+0000","last_became_active":"2024-03-12T17:08:10.214015+0000","last_became_peered":"2024-03-12T17:08:10.214015+0000","last_unstale":"2024-03-12T17:08:11.898163+0000","last_undegraded":"2024-03-12T17:08:11.898163+0000","last_fullsized":"2024-03-12T17:08:11.898163+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:12:43.823882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.997321+0000","last_change":"2024-03-12T17:08:10.711888+0000","last_active":"2024-03-12T17:08:12.997321+0000","last_peered":"2024-03-12T17:08:12.997321+0000","last_clean":"2024-03-12T17:08:12.997321+0000","last_became_active":"2024-03-12T17:08:10.711638+0000","last_became_peered":"2024-03-12T17:08:10.711638+0000","last_unstale":"2024-03-12T17:08:12.997321+0000","last_undegraded":"2024-03-12T17:08:12.997321+0000","last_fullsized":"2024-03-12T17:08:12.997321+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:44:36.258866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.833905+0000","last_change":"2024-03-12T17:08:10.209448+0000","last_active":"2024-03-12T17:08:11.833905+0000","last_peered":"2024-03-12T17:08:11.833905+0000","last_clean":"2024-03-12T17:08:11.833905+0000","last_became_active":"2024-03-12T17:08:10.208888+0000","last_became_peered":"2024-03-12T17:08:10.208888+0000","last_unstale":"2024-03-12T17:08:11.833905+0000","last_undegraded":"2024-03-12T17:08:11.833905+0000","last_fullsized":"2024-03-12T17:08:11.833905+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:29:58.170701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.695512+0000","last_change":"2024-03-12T17:08:10.722062+0000","last_active":"2024-03-12T17:08:11.695512+0000","last_peered":"2024-03-12T17:08:11.695512+0000","last_clean":"2024-03-12T17:08:11.695512+0000","last_became_active":"2024-03-12T17:08:10.721753+0000","last_became_peered":"2024-03-12T17:08:10.721753+0000","last_unstale":"2024-03-12T17:08:11.695512+0000","last_undegraded":"2024-03-12T17:08:11.695512+0000","last_fullsized":"2024-03-12T17:08:11.695512+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:33:01.223233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.694540+0000","last_change":"2024-03-12T17:08:10.219399+0000","last_active":"2024-03-12T17:08:11.694540+0000","last_peered":"2024-03-12T17:08:11.694540+0000","last_clean":"2024-03-12T17:08:11.694540+0000","last_became_active":"2024-03-12T17:08:10.219119+0000","last_became_peered":"2024-03-12T17:08:10.219119+0000","last_unstale":"2024-03-12T17:08:11.694540+0000","last_undegraded":"2024-03-12T17:08:11.694540+0000","last_fullsized":"2024-03-12T17:08:11.694540+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:39:57.420626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.999170+0000","last_change":"2024-03-12T17:08:10.735540+0000","last_active":"2024-03-12T17:08:12.999170+0000","last_peered":"2024-03-12T17:08:12.999170+0000","last_clean":"2024-03-12T17:08:12.999170+0000","last_became_active":"2024-03-12T17:08:10.735262+0000","last_became_peered":"2024-03-12T17:08:10.735262+0000","last_unstale":"2024-03-12T17:08:12.999170+0000","last_undegraded":"2024-03-12T17:08:12.999170+0000","last_fullsized":"2024-03-12T17:08:12.999170+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:23:32.915382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.690178+0000","last_change":"2024-03-12T17:08:10.212008+0000","last_active":"2024-03-12T17:08:11.690178+0000","last_peered":"2024-03-12T17:08:11.690178+0000","last_clean":"2024-03-12T17:08:11.690178+0000","last_became_active":"2024-03-12T17:08:10.211720+0000","last_became_peered":"2024-03-12T17:08:10.211720+0000","last_unstale":"2024-03-12T17:08:11.690178+0000","last_undegraded":"2024-03-12T17:08:11.690178+0000","last_fullsized":"2024-03-12T17:08:11.690178+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:06:31.032998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.696015+0000","last_change":"2024-03-12T17:08:10.718404+0000","last_active":"2024-03-12T17:08:11.696015+0000","last_peered":"2024-03-12T17:08:11.696015+0000","last_clean":"2024-03-12T17:08:11.696015+0000","last_became_active":"2024-03-12T17:08:10.715811+0000","last_became_peered":"2024-03-12T17:08:10.715811+0000","last_unstale":"2024-03-12T17:08:11.696015+0000","last_undegraded":"2024-03-12T17:08:11.696015+0000","last_fullsized":"2024-03-12T17:08:11.696015+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:26:50.546066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.998948+0000","last_change":"2024-03-12T17:08:10.213481+0000","last_active":"2024-03-12T17:08:12.998948+0000","last_peered":"2024-03-12T17:08:12.998948+0000","last_clean":"2024-03-12T17:08:12.998948+0000","last_became_active":"2024-03-12T17:08:10.213191+0000","last_became_peered":"2024-03-12T17:08:10.213191+0000","last_unstale":"2024-03-12T17:08:12.998948+0000","last_undegraded":"2024-03-12T17:08:12.998948+0000","last_fullsized":"2024-03-12T17:08:12.998948+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:20:32.413027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":179,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-12T17:08:10.716662+0000","last_change":"2024-03-12T17:06:21.714813+0000","last_active":"2024-03-12T17:08:10.716662+0000","last_peered":"2024-03-12T17:08:10.716662+0000","last_clean":"2024-03-12T17:08:10.716662+0000","last_became_active":"2024-03-12T17:06:21.712488+0000","last_became_peered":"2024-03-12T17:06:21.712488+0000","last_unstale":"2024-03-12T17:08:10.716662+0000","last_undegraded":"2024-03-12T17:08:10.716662+0000","last_fullsized":"2024-03-12T17:08:10.716662+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:19.815688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:19.815688+0000","last_clean_scrub_stamp":"2024-03-12T17:06:19.815688+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:38:54.300793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28064,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343776,"statfs":{"total":96636764160,"available":96608026624,"internally_reserved":0,"allocated":1060864,"data_stored":784160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:07:22 2024","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":1.095}]},{"osd":1,"last update":"Tue Mar 12 17:07:22 2024","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.845,"5min":1.845,"15min":1.845},"last":1.379},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.2889999999999999}]},{"osd":2,"last update":"Tue Mar 12 17:07:22 2024","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.121,"15min":1.121},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.841,"5min":1.841,"15min":1.841},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.861,"5min":1.861,"15min":1.861},"last":1.214}]},{"osd":3,"last update":"Tue Mar 12 17:07:22 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.4039999999999999,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.522}]},{"osd":4,"last update":"Tue Mar 12 17:07:22 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":3.5680000000000001,"5min":3.5680000000000001,"15min":3.5680000000000001},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.464,"5min":2.464,"15min":2.464},"last":1.6140000000000001}]},{"osd":5,"last update":"Tue Mar 12 17:07:22 2024","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.474},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.5310000000000001,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.157}]},{"osd":6,"last update":"Tue Mar 12 17:07:22 2024","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.401,"15min":1.401},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.587},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":4.625,"5min":4.625,"15min":4.625},"last":1.339}]}]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":466944,"data_stored":199360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.2170000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":0.876}]},{"osd":1,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.1120000000000001}]},{"osd":2,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.177,"5min":1.177,"15min":1.177},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":1.2969999999999999}]},{"osd":3,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.419},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.8380000000000001}]},{"osd":4,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.5030000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":2.2410000000000001},{"interface":"front","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.772}]},{"osd":5,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.9670000000000001},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":2.1320000000000001,"5min":2.1320000000000001,"15min":2.1320000000000001},"last":2.1459999999999999}]},{"osd":7,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.8999999999999999},{"interface":"front","average":{"1min":1.466,"5min":1.466,"15min":1.466},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.6560000000000001,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":2.4049999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":198890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":3.6709999999999998,"5min":3.6709999999999998,"15min":3.6709999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":3.706,"5min":3.706,"15min":3.706},"last":0.55600000000000005}]},{"osd":1,"last update":"Tue Mar 12 17:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":3.0390000000000001,"5min":3.0390000000000001,"15min":3.0390000000000001},"last":0.442},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.56599999999999995}]},{"osd":2,"last update":"Tue Mar 12 17:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.498},{"interface":"front","average":{"1min":1.264,"5min":1.264,"15min":1.264},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.53500000000000003}]},{"osd":3,"last update":"Tue Mar 12 17:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":3.7440000000000002,"5min":3.7440000000000002,"15min":3.7440000000000002},"last":0.38600000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.294,"15min":1.294},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":0.58599999999999997}]},{"osd":4,"last update":"Tue Mar 12 17:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.893,"5min":1.893,"15min":1.893},"last":0.34499999999999997}]},{"osd":6,"last update":"Tue Mar 12 17:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":8.2080000000000002,"5min":8.2080000000000002,"15min":8.2080000000000002},"last":0.41099999999999998},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.51400000000000001}]},{"osd":7,"last update":"Tue Mar 12 17:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":1.859,"5min":1.859,"15min":1.859},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":8.0139999999999993,"5min":8.0139999999999993,"15min":8.0139999999999993},"last":0.46600000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":199382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.724,"5min":1.724,"15min":1.724},"last":1.0489999999999999}]},{"osd":1,"last update":"Tue Mar 12 17:07:17 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":2.1309999999999998,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.639}]},{"osd":2,"last update":"Tue Mar 12 17:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.224,"15min":1.224},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":1.982,"5min":1.982,"15min":1.982},"last":1.141},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.1640000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":0.97499999999999998}]},{"osd":3,"last update":"Tue Mar 12 17:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.177,"15min":1.177},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":1.399},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.444}]},{"osd":5,"last update":"Tue Mar 12 17:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":2.2589999999999999,"5min":2.2589999999999999,"15min":2.2589999999999999},"last":1.762},{"interface":"front","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":1.5069999999999999}]},{"osd":6,"last update":"Tue Mar 12 17:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.385,"15min":1.385},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.3559999999999999,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":1.831}]},{"osd":7,"last update":"Tue Mar 12 17:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.425,"15min":1.425},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.0619999999999998,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":1.601},{"interface":"front","average":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":3.5720000000000001,"5min":3.5720000000000001,"15min":3.5720000000000001},"last":1.6879999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31592,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340248,"statfs":{"total":96636764160,"available":96604413952,"internally_reserved":0,"allocated":479232,"data_stored":200376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":5.0039999999999996,"5min":5.0039999999999996,"15min":5.0039999999999996},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.0449999999999999,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":1.1639999999999999}]},{"osd":1,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":4.9080000000000004,"5min":4.9080000000000004,"15min":4.9080000000000004},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.2410000000000001,"5min":3.2410000000000001,"15min":3.2410000000000001},"last":1.355}]},{"osd":2,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":3.4990000000000001,"5min":3.4990000000000001,"15min":3.4990000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":5.1870000000000003,"5min":5.1870000000000003,"15min":5.1870000000000003},"last":1.6679999999999999}]},{"osd":4,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":9.9299999999999997,"5min":9.9299999999999997,"15min":9.9299999999999997},"last":1.008},{"interface":"front","average":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":5.2450000000000001,"5min":5.2450000000000001,"15min":5.2450000000000001},"last":1.5740000000000001}]},{"osd":5,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.369,"15min":1.369},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.9550000000000001,"5min":2.9550000000000001,"15min":2.9550000000000001},"last":1.048},{"interface":"front","average":{"1min":1.589,"5min":1.589,"15min":1.589},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":3.089,"5min":3.089,"15min":3.089},"last":1.26}]},{"osd":6,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":3.3170000000000002,"5min":3.3170000000000002,"15min":3.3170000000000002},"last":1.732},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":2.722,"5min":2.722,"15min":2.722},"last":1.1439999999999999}]},{"osd":7,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.7559999999999998,"5min":2.7559999999999998,"15min":2.7559999999999998},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":3.1389999999999998,"5min":3.1389999999999998,"15min":3.1389999999999998},"last":1.4019999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":466944,"data_stored":198848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:07:18 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.39400000000000002},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":3.3879999999999999,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":0.94099999999999995}]},{"osd":1,"last update":"Tue Mar 12 17:07:18 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.7639999999999998,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":0.86499999999999999}]},{"osd":3,"last update":"Tue Mar 12 17:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.46200000000000002}]},{"osd":4,"last update":"Tue Mar 12 17:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.26,"15min":1.26},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":2.9569999999999999,"5min":2.9569999999999999,"15min":2.9569999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.46,"5min":1.46,"15min":1.46},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":4.4640000000000004,"5min":4.4640000000000004,"15min":4.4640000000000004},"last":1.093}]},{"osd":5,"last update":"Tue Mar 12 17:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.8069999999999999,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":1.018}]},{"osd":6,"last update":"Tue Mar 12 17:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.502,"15min":1.502},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":3.5179999999999998,"5min":3.5179999999999998,"15min":3.5179999999999998},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.492,"5min":1.492,"15min":1.492},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.464,"5min":2.464,"15min":2.464},"last":0.92900000000000005}]},{"osd":7,"last update":"Tue Mar 12 17:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":3.6110000000000002,"5min":3.6110000000000002,"15min":3.6110000000000002},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.571,"5min":1.571,"15min":1.571},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.8730000000000002,"5min":2.8730000000000002,"15min":2.8730000000000002},"last":0.64800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":454656,"data_stored":192248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":1.026},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":3.6419999999999999,"5min":3.6419999999999999,"15min":3.6419999999999999},"last":2.1459999999999999}]},{"osd":2,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.5049999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.84699999999999998}]},{"osd":3,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":3.9580000000000002,"5min":3.9580000000000002,"15min":3.9580000000000002},"last":1.119},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.8,"5min":1.8,"15min":1.8},"last":1.1579999999999999}]},{"osd":4,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.694,"5min":2.694,"15min":2.694},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":3.5150000000000001,"5min":3.5150000000000001,"15min":3.5150000000000001},"last":1.2430000000000001}]},{"osd":5,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":1.23},{"interface":"front","average":{"1min":1.536,"5min":1.536,"15min":1.536},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.7709999999999999,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":1.873}]},{"osd":6,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":4.1269999999999998,"5min":4.1269999999999998,"15min":4.1269999999999998},"last":1.391},{"interface":"front","average":{"1min":1.49,"5min":1.49,"15min":1.49},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":3.7850000000000001,"5min":3.7850000000000001,"15min":3.7850000000000001},"last":1.0800000000000001}]},{"osd":7,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":4.0030000000000001,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.357},{"interface":"front","average":{"1min":1.623,"5min":1.623,"15min":1.623},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.8660000000000001,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.9930000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1060864,"data_stored":789180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.71299999999999997}]},{"osd":2,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":0.67600000000000005},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":0.63100000000000001}]},{"osd":3,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":1.139},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.091}]},{"osd":4,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.403,"15min":1.403},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":3.0499999999999998,"5min":3.0499999999999998,"15min":3.0499999999999998},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":2.8319999999999999,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":1.2470000000000001}]},{"osd":5,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":2.2850000000000001,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.45}]},{"osd":6,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"min":{"1min":0.999,"5min":0.999,"15min":0.999},"max":{"1min":2.597,"5min":2.597,"15min":2.597},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"min":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"max":{"1min":2.452,"5min":2.452,"15min":2.452},"last":1.038}]},{"osd":7,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"min":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"max":{"1min":2.7240000000000002,"5min":2.7240000000000002,"15min":2.7240000000000002},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.617,"5min":1.617,"15min":1.617},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.6760000000000002,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":1.2789999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-12T17:08:15.615 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-12T17:08:15.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.887+0000 7f0cce600640 1 Processor -- start 2024-03-12T17:08:15.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.887+0000 7f0cce600640 1 -- start start 2024-03-12T17:08:15.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.889+0000 7f0cce600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0cc8160ef0 0x7f0cc80b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:15.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.889+0000 7f0cce600640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f0cc80ae6f0 0x7f0cc8069c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:15.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.889+0000 7f0cce600640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cc80acdb0 con 0x7f0cc8160b10 2024-03-12T17:08:15.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.889+0000 7f0cce600640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cc80ad650 con 0x7f0cc80ae6f0 2024-03-12T17:08:15.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.889+0000 7f0cce600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cc80b4540 con 0x7f0cc8160ef0 2024-03-12T17:08:15.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.889+0000 7f0ccd200640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f0cc8160b10 0x7f0cc80b23c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:34644/0 (socket says 172.21.15.52:34644) 2024-03-12T17:08:15.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.889+0000 7f0ccdc00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f0cc80ae6f0 0x7f0cc8069c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:15.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.889+0000 7f0ccd200640 1 -- 172.21.15.52:0/1085708871 learned_addr learned my addr 172.21.15.52:0/1085708871 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:08:15.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.889+0000 7f0cc7e00640 1 --2- 172.21.15.52:0/1085708871 >> v2:172.21.15.102:3300/0 conn(0x7f0cc8160ef0 0x7f0cc80b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:15.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.889+0000 7f0cc7400640 1 -- 172.21.15.52:0/1085708871 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1494262768 0 0) 0x7f0cc80acdb0 con 0x7f0cc8160b10 2024-03-12T17:08:15.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.890+0000 7f0ccdc00640 1 -- 172.21.15.52:0/1085708871 >> v2:172.21.15.102:3300/0 conn(0x7f0cc8160ef0 msgr2=0x7f0cc80b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:15.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.890+0000 7f0ccdc00640 1 --2- 172.21.15.52:0/1085708871 >> v2:172.21.15.102:3300/0 conn(0x7f0cc8160ef0 0x7f0cc80b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:15.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.890+0000 7f0ccdc00640 1 -- 172.21.15.52:0/1085708871 >> v1:172.21.15.52:6789/0 conn(0x7f0cc8160b10 legacy=0x7f0cc80b23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:15.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.890+0000 7f0ccdc00640 1 -- 172.21.15.52:0/1085708871 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc80ad000 con 0x7f0cc80ae6f0 2024-03-12T17:08:15.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.891+0000 7f0ccdc00640 1 --2- 172.21.15.52:0/1085708871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0cc80ae6f0 0x7f0cc8069c90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0cc80b39d0 tx=0x7f0cc002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cecc5143d4a8ae45 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:15.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.891+0000 7f0cc7400640 1 -- 172.21.15.52:0/1085708871 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc002cc80 con 0x7f0cc80ae6f0 2024-03-12T17:08:15.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.891+0000 7f0cc7400640 1 -- 172.21.15.52:0/1085708871 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc002cde0 con 0x7f0cc80ae6f0 2024-03-12T17:08:15.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.891+0000 7f0cc7400640 1 -- 172.21.15.52:0/1085708871 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0034660 con 0x7f0cc80ae6f0 2024-03-12T17:08:15.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.891+0000 7f0cce600640 1 -- 172.21.15.52:0/1085708871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0cc80ae6f0 msgr2=0x7f0cc8069c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:15.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.891+0000 7f0cce600640 1 --2- 172.21.15.52:0/1085708871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0cc80ae6f0 0x7f0cc8069c90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0cc80b39d0 tx=0x7f0cc002bed0 comp rx=0 tx=0).stop 2024-03-12T17:08:15.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.892+0000 7f0cce600640 1 -- 172.21.15.52:0/1085708871 shutdown_connections 2024-03-12T17:08:15.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.892+0000 7f0cce600640 1 --2- 172.21.15.52:0/1085708871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0cc80ae6f0 0x7f0cc8069c90 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:15.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.892+0000 7f0cce600640 1 --2- 172.21.15.52:0/1085708871 >> v2:172.21.15.102:3300/0 conn(0x7f0cc8160ef0 0x7f0cc80b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:15.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.892+0000 7f0cce600640 1 -- 172.21.15.52:0/1085708871 >> 172.21.15.52:0/1085708871 conn(0x7f0cc80b48e0 msgr2=0x7f0cc80b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:15.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.892+0000 7f0cce600640 1 -- 172.21.15.52:0/1085708871 shutdown_connections 2024-03-12T17:08:15.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.892+0000 7f0cce600640 1 -- 172.21.15.52:0/1085708871 wait complete. 2024-03-12T17:08:15.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.893+0000 7f0cce600640 1 Processor -- start 2024-03-12T17:08:15.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.894+0000 7f0cce600640 1 -- start start 2024-03-12T17:08:15.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.894+0000 7f0cce600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0cc80ae6f0 0x7f0cc814ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:15.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.894+0000 7f0cce600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0cc8160b10 0x7f0cc80cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:15.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.894+0000 7f0cce600640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0cc8160ef0 0x7f0cc80ce210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:15.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.894+0000 7f0cce600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc815c970 con 0x7f0cc8160b10 2024-03-12T17:08:15.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.894+0000 7f0cce600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc80ce840 con 0x7f0cc80ae6f0 2024-03-12T17:08:15.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.894+0000 7f0cce600640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc80ceb20 con 0x7f0cc8160ef0 2024-03-12T17:08:15.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.895+0000 7f0cc7e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0cc8160b10 0x7f0cc80cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:15.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.895+0000 7f0cc7e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0cc8160b10 0x7f0cc80cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36924/0 (socket says 172.21.15.52:36924) 2024-03-12T17:08:15.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.895+0000 7f0cc7e00640 1 -- 172.21.15.52:0/3803238061 learned_addr learned my addr 172.21.15.52:0/3803238061 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:15.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.895+0000 7f0ccdc00640 1 --2- 172.21.15.52:0/3803238061 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0cc8160ef0 0x7f0cc80ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:15.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.895+0000 7f0ccd200640 1 --2- 172.21.15.52:0/3803238061 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0cc80ae6f0 0x7f0cc814ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:15.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.895+0000 7f0cc7e00640 1 -- 172.21.15.52:0/3803238061 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0cc8160ef0 msgr2=0x7f0cc80ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:15.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.895+0000 7f0cc7e00640 1 --2- 172.21.15.52:0/3803238061 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0cc8160ef0 0x7f0cc80ce210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:15.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.895+0000 7f0cc7e00640 1 -- 172.21.15.52:0/3803238061 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0cc80ae6f0 msgr2=0x7f0cc814ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:15.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.895+0000 7f0cc7e00640 1 --2- 172.21.15.52:0/3803238061 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0cc80ae6f0 0x7f0cc814ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:15.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.895+0000 7f0cc7e00640 1 -- 172.21.15.52:0/3803238061 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc0009a70 con 0x7f0cc8160b10 2024-03-12T17:08:15.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.895+0000 7f0ccd200640 1 --2- 172.21.15.52:0/3803238061 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0cc80ae6f0 0x7f0cc814ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:08:15.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.896+0000 7f0cc7e00640 1 --2- 172.21.15.52:0/3803238061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0cc8160b10 0x7f0cc80cdcd0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0cb400d5f0 tx=0x7f0cb400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:15.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.896+0000 7f0cc4c00640 1 -- 172.21.15.52:0/3803238061 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb40125f0 con 0x7f0cc8160b10 2024-03-12T17:08:15.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.896+0000 7f0cc4c00640 1 -- 172.21.15.52:0/3803238061 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb4012750 con 0x7f0cc8160b10 2024-03-12T17:08:15.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.896+0000 7f0cc4c00640 1 -- 172.21.15.52:0/3803238061 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4012b00 con 0x7f0cc8160b10 2024-03-12T17:08:15.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.896+0000 7f0cce600640 1 -- 172.21.15.52:0/3803238061 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cc80cee00 con 0x7f0cc8160b10 2024-03-12T17:08:15.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.896+0000 7f0cce600640 1 -- 172.21.15.52:0/3803238061 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc80cb8e0 con 0x7f0cc8160b10 2024-03-12T17:08:15.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.897+0000 7f0cce600640 1 -- 172.21.15.52:0/3803238061 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cc804e030 con 0x7f0cc8160b10 2024-03-12T17:08:15.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.898+0000 7f0cc4c00640 1 -- 172.21.15.52:0/3803238061 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0cb4012c60 con 0x7f0cc8160b10 2024-03-12T17:08:15.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.899+0000 7f0cc4c00640 1 --2- 172.21.15.52:0/3803238061 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f0ca806ac00 0x7f0ca806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:15.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.899+0000 7f0cc4c00640 1 -- 172.21.15.52:0/3803238061 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0cb40899f0 con 0x7f0cc8160b10 2024-03-12T17:08:15.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.900+0000 7f0ccd200640 1 --2- 172.21.15.52:0/3803238061 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f0ca806ac00 0x7f0ca806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:15.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.901+0000 7f0ccd200640 1 --2- 172.21.15.52:0/3803238061 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f0ca806ac00 0x7f0ca806d0c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8000f60 tx=0x7f0cb8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:15.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:15.904+0000 7f0cc4c00640 1 -- 172.21.15.52:0/3803238061 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f0cb40549f0 con 0x7f0cc8160b10 2024-03-12T17:08:16.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.104+0000 7f0cce600640 1 -- 172.21.15.52:0/3803238061 --> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0cc80cf320 con 0x7f0ca806ac00 2024-03-12T17:08:16.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.127+0000 7f0cc4c00640 1 -- 172.21.15.52:0/3803238061 <== mgr.4109 v2:172.21.15.102:6832/36622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287636 (secure 0 0 0) 0x7f0cc80cf320 con 0x7f0ca806ac00 2024-03-12T17:08:16.122 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:08:16.135 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-03-12T17:08:16.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.134+0000 7f0cce600640 1 -- 172.21.15.52:0/3803238061 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f0ca806ac00 msgr2=0x7f0ca806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:16.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.134+0000 7f0cce600640 1 --2- 172.21.15.52:0/3803238061 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f0ca806ac00 0x7f0ca806d0c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8000f60 tx=0x7f0cb8009040 comp rx=0 tx=0).stop 2024-03-12T17:08:16.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.134+0000 7f0cce600640 1 -- 172.21.15.52:0/3803238061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0cc8160b10 msgr2=0x7f0cc80cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:16.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.134+0000 7f0cce600640 1 --2- 172.21.15.52:0/3803238061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0cc8160b10 0x7f0cc80cdcd0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0cb400d5f0 tx=0x7f0cb400dab0 comp rx=0 tx=0).stop 2024-03-12T17:08:16.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.134+0000 7f0cce600640 1 -- 172.21.15.52:0/3803238061 shutdown_connections 2024-03-12T17:08:16.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.134+0000 7f0cce600640 1 --2- 172.21.15.52:0/3803238061 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f0ca806ac00 0x7f0ca806d0c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:16.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.134+0000 7f0cce600640 1 --2- 172.21.15.52:0/3803238061 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0cc8160ef0 0x7f0cc80ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:16.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.134+0000 7f0cce600640 1 --2- 172.21.15.52:0/3803238061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0cc8160b10 0x7f0cc80cdcd0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:16.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.134+0000 7f0cce600640 1 --2- 172.21.15.52:0/3803238061 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0cc80ae6f0 0x7f0cc814ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:16.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.134+0000 7f0cce600640 1 -- 172.21.15.52:0/3803238061 >> 172.21.15.52:0/3803238061 conn(0x7f0cc80b48e0 msgr2=0x7f0cc815a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:16.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.134+0000 7f0cce600640 1 -- 172.21.15.52:0/3803238061 shutdown_connections 2024-03-12T17:08:16.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.134+0000 7f0cce600640 1 -- 172.21.15.52:0/3803238061 wait complete. 2024-03-12T17:08:16.148 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2024-03-12T17:08:15.189782+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":121,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237424,"kb_used_data":4816,"kb_used_omap":13,"kb_used_meta":232434,"kb_avail":754737296,"statfs":{"total":773094113280,"available":772850991104,"internally_reserved":0,"allocated":4931584,"data_stored":2762444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":238012504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001867"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.691460+0000","last_change":"2024-03-12T17:08:10.705329+0000","last_active":"2024-03-12T17:08:11.691460+0000","last_peered":"2024-03-12T17:08:11.691460+0000","last_clean":"2024-03-12T17:08:11.691460+0000","last_became_active":"2024-03-12T17:08:10.704448+0000","last_became_peered":"2024-03-12T17:08:10.704448+0000","last_unstale":"2024-03-12T17:08:11.691460+0000","last_undegraded":"2024-03-12T17:08:11.691460+0000","last_fullsized":"2024-03-12T17:08:11.691460+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:26:50.546066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.690621+0000","last_change":"2024-03-12T17:08:10.706543+0000","last_active":"2024-03-12T17:08:11.690621+0000","last_peered":"2024-03-12T17:08:11.690621+0000","last_clean":"2024-03-12T17:08:11.690621+0000","last_became_active":"2024-03-12T17:08:10.706231+0000","last_became_peered":"2024-03-12T17:08:10.706231+0000","last_unstale":"2024-03-12T17:08:11.690621+0000","last_undegraded":"2024-03-12T17:08:11.690621+0000","last_fullsized":"2024-03-12T17:08:11.690621+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:51:16.461818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.898295+0000","last_change":"2024-03-12T17:08:10.708889+0000","last_active":"2024-03-12T17:08:11.898295+0000","last_peered":"2024-03-12T17:08:11.898295+0000","last_clean":"2024-03-12T17:08:11.898295+0000","last_became_active":"2024-03-12T17:08:10.708618+0000","last_became_peered":"2024-03-12T17:08:10.708618+0000","last_unstale":"2024-03-12T17:08:11.898295+0000","last_undegraded":"2024-03-12T17:08:11.898295+0000","last_fullsized":"2024-03-12T17:08:11.898295+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:17:24.644009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.479806+0000","last_change":"2024-03-12T17:08:10.730011+0000","last_active":"2024-03-12T17:08:12.479806+0000","last_peered":"2024-03-12T17:08:12.479806+0000","last_clean":"2024-03-12T17:08:12.479806+0000","last_became_active":"2024-03-12T17:08:10.729713+0000","last_became_peered":"2024-03-12T17:08:10.729713+0000","last_unstale":"2024-03-12T17:08:12.479806+0000","last_undegraded":"2024-03-12T17:08:12.479806+0000","last_fullsized":"2024-03-12T17:08:12.479806+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:06:32.038325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.479156+0000","last_change":"2024-03-12T17:08:10.730105+0000","last_active":"2024-03-12T17:08:12.479156+0000","last_peered":"2024-03-12T17:08:12.479156+0000","last_clean":"2024-03-12T17:08:12.479156+0000","last_became_active":"2024-03-12T17:08:10.729706+0000","last_became_peered":"2024-03-12T17:08:10.729706+0000","last_unstale":"2024-03-12T17:08:12.479156+0000","last_undegraded":"2024-03-12T17:08:12.479156+0000","last_fullsized":"2024-03-12T17:08:12.479156+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T02:54:29.108633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.997336+0000","last_change":"2024-03-12T17:08:10.705015+0000","last_active":"2024-03-12T17:08:12.997336+0000","last_peered":"2024-03-12T17:08:12.997336+0000","last_clean":"2024-03-12T17:08:12.997336+0000","last_became_active":"2024-03-12T17:08:10.703748+0000","last_became_peered":"2024-03-12T17:08:10.703748+0000","last_unstale":"2024-03-12T17:08:12.997336+0000","last_undegraded":"2024-03-12T17:08:12.997336+0000","last_fullsized":"2024-03-12T17:08:12.997336+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:23:31.014856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.695191+0000","last_change":"2024-03-12T17:08:10.711451+0000","last_active":"2024-03-12T17:08:11.695191+0000","last_peered":"2024-03-12T17:08:11.695191+0000","last_clean":"2024-03-12T17:08:11.695191+0000","last_became_active":"2024-03-12T17:08:10.711089+0000","last_became_peered":"2024-03-12T17:08:10.711089+0000","last_unstale":"2024-03-12T17:08:11.695191+0000","last_undegraded":"2024-03-12T17:08:11.695191+0000","last_fullsized":"2024-03-12T17:08:11.695191+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:16:52.065023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.690284+0000","last_change":"2024-03-12T17:08:10.705241+0000","last_active":"2024-03-12T17:08:11.690284+0000","last_peered":"2024-03-12T17:08:11.690284+0000","last_clean":"2024-03-12T17:08:11.690284+0000","last_became_active":"2024-03-12T17:08:10.704955+0000","last_became_peered":"2024-03-12T17:08:10.704955+0000","last_unstale":"2024-03-12T17:08:11.690284+0000","last_undegraded":"2024-03-12T17:08:11.690284+0000","last_fullsized":"2024-03-12T17:08:11.690284+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:49:09.840848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.898509+0000","last_change":"2024-03-12T17:08:10.719672+0000","last_active":"2024-03-12T17:08:11.898509+0000","last_peered":"2024-03-12T17:08:11.898509+0000","last_clean":"2024-03-12T17:08:11.898509+0000","last_became_active":"2024-03-12T17:08:10.719435+0000","last_became_peered":"2024-03-12T17:08:10.719435+0000","last_unstale":"2024-03-12T17:08:11.898509+0000","last_undegraded":"2024-03-12T17:08:11.898509+0000","last_fullsized":"2024-03-12T17:08:11.898509+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:31:56.609569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.696347+0000","last_change":"2024-03-12T17:08:10.737543+0000","last_active":"2024-03-12T17:08:11.696347+0000","last_peered":"2024-03-12T17:08:11.696347+0000","last_clean":"2024-03-12T17:08:11.696347+0000","last_became_active":"2024-03-12T17:08:10.737287+0000","last_became_peered":"2024-03-12T17:08:10.737287+0000","last_unstale":"2024-03-12T17:08:11.696347+0000","last_undegraded":"2024-03-12T17:08:11.696347+0000","last_fullsized":"2024-03-12T17:08:11.696347+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:23:32.915382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.695546+0000","last_change":"2024-03-12T17:08:10.732923+0000","last_active":"2024-03-12T17:08:11.695546+0000","last_peered":"2024-03-12T17:08:11.695546+0000","last_clean":"2024-03-12T17:08:11.695546+0000","last_became_active":"2024-03-12T17:08:10.732654+0000","last_became_peered":"2024-03-12T17:08:10.732654+0000","last_unstale":"2024-03-12T17:08:11.695546+0000","last_undegraded":"2024-03-12T17:08:11.695546+0000","last_fullsized":"2024-03-12T17:08:11.695546+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:31:56.609569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.898085+0000","last_change":"2024-03-12T17:08:10.715031+0000","last_active":"2024-03-12T17:08:11.898085+0000","last_peered":"2024-03-12T17:08:11.898085+0000","last_clean":"2024-03-12T17:08:11.898085+0000","last_became_active":"2024-03-12T17:08:10.714743+0000","last_became_peered":"2024-03-12T17:08:10.714743+0000","last_unstale":"2024-03-12T17:08:11.898085+0000","last_undegraded":"2024-03-12T17:08:11.898085+0000","last_fullsized":"2024-03-12T17:08:11.898085+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:23:31.014856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.694233+0000","last_change":"2024-03-12T17:08:10.730025+0000","last_active":"2024-03-12T17:08:11.694233+0000","last_peered":"2024-03-12T17:08:11.694233+0000","last_clean":"2024-03-12T17:08:11.694233+0000","last_became_active":"2024-03-12T17:08:10.729354+0000","last_became_peered":"2024-03-12T17:08:10.729354+0000","last_unstale":"2024-03-12T17:08:11.694233+0000","last_undegraded":"2024-03-12T17:08:11.694233+0000","last_fullsized":"2024-03-12T17:08:11.694233+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:13:11.256122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.694079+0000","last_change":"2024-03-12T17:08:10.722267+0000","last_active":"2024-03-12T17:08:11.694079+0000","last_peered":"2024-03-12T17:08:11.694079+0000","last_clean":"2024-03-12T17:08:11.694079+0000","last_became_active":"2024-03-12T17:08:10.721988+0000","last_became_peered":"2024-03-12T17:08:10.721988+0000","last_unstale":"2024-03-12T17:08:11.694079+0000","last_undegraded":"2024-03-12T17:08:11.694079+0000","last_fullsized":"2024-03-12T17:08:11.694079+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:51:16.461818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.696151+0000","last_change":"2024-03-12T17:08:10.713034+0000","last_active":"2024-03-12T17:08:11.696151+0000","last_peered":"2024-03-12T17:08:11.696151+0000","last_clean":"2024-03-12T17:08:11.696151+0000","last_became_active":"2024-03-12T17:08:10.712704+0000","last_became_peered":"2024-03-12T17:08:10.712704+0000","last_unstale":"2024-03-12T17:08:11.696151+0000","last_undegraded":"2024-03-12T17:08:11.696151+0000","last_fullsized":"2024-03-12T17:08:11.696151+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T02:41:04.330660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.998095+0000","last_change":"2024-03-12T17:08:10.728808+0000","last_active":"2024-03-12T17:08:12.998095+0000","last_peered":"2024-03-12T17:08:12.998095+0000","last_clean":"2024-03-12T17:08:12.998095+0000","last_became_active":"2024-03-12T17:08:10.728334+0000","last_became_peered":"2024-03-12T17:08:10.728334+0000","last_unstale":"2024-03-12T17:08:12.998095+0000","last_undegraded":"2024-03-12T17:08:12.998095+0000","last_fullsized":"2024-03-12T17:08:12.998095+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:23:31.014856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.998803+0000","last_change":"2024-03-12T17:08:10.734394+0000","last_active":"2024-03-12T17:08:12.998803+0000","last_peered":"2024-03-12T17:08:12.998803+0000","last_clean":"2024-03-12T17:08:12.998803+0000","last_became_active":"2024-03-12T17:08:10.733323+0000","last_became_peered":"2024-03-12T17:08:10.733323+0000","last_unstale":"2024-03-12T17:08:12.998803+0000","last_undegraded":"2024-03-12T17:08:12.998803+0000","last_fullsized":"2024-03-12T17:08:12.998803+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T02:41:04.330660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.834163+0000","last_change":"2024-03-12T17:08:10.730315+0000","last_active":"2024-03-12T17:08:11.834163+0000","last_peered":"2024-03-12T17:08:11.834163+0000","last_clean":"2024-03-12T17:08:11.834163+0000","last_became_active":"2024-03-12T17:08:10.729938+0000","last_became_peered":"2024-03-12T17:08:10.729938+0000","last_unstale":"2024-03-12T17:08:11.834163+0000","last_undegraded":"2024-03-12T17:08:11.834163+0000","last_fullsized":"2024-03-12T17:08:11.834163+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:17:24.644009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.479981+0000","last_change":"2024-03-12T17:08:10.707305+0000","last_active":"2024-03-12T17:08:12.479981+0000","last_peered":"2024-03-12T17:08:12.479981+0000","last_clean":"2024-03-12T17:08:12.479981+0000","last_became_active":"2024-03-12T17:08:10.707029+0000","last_became_peered":"2024-03-12T17:08:10.707029+0000","last_unstale":"2024-03-12T17:08:12.479981+0000","last_undegraded":"2024-03-12T17:08:12.479981+0000","last_fullsized":"2024-03-12T17:08:12.479981+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:22:08.884650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.833904+0000","last_change":"2024-03-12T17:08:10.731158+0000","last_active":"2024-03-12T17:08:11.833904+0000","last_peered":"2024-03-12T17:08:11.833904+0000","last_clean":"2024-03-12T17:08:11.833904+0000","last_became_active":"2024-03-12T17:08:10.730829+0000","last_became_peered":"2024-03-12T17:08:10.730829+0000","last_unstale":"2024-03-12T17:08:11.833904+0000","last_undegraded":"2024-03-12T17:08:11.833904+0000","last_fullsized":"2024-03-12T17:08:11.833904+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:31:56.609569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.694512+0000","last_change":"2024-03-12T17:08:10.714510+0000","last_active":"2024-03-12T17:08:11.694512+0000","last_peered":"2024-03-12T17:08:11.694512+0000","last_clean":"2024-03-12T17:08:11.694512+0000","last_became_active":"2024-03-12T17:08:10.713860+0000","last_became_peered":"2024-03-12T17:08:10.713860+0000","last_unstale":"2024-03-12T17:08:11.694512+0000","last_undegraded":"2024-03-12T17:08:11.694512+0000","last_fullsized":"2024-03-12T17:08:11.694512+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:33:01.223233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.695741+0000","last_change":"2024-03-12T17:08:10.722283+0000","last_active":"2024-03-12T17:08:11.695741+0000","last_peered":"2024-03-12T17:08:11.695741+0000","last_clean":"2024-03-12T17:08:11.695741+0000","last_became_active":"2024-03-12T17:08:10.721775+0000","last_became_peered":"2024-03-12T17:08:10.721775+0000","last_unstale":"2024-03-12T17:08:11.695741+0000","last_undegraded":"2024-03-12T17:08:11.695741+0000","last_fullsized":"2024-03-12T17:08:11.695741+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T02:41:04.330660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.998275+0000","last_change":"2024-03-12T17:08:10.735342+0000","last_active":"2024-03-12T17:08:12.998275+0000","last_peered":"2024-03-12T17:08:12.998275+0000","last_clean":"2024-03-12T17:08:12.998275+0000","last_became_active":"2024-03-12T17:08:10.732932+0000","last_became_peered":"2024-03-12T17:08:10.732932+0000","last_unstale":"2024-03-12T17:08:12.998275+0000","last_undegraded":"2024-03-12T17:08:12.998275+0000","last_fullsized":"2024-03-12T17:08:12.998275+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:18:33.858639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.479369+0000","last_change":"2024-03-12T17:08:10.735946+0000","last_active":"2024-03-12T17:08:12.479369+0000","last_peered":"2024-03-12T17:08:12.479369+0000","last_clean":"2024-03-12T17:08:12.479369+0000","last_became_active":"2024-03-12T17:08:10.735517+0000","last_became_peered":"2024-03-12T17:08:10.735517+0000","last_unstale":"2024-03-12T17:08:12.479369+0000","last_undegraded":"2024-03-12T17:08:12.479369+0000","last_fullsized":"2024-03-12T17:08:12.479369+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:29:59.176028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.834617+0000","last_change":"2024-03-12T17:08:10.714628+0000","last_active":"2024-03-12T17:08:11.834617+0000","last_peered":"2024-03-12T17:08:11.834617+0000","last_clean":"2024-03-12T17:08:11.834617+0000","last_became_active":"2024-03-12T17:08:10.714287+0000","last_became_peered":"2024-03-12T17:08:10.714287+0000","last_unstale":"2024-03-12T17:08:11.834617+0000","last_undegraded":"2024-03-12T17:08:11.834617+0000","last_fullsized":"2024-03-12T17:08:11.834617+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:23:31.014856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.997652+0000","last_change":"2024-03-12T17:08:10.721517+0000","last_active":"2024-03-12T17:08:12.997652+0000","last_peered":"2024-03-12T17:08:12.997652+0000","last_clean":"2024-03-12T17:08:12.997652+0000","last_became_active":"2024-03-12T17:08:10.721204+0000","last_became_peered":"2024-03-12T17:08:10.721204+0000","last_unstale":"2024-03-12T17:08:12.997652+0000","last_undegraded":"2024-03-12T17:08:12.997652+0000","last_fullsized":"2024-03-12T17:08:12.997652+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:17:24.644009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.834154+0000","last_change":"2024-03-12T17:08:10.712942+0000","last_active":"2024-03-12T17:08:11.834154+0000","last_peered":"2024-03-12T17:08:11.834154+0000","last_clean":"2024-03-12T17:08:11.834154+0000","last_became_active":"2024-03-12T17:08:10.711213+0000","last_became_peered":"2024-03-12T17:08:10.711213+0000","last_unstale":"2024-03-12T17:08:11.834154+0000","last_undegraded":"2024-03-12T17:08:11.834154+0000","last_fullsized":"2024-03-12T17:08:11.834154+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:18:33.858639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.694802+0000","last_change":"2024-03-12T17:08:10.721650+0000","last_active":"2024-03-12T17:08:11.694802+0000","last_peered":"2024-03-12T17:08:11.694802+0000","last_clean":"2024-03-12T17:08:11.694802+0000","last_became_active":"2024-03-12T17:08:10.721313+0000","last_became_peered":"2024-03-12T17:08:10.721313+0000","last_unstale":"2024-03-12T17:08:11.694802+0000","last_undegraded":"2024-03-12T17:08:11.694802+0000","last_fullsized":"2024-03-12T17:08:11.694802+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:39:34.128499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.833498+0000","last_change":"2024-03-12T17:08:10.710889+0000","last_active":"2024-03-12T17:08:11.833498+0000","last_peered":"2024-03-12T17:08:11.833498+0000","last_clean":"2024-03-12T17:08:11.833498+0000","last_became_active":"2024-03-12T17:08:10.710575+0000","last_became_peered":"2024-03-12T17:08:10.710575+0000","last_unstale":"2024-03-12T17:08:11.833498+0000","last_undegraded":"2024-03-12T17:08:11.833498+0000","last_fullsized":"2024-03-12T17:08:11.833498+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:13:11.256122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.833690+0000","last_change":"2024-03-12T17:08:10.704928+0000","last_active":"2024-03-12T17:08:11.833690+0000","last_peered":"2024-03-12T17:08:11.833690+0000","last_clean":"2024-03-12T17:08:11.833690+0000","last_became_active":"2024-03-12T17:08:10.704232+0000","last_became_peered":"2024-03-12T17:08:10.704232+0000","last_unstale":"2024-03-12T17:08:11.833690+0000","last_undegraded":"2024-03-12T17:08:11.833690+0000","last_fullsized":"2024-03-12T17:08:11.833690+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:04:26.911580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.998494+0000","last_change":"2024-03-12T17:08:10.737536+0000","last_active":"2024-03-12T17:08:12.998494+0000","last_peered":"2024-03-12T17:08:12.998494+0000","last_clean":"2024-03-12T17:08:12.998494+0000","last_became_active":"2024-03-12T17:08:10.733851+0000","last_became_peered":"2024-03-12T17:08:10.733851+0000","last_unstale":"2024-03-12T17:08:12.998494+0000","last_undegraded":"2024-03-12T17:08:12.998494+0000","last_fullsized":"2024-03-12T17:08:12.998494+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:09:44.012820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.997267+0000","last_change":"2024-03-12T17:08:10.710798+0000","last_active":"2024-03-12T17:08:12.997267+0000","last_peered":"2024-03-12T17:08:12.997267+0000","last_clean":"2024-03-12T17:08:12.997267+0000","last_became_active":"2024-03-12T17:08:10.710188+0000","last_became_peered":"2024-03-12T17:08:10.710188+0000","last_unstale":"2024-03-12T17:08:12.997267+0000","last_undegraded":"2024-03-12T17:08:12.997267+0000","last_fullsized":"2024-03-12T17:08:12.997267+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:09:44.012820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.998919+0000","last_change":"2024-03-12T17:08:10.739282+0000","last_active":"2024-03-12T17:08:12.998919+0000","last_peered":"2024-03-12T17:08:12.998919+0000","last_clean":"2024-03-12T17:08:12.998919+0000","last_became_active":"2024-03-12T17:08:10.738996+0000","last_became_peered":"2024-03-12T17:08:10.738996+0000","last_unstale":"2024-03-12T17:08:12.998919+0000","last_undegraded":"2024-03-12T17:08:12.998919+0000","last_fullsized":"2024-03-12T17:08:12.998919+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:04:26.911580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.834663+0000","last_change":"2024-03-12T17:08:10.218826+0000","last_active":"2024-03-12T17:08:11.834663+0000","last_peered":"2024-03-12T17:08:11.834663+0000","last_clean":"2024-03-12T17:08:11.834663+0000","last_became_active":"2024-03-12T17:08:10.218560+0000","last_became_peered":"2024-03-12T17:08:10.218560+0000","last_unstale":"2024-03-12T17:08:11.834663+0000","last_undegraded":"2024-03-12T17:08:11.834663+0000","last_fullsized":"2024-03-12T17:08:11.834663+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:25:24.925580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.997750+0000","last_change":"2024-03-12T17:08:10.712062+0000","last_active":"2024-03-12T17:08:12.997750+0000","last_peered":"2024-03-12T17:08:12.997750+0000","last_clean":"2024-03-12T17:08:12.997750+0000","last_became_active":"2024-03-12T17:08:10.710957+0000","last_became_peered":"2024-03-12T17:08:10.710957+0000","last_unstale":"2024-03-12T17:08:12.997750+0000","last_undegraded":"2024-03-12T17:08:12.997750+0000","last_fullsized":"2024-03-12T17:08:12.997750+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:13:11.256122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.898284+0000","last_change":"2024-03-12T17:08:10.209162+0000","last_active":"2024-03-12T17:08:11.898284+0000","last_peered":"2024-03-12T17:08:11.898284+0000","last_clean":"2024-03-12T17:08:11.898284+0000","last_became_active":"2024-03-12T17:08:10.208682+0000","last_became_peered":"2024-03-12T17:08:10.208682+0000","last_unstale":"2024-03-12T17:08:11.898284+0000","last_undegraded":"2024-03-12T17:08:11.898284+0000","last_fullsized":"2024-03-12T17:08:11.898284+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:29:17.220320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.695170+0000","last_change":"2024-03-12T17:08:10.731462+0000","last_active":"2024-03-12T17:08:11.695170+0000","last_peered":"2024-03-12T17:08:11.695170+0000","last_clean":"2024-03-12T17:08:11.695170+0000","last_became_active":"2024-03-12T17:08:10.728086+0000","last_became_peered":"2024-03-12T17:08:10.728086+0000","last_unstale":"2024-03-12T17:08:11.695170+0000","last_undegraded":"2024-03-12T17:08:11.695170+0000","last_fullsized":"2024-03-12T17:08:11.695170+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:56:08.915410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.997826+0000","last_change":"2024-03-12T17:08:10.220415+0000","last_active":"2024-03-12T17:08:12.997826+0000","last_peered":"2024-03-12T17:08:12.997826+0000","last_clean":"2024-03-12T17:08:12.997826+0000","last_became_active":"2024-03-12T17:08:10.219966+0000","last_became_peered":"2024-03-12T17:08:10.219966+0000","last_unstale":"2024-03-12T17:08:12.997826+0000","last_undegraded":"2024-03-12T17:08:12.997826+0000","last_fullsized":"2024-03-12T17:08:12.997826+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:19:39.777696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.691461+0000","last_change":"2024-03-12T17:08:10.719305+0000","last_active":"2024-03-12T17:08:11.691461+0000","last_peered":"2024-03-12T17:08:11.691461+0000","last_clean":"2024-03-12T17:08:11.691461+0000","last_became_active":"2024-03-12T17:08:10.716803+0000","last_became_peered":"2024-03-12T17:08:10.716803+0000","last_unstale":"2024-03-12T17:08:11.691461+0000","last_undegraded":"2024-03-12T17:08:11.691461+0000","last_fullsized":"2024-03-12T17:08:11.691461+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:21:26.723972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.695310+0000","last_change":"2024-03-12T17:08:10.220844+0000","last_active":"2024-03-12T17:08:11.695310+0000","last_peered":"2024-03-12T17:08:11.695310+0000","last_clean":"2024-03-12T17:08:11.695310+0000","last_became_active":"2024-03-12T17:08:10.218208+0000","last_became_peered":"2024-03-12T17:08:10.218208+0000","last_unstale":"2024-03-12T17:08:11.695310+0000","last_undegraded":"2024-03-12T17:08:11.695310+0000","last_fullsized":"2024-03-12T17:08:11.695310+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:06:31.032998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.897669+0000","last_change":"2024-03-12T17:08:10.700350+0000","last_active":"2024-03-12T17:08:11.897669+0000","last_peered":"2024-03-12T17:08:11.897669+0000","last_clean":"2024-03-12T17:08:11.897669+0000","last_became_active":"2024-03-12T17:08:10.700059+0000","last_became_peered":"2024-03-12T17:08:10.700059+0000","last_unstale":"2024-03-12T17:08:11.897669+0000","last_undegraded":"2024-03-12T17:08:11.897669+0000","last_fullsized":"2024-03-12T17:08:11.897669+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:13:11.256122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.694737+0000","last_change":"2024-03-12T17:08:10.214240+0000","last_active":"2024-03-12T17:08:11.694737+0000","last_peered":"2024-03-12T17:08:11.694737+0000","last_clean":"2024-03-12T17:08:11.694737+0000","last_became_active":"2024-03-12T17:08:10.213955+0000","last_became_peered":"2024-03-12T17:08:10.213955+0000","last_unstale":"2024-03-12T17:08:11.694737+0000","last_undegraded":"2024-03-12T17:08:11.694737+0000","last_fullsized":"2024-03-12T17:08:11.694737+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:19:39.777696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.695504+0000","last_change":"2024-03-12T17:08:10.710702+0000","last_active":"2024-03-12T17:08:11.695504+0000","last_peered":"2024-03-12T17:08:11.695504+0000","last_clean":"2024-03-12T17:08:11.695504+0000","last_became_active":"2024-03-12T17:08:10.705875+0000","last_became_peered":"2024-03-12T17:08:10.705875+0000","last_unstale":"2024-03-12T17:08:11.695504+0000","last_undegraded":"2024-03-12T17:08:11.695504+0000","last_fullsized":"2024-03-12T17:08:11.695504+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:27:10.880424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.998824+0000","last_change":"2024-03-12T17:08:10.218076+0000","last_active":"2024-03-12T17:08:12.998824+0000","last_peered":"2024-03-12T17:08:12.998824+0000","last_clean":"2024-03-12T17:08:12.998824+0000","last_became_active":"2024-03-12T17:08:10.217784+0000","last_became_peered":"2024-03-12T17:08:10.217784+0000","last_unstale":"2024-03-12T17:08:12.998824+0000","last_undegraded":"2024-03-12T17:08:12.998824+0000","last_fullsized":"2024-03-12T17:08:12.998824+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:09:20.404327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.696005+0000","last_change":"2024-03-12T17:08:10.723419+0000","last_active":"2024-03-12T17:08:11.696005+0000","last_peered":"2024-03-12T17:08:11.696005+0000","last_clean":"2024-03-12T17:08:11.696005+0000","last_became_active":"2024-03-12T17:08:10.723156+0000","last_became_peered":"2024-03-12T17:08:10.723156+0000","last_unstale":"2024-03-12T17:08:11.696005+0000","last_undegraded":"2024-03-12T17:08:11.696005+0000","last_fullsized":"2024-03-12T17:08:11.696005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:26:50.546066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.997460+0000","last_change":"2024-03-12T17:08:10.220396+0000","last_active":"2024-03-12T17:08:12.997460+0000","last_peered":"2024-03-12T17:08:12.997460+0000","last_clean":"2024-03-12T17:08:12.997460+0000","last_became_active":"2024-03-12T17:08:10.220068+0000","last_became_peered":"2024-03-12T17:08:10.220068+0000","last_unstale":"2024-03-12T17:08:12.997460+0000","last_undegraded":"2024-03-12T17:08:12.997460+0000","last_fullsized":"2024-03-12T17:08:12.997460+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T20:40:15.944188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.690410+0000","last_change":"2024-03-12T17:08:10.728470+0000","last_active":"2024-03-12T17:08:11.690410+0000","last_peered":"2024-03-12T17:08:11.690410+0000","last_clean":"2024-03-12T17:08:11.690410+0000","last_became_active":"2024-03-12T17:08:10.728172+0000","last_became_peered":"2024-03-12T17:08:10.728172+0000","last_unstale":"2024-03-12T17:08:11.690410+0000","last_undegraded":"2024-03-12T17:08:11.690410+0000","last_fullsized":"2024-03-12T17:08:11.690410+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:39:34.128499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.693989+0000","last_change":"2024-03-12T17:08:10.217263+0000","last_active":"2024-03-12T17:08:11.693989+0000","last_peered":"2024-03-12T17:08:11.693989+0000","last_clean":"2024-03-12T17:08:11.693989+0000","last_became_active":"2024-03-12T17:08:10.215937+0000","last_became_peered":"2024-03-12T17:08:10.215937+0000","last_unstale":"2024-03-12T17:08:11.693989+0000","last_undegraded":"2024-03-12T17:08:11.693989+0000","last_fullsized":"2024-03-12T17:08:11.693989+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T02:54:28.103306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.479883+0000","last_change":"2024-03-12T17:08:10.729889+0000","last_active":"2024-03-12T17:08:12.479883+0000","last_peered":"2024-03-12T17:08:12.479883+0000","last_clean":"2024-03-12T17:08:12.479883+0000","last_became_active":"2024-03-12T17:08:10.729600+0000","last_became_peered":"2024-03-12T17:08:10.729600+0000","last_unstale":"2024-03-12T17:08:12.479883+0000","last_undegraded":"2024-03-12T17:08:12.479883+0000","last_fullsized":"2024-03-12T17:08:12.479883+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:34:31.313582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.691641+0000","last_change":"2024-03-12T17:08:10.214927+0000","last_active":"2024-03-12T17:08:11.691641+0000","last_peered":"2024-03-12T17:08:11.691641+0000","last_clean":"2024-03-12T17:08:11.691641+0000","last_became_active":"2024-03-12T17:08:10.214638+0000","last_became_peered":"2024-03-12T17:08:10.214638+0000","last_unstale":"2024-03-12T17:08:11.691641+0000","last_undegraded":"2024-03-12T17:08:11.691641+0000","last_fullsized":"2024-03-12T17:08:11.691641+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:39:57.420626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.999058+0000","last_change":"2024-03-12T17:08:10.734809+0000","last_active":"2024-03-12T17:08:12.999058+0000","last_peered":"2024-03-12T17:08:12.999058+0000","last_clean":"2024-03-12T17:08:12.999058+0000","last_became_active":"2024-03-12T17:08:10.734568+0000","last_became_peered":"2024-03-12T17:08:10.734568+0000","last_unstale":"2024-03-12T17:08:12.999058+0000","last_undegraded":"2024-03-12T17:08:12.999058+0000","last_fullsized":"2024-03-12T17:08:12.999058+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:31:56.609569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.696405+0000","last_change":"2024-03-12T17:08:10.224618+0000","last_active":"2024-03-12T17:08:11.696405+0000","last_peered":"2024-03-12T17:08:11.696405+0000","last_clean":"2024-03-12T17:08:11.696405+0000","last_became_active":"2024-03-12T17:08:10.224322+0000","last_became_peered":"2024-03-12T17:08:10.224322+0000","last_unstale":"2024-03-12T17:08:11.696405+0000","last_undegraded":"2024-03-12T17:08:11.696405+0000","last_fullsized":"2024-03-12T17:08:11.696405+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:21:28.144885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.898032+0000","last_change":"2024-03-12T17:08:10.737489+0000","last_active":"2024-03-12T17:08:11.898032+0000","last_peered":"2024-03-12T17:08:11.898032+0000","last_clean":"2024-03-12T17:08:11.898032+0000","last_became_active":"2024-03-12T17:08:10.737097+0000","last_became_peered":"2024-03-12T17:08:10.737097+0000","last_unstale":"2024-03-12T17:08:11.898032+0000","last_undegraded":"2024-03-12T17:08:11.898032+0000","last_fullsized":"2024-03-12T17:08:11.898032+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:44:36.258866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.695783+0000","last_change":"2024-03-12T17:08:10.223815+0000","last_active":"2024-03-12T17:08:11.695783+0000","last_peered":"2024-03-12T17:08:11.695783+0000","last_clean":"2024-03-12T17:08:11.695783+0000","last_became_active":"2024-03-12T17:08:10.223501+0000","last_became_peered":"2024-03-12T17:08:10.223501+0000","last_unstale":"2024-03-12T17:08:11.695783+0000","last_undegraded":"2024-03-12T17:08:11.695783+0000","last_fullsized":"2024-03-12T17:08:11.695783+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:29:17.220320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.695156+0000","last_change":"2024-03-12T17:08:10.731336+0000","last_active":"2024-03-12T17:08:11.695156+0000","last_peered":"2024-03-12T17:08:11.695156+0000","last_clean":"2024-03-12T17:08:11.695156+0000","last_became_active":"2024-03-12T17:08:10.730543+0000","last_became_peered":"2024-03-12T17:08:10.730543+0000","last_unstale":"2024-03-12T17:08:11.695156+0000","last_undegraded":"2024-03-12T17:08:11.695156+0000","last_fullsized":"2024-03-12T17:08:11.695156+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:27:10.880424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.691196+0000","last_change":"2024-03-12T17:08:10.212100+0000","last_active":"2024-03-12T17:08:11.691196+0000","last_peered":"2024-03-12T17:08:11.691196+0000","last_clean":"2024-03-12T17:08:11.691196+0000","last_became_active":"2024-03-12T17:08:10.209780+0000","last_became_peered":"2024-03-12T17:08:10.209780+0000","last_unstale":"2024-03-12T17:08:11.691196+0000","last_undegraded":"2024-03-12T17:08:11.691196+0000","last_fullsized":"2024-03-12T17:08:11.691196+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:25:31.957988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.897713+0000","last_change":"2024-03-12T17:08:10.708692+0000","last_active":"2024-03-12T17:08:11.897713+0000","last_peered":"2024-03-12T17:08:11.897713+0000","last_clean":"2024-03-12T17:08:11.897713+0000","last_became_active":"2024-03-12T17:08:10.708377+0000","last_became_peered":"2024-03-12T17:08:10.708377+0000","last_unstale":"2024-03-12T17:08:11.897713+0000","last_undegraded":"2024-03-12T17:08:11.897713+0000","last_fullsized":"2024-03-12T17:08:11.897713+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:18:33.858639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.690931+0000","last_change":"2024-03-12T17:08:10.207696+0000","last_active":"2024-03-12T17:08:11.690931+0000","last_peered":"2024-03-12T17:08:11.690931+0000","last_clean":"2024-03-12T17:08:11.690931+0000","last_became_active":"2024-03-12T17:08:10.207049+0000","last_became_peered":"2024-03-12T17:08:10.207049+0000","last_unstale":"2024-03-12T17:08:11.690931+0000","last_undegraded":"2024-03-12T17:08:11.690931+0000","last_fullsized":"2024-03-12T17:08:11.690931+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T02:54:28.103306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.690401+0000","last_change":"2024-03-12T17:08:10.702871+0000","last_active":"2024-03-12T17:08:11.690401+0000","last_peered":"2024-03-12T17:08:11.690401+0000","last_clean":"2024-03-12T17:08:11.690401+0000","last_became_active":"2024-03-12T17:08:10.702594+0000","last_became_peered":"2024-03-12T17:08:10.702594+0000","last_unstale":"2024-03-12T17:08:11.690401+0000","last_undegraded":"2024-03-12T17:08:11.690401+0000","last_fullsized":"2024-03-12T17:08:11.690401+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:27:10.880424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.997569+0000","last_change":"2024-03-12T17:08:10.225233+0000","last_active":"2024-03-12T17:08:12.997569+0000","last_peered":"2024-03-12T17:08:12.997569+0000","last_clean":"2024-03-12T17:08:12.997569+0000","last_became_active":"2024-03-12T17:08:10.224937+0000","last_became_peered":"2024-03-12T17:08:10.224937+0000","last_unstale":"2024-03-12T17:08:12.997569+0000","last_undegraded":"2024-03-12T17:08:12.997569+0000","last_fullsized":"2024-03-12T17:08:12.997569+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:29:17.220320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.998497+0000","last_change":"2024-03-12T17:08:10.728427+0000","last_active":"2024-03-12T17:08:12.998497+0000","last_peered":"2024-03-12T17:08:12.998497+0000","last_clean":"2024-03-12T17:08:12.998497+0000","last_became_active":"2024-03-12T17:08:10.726613+0000","last_became_peered":"2024-03-12T17:08:10.726613+0000","last_unstale":"2024-03-12T17:08:12.998497+0000","last_undegraded":"2024-03-12T17:08:12.998497+0000","last_fullsized":"2024-03-12T17:08:12.998497+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:13:11.256122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.833793+0000","last_change":"2024-03-12T17:08:10.212226+0000","last_active":"2024-03-12T17:08:11.833793+0000","last_peered":"2024-03-12T17:08:11.833793+0000","last_clean":"2024-03-12T17:08:11.833793+0000","last_became_active":"2024-03-12T17:08:10.211929+0000","last_became_peered":"2024-03-12T17:08:10.211929+0000","last_unstale":"2024-03-12T17:08:11.833793+0000","last_undegraded":"2024-03-12T17:08:11.833793+0000","last_fullsized":"2024-03-12T17:08:11.833793+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:22:07.879324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.690710+0000","last_change":"2024-03-12T17:08:10.708949+0000","last_active":"2024-03-12T17:08:11.690710+0000","last_peered":"2024-03-12T17:08:11.690710+0000","last_clean":"2024-03-12T17:08:11.690710+0000","last_became_active":"2024-03-12T17:08:10.708684+0000","last_became_peered":"2024-03-12T17:08:10.708684+0000","last_unstale":"2024-03-12T17:08:11.690710+0000","last_undegraded":"2024-03-12T17:08:11.690710+0000","last_fullsized":"2024-03-12T17:08:11.690710+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:33:01.223233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":19,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.998405+0000","last_change":"2024-03-12T17:08:10.212353+0000","last_active":"2024-03-12T17:08:12.998405+0000","last_peered":"2024-03-12T17:08:12.998405+0000","last_clean":"2024-03-12T17:08:12.998405+0000","last_became_active":"2024-03-12T17:08:10.212069+0000","last_became_peered":"2024-03-12T17:08:10.212069+0000","last_unstale":"2024-03-12T17:08:12.998405+0000","last_undegraded":"2024-03-12T17:08:12.998405+0000","last_fullsized":"2024-03-12T17:08:12.998405+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:52:00.006743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.691140+0000","last_change":"2024-03-12T17:08:10.714647+0000","last_active":"2024-03-12T17:08:11.691140+0000","last_peered":"2024-03-12T17:08:11.691140+0000","last_clean":"2024-03-12T17:08:11.691140+0000","last_became_active":"2024-03-12T17:08:10.713543+0000","last_became_peered":"2024-03-12T17:08:10.713543+0000","last_unstale":"2024-03-12T17:08:11.691140+0000","last_undegraded":"2024-03-12T17:08:11.691140+0000","last_fullsized":"2024-03-12T17:08:11.691140+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:28:01.785461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.696335+0000","last_change":"2024-03-12T17:08:10.222123+0000","last_active":"2024-03-12T17:08:11.696335+0000","last_peered":"2024-03-12T17:08:11.696335+0000","last_clean":"2024-03-12T17:08:11.696335+0000","last_became_active":"2024-03-12T17:08:10.221168+0000","last_became_peered":"2024-03-12T17:08:10.221168+0000","last_unstale":"2024-03-12T17:08:11.696335+0000","last_undegraded":"2024-03-12T17:08:11.696335+0000","last_fullsized":"2024-03-12T17:08:11.696335+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:29:58.170701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.695665+0000","last_change":"2024-03-12T17:08:10.704614+0000","last_active":"2024-03-12T17:08:11.695665+0000","last_peered":"2024-03-12T17:08:11.695665+0000","last_clean":"2024-03-12T17:08:11.695665+0000","last_became_active":"2024-03-12T17:08:10.704243+0000","last_became_peered":"2024-03-12T17:08:10.704243+0000","last_unstale":"2024-03-12T17:08:11.695665+0000","last_undegraded":"2024-03-12T17:08:11.695665+0000","last_fullsized":"2024-03-12T17:08:11.695665+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:51:16.461818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.695431+0000","last_change":"2024-03-12T17:08:10.220809+0000","last_active":"2024-03-12T17:08:11.695431+0000","last_peered":"2024-03-12T17:08:11.695431+0000","last_clean":"2024-03-12T17:08:11.695431+0000","last_became_active":"2024-03-12T17:08:10.220495+0000","last_became_peered":"2024-03-12T17:08:10.220495+0000","last_unstale":"2024-03-12T17:08:11.695431+0000","last_undegraded":"2024-03-12T17:08:11.695431+0000","last_fullsized":"2024-03-12T17:08:11.695431+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:25:31.957988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.694940+0000","last_change":"2024-03-12T17:08:10.732113+0000","last_active":"2024-03-12T17:08:11.694940+0000","last_peered":"2024-03-12T17:08:11.694940+0000","last_clean":"2024-03-12T17:08:11.694940+0000","last_became_active":"2024-03-12T17:08:10.731823+0000","last_became_peered":"2024-03-12T17:08:10.731823+0000","last_unstale":"2024-03-12T17:08:11.694940+0000","last_undegraded":"2024-03-12T17:08:11.694940+0000","last_fullsized":"2024-03-12T17:08:11.694940+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:23:32.915382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.480191+0000","last_change":"2024-03-12T17:08:10.208921+0000","last_active":"2024-03-12T17:08:12.480191+0000","last_peered":"2024-03-12T17:08:12.480191+0000","last_clean":"2024-03-12T17:08:12.480191+0000","last_became_active":"2024-03-12T17:08:10.208638+0000","last_became_peered":"2024-03-12T17:08:10.208638+0000","last_unstale":"2024-03-12T17:08:12.480191+0000","last_undegraded":"2024-03-12T17:08:12.480191+0000","last_fullsized":"2024-03-12T17:08:12.480191+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:39:11.129967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.479915+0000","last_change":"2024-03-12T17:08:10.709839+0000","last_active":"2024-03-12T17:08:12.479915+0000","last_peered":"2024-03-12T17:08:12.479915+0000","last_clean":"2024-03-12T17:08:12.479915+0000","last_became_active":"2024-03-12T17:08:10.709514+0000","last_became_peered":"2024-03-12T17:08:10.709514+0000","last_unstale":"2024-03-12T17:08:12.479915+0000","last_undegraded":"2024-03-12T17:08:12.479915+0000","last_fullsized":"2024-03-12T17:08:12.479915+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:25:25.930906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.997786+0000","last_change":"2024-03-12T17:08:10.219600+0000","last_active":"2024-03-12T17:08:12.997786+0000","last_peered":"2024-03-12T17:08:12.997786+0000","last_clean":"2024-03-12T17:08:12.997786+0000","last_became_active":"2024-03-12T17:08:10.219314+0000","last_became_peered":"2024-03-12T17:08:10.219314+0000","last_unstale":"2024-03-12T17:08:12.997786+0000","last_undegraded":"2024-03-12T17:08:12.997786+0000","last_fullsized":"2024-03-12T17:08:12.997786+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:29:58.170701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.998422+0000","last_change":"2024-03-12T17:08:10.728976+0000","last_active":"2024-03-12T17:08:12.998422+0000","last_peered":"2024-03-12T17:08:12.998422+0000","last_clean":"2024-03-12T17:08:12.998422+0000","last_became_active":"2024-03-12T17:08:10.728612+0000","last_became_peered":"2024-03-12T17:08:10.728612+0000","last_unstale":"2024-03-12T17:08:12.998422+0000","last_undegraded":"2024-03-12T17:08:12.998422+0000","last_fullsized":"2024-03-12T17:08:12.998422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:44:36.258866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.479528+0000","last_change":"2024-03-12T17:08:10.204338+0000","last_active":"2024-03-12T17:08:12.479528+0000","last_peered":"2024-03-12T17:08:12.479528+0000","last_clean":"2024-03-12T17:08:12.479528+0000","last_became_active":"2024-03-12T17:08:10.203258+0000","last_became_peered":"2024-03-12T17:08:10.203258+0000","last_unstale":"2024-03-12T17:08:12.479528+0000","last_undegraded":"2024-03-12T17:08:12.479528+0000","last_fullsized":"2024-03-12T17:08:12.479528+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:43:43.249201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.897868+0000","last_change":"2024-03-12T17:08:10.727663+0000","last_active":"2024-03-12T17:08:11.897868+0000","last_peered":"2024-03-12T17:08:11.897868+0000","last_clean":"2024-03-12T17:08:11.897868+0000","last_became_active":"2024-03-12T17:08:10.727257+0000","last_became_peered":"2024-03-12T17:08:10.727257+0000","last_unstale":"2024-03-12T17:08:11.897868+0000","last_undegraded":"2024-03-12T17:08:11.897868+0000","last_fullsized":"2024-03-12T17:08:11.897868+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:09:44.012820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.479312+0000","last_change":"2024-03-12T17:08:10.217052+0000","last_active":"2024-03-12T17:08:12.479312+0000","last_peered":"2024-03-12T17:08:12.479312+0000","last_clean":"2024-03-12T17:08:12.479312+0000","last_became_active":"2024-03-12T17:08:10.215880+0000","last_became_peered":"2024-03-12T17:08:10.215880+0000","last_unstale":"2024-03-12T17:08:12.479312+0000","last_undegraded":"2024-03-12T17:08:12.479312+0000","last_fullsized":"2024-03-12T17:08:12.479312+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:52:00.006743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.695166+0000","last_change":"2024-03-12T17:08:10.718800+0000","last_active":"2024-03-12T17:08:11.695166+0000","last_peered":"2024-03-12T17:08:11.695166+0000","last_clean":"2024-03-12T17:08:11.695166+0000","last_became_active":"2024-03-12T17:08:10.713494+0000","last_became_peered":"2024-03-12T17:08:10.713494+0000","last_unstale":"2024-03-12T17:08:11.695166+0000","last_undegraded":"2024-03-12T17:08:11.695166+0000","last_fullsized":"2024-03-12T17:08:11.695166+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:56:08.915410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.998534+0000","last_change":"2024-03-12T17:08:10.217017+0000","last_active":"2024-03-12T17:08:12.998534+0000","last_peered":"2024-03-12T17:08:12.998534+0000","last_clean":"2024-03-12T17:08:12.998534+0000","last_became_active":"2024-03-12T17:08:10.215244+0000","last_became_peered":"2024-03-12T17:08:10.215244+0000","last_unstale":"2024-03-12T17:08:12.998534+0000","last_undegraded":"2024-03-12T17:08:12.998534+0000","last_fullsized":"2024-03-12T17:08:12.998534+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:00:06.057527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.690683+0000","last_change":"2024-03-12T17:08:10.728322+0000","last_active":"2024-03-12T17:08:11.690683+0000","last_peered":"2024-03-12T17:08:11.690683+0000","last_clean":"2024-03-12T17:08:11.690683+0000","last_became_active":"2024-03-12T17:08:10.727874+0000","last_became_peered":"2024-03-12T17:08:10.727874+0000","last_unstale":"2024-03-12T17:08:11.690683+0000","last_undegraded":"2024-03-12T17:08:11.690683+0000","last_fullsized":"2024-03-12T17:08:11.690683+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:16:52.065023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.479396+0000","last_change":"2024-03-12T17:08:10.203965+0000","last_active":"2024-03-12T17:08:12.479396+0000","last_peered":"2024-03-12T17:08:12.479396+0000","last_clean":"2024-03-12T17:08:12.479396+0000","last_became_active":"2024-03-12T17:08:10.203610+0000","last_became_peered":"2024-03-12T17:08:10.203610+0000","last_unstale":"2024-03-12T17:08:12.479396+0000","last_undegraded":"2024-03-12T17:08:12.479396+0000","last_fullsized":"2024-03-12T17:08:12.479396+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:00:06.057527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.997665+0000","last_change":"2024-03-12T17:08:10.711425+0000","last_active":"2024-03-12T17:08:12.997665+0000","last_peered":"2024-03-12T17:08:12.997665+0000","last_clean":"2024-03-12T17:08:12.997665+0000","last_became_active":"2024-03-12T17:08:10.711171+0000","last_became_peered":"2024-03-12T17:08:10.711171+0000","last_unstale":"2024-03-12T17:08:12.997665+0000","last_undegraded":"2024-03-12T17:08:12.997665+0000","last_fullsized":"2024-03-12T17:08:12.997665+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:18:33.858639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.696402+0000","last_change":"2024-03-12T17:08:10.222524+0000","last_active":"2024-03-12T17:08:11.696402+0000","last_peered":"2024-03-12T17:08:11.696402+0000","last_clean":"2024-03-12T17:08:11.696402+0000","last_became_active":"2024-03-12T17:08:10.221941+0000","last_became_peered":"2024-03-12T17:08:10.221941+0000","last_unstale":"2024-03-12T17:08:11.696402+0000","last_undegraded":"2024-03-12T17:08:11.696402+0000","last_fullsized":"2024-03-12T17:08:11.696402+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T20:40:15.944188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.999156+0000","last_change":"2024-03-12T17:08:10.734848+0000","last_active":"2024-03-12T17:08:12.999156+0000","last_peered":"2024-03-12T17:08:12.999156+0000","last_clean":"2024-03-12T17:08:12.999156+0000","last_became_active":"2024-03-12T17:08:10.734623+0000","last_became_peered":"2024-03-12T17:08:10.734623+0000","last_unstale":"2024-03-12T17:08:12.999156+0000","last_undegraded":"2024-03-12T17:08:12.999156+0000","last_fullsized":"2024-03-12T17:08:12.999156+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T04:17:24.644009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"40'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.483833+0000","last_change":"2024-03-12T17:08:10.224510+0000","last_active":"2024-03-12T17:08:12.483833+0000","last_peered":"2024-03-12T17:08:12.483833+0000","last_clean":"2024-03-12T17:08:12.483833+0000","last_became_active":"2024-03-12T17:08:10.224192+0000","last_became_peered":"2024-03-12T17:08:10.224192+0000","last_unstale":"2024-03-12T17:08:12.483833+0000","last_undegraded":"2024-03-12T17:08:12.483833+0000","last_fullsized":"2024-03-12T17:08:12.483833+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:35:52.711369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.690765+0000","last_change":"2024-03-12T17:08:10.730592+0000","last_active":"2024-03-12T17:08:11.690765+0000","last_peered":"2024-03-12T17:08:11.690765+0000","last_clean":"2024-03-12T17:08:11.690765+0000","last_became_active":"2024-03-12T17:08:10.729663+0000","last_became_peered":"2024-03-12T17:08:10.729663+0000","last_unstale":"2024-03-12T17:08:11.690765+0000","last_undegraded":"2024-03-12T17:08:11.690765+0000","last_fullsized":"2024-03-12T17:08:11.690765+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:52:53.108451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.898348+0000","last_change":"2024-03-12T17:08:10.212568+0000","last_active":"2024-03-12T17:08:11.898348+0000","last_peered":"2024-03-12T17:08:11.898348+0000","last_clean":"2024-03-12T17:08:11.898348+0000","last_became_active":"2024-03-12T17:08:10.212285+0000","last_became_peered":"2024-03-12T17:08:10.212285+0000","last_unstale":"2024-03-12T17:08:11.898348+0000","last_undegraded":"2024-03-12T17:08:11.898348+0000","last_fullsized":"2024-03-12T17:08:11.898348+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:21:28.144885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.897924+0000","last_change":"2024-03-12T17:08:10.702162+0000","last_active":"2024-03-12T17:08:11.897924+0000","last_peered":"2024-03-12T17:08:11.897924+0000","last_clean":"2024-03-12T17:08:11.897924+0000","last_became_active":"2024-03-12T17:08:10.701911+0000","last_became_peered":"2024-03-12T17:08:10.701911+0000","last_unstale":"2024-03-12T17:08:11.897924+0000","last_undegraded":"2024-03-12T17:08:11.897924+0000","last_fullsized":"2024-03-12T17:08:11.897924+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:25:32.963315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.898163+0000","last_change":"2024-03-12T17:08:10.214326+0000","last_active":"2024-03-12T17:08:11.898163+0000","last_peered":"2024-03-12T17:08:11.898163+0000","last_clean":"2024-03-12T17:08:11.898163+0000","last_became_active":"2024-03-12T17:08:10.214015+0000","last_became_peered":"2024-03-12T17:08:10.214015+0000","last_unstale":"2024-03-12T17:08:11.898163+0000","last_undegraded":"2024-03-12T17:08:11.898163+0000","last_fullsized":"2024-03-12T17:08:11.898163+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:12:43.823882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.997321+0000","last_change":"2024-03-12T17:08:10.711888+0000","last_active":"2024-03-12T17:08:12.997321+0000","last_peered":"2024-03-12T17:08:12.997321+0000","last_clean":"2024-03-12T17:08:12.997321+0000","last_became_active":"2024-03-12T17:08:10.711638+0000","last_became_peered":"2024-03-12T17:08:10.711638+0000","last_unstale":"2024-03-12T17:08:12.997321+0000","last_undegraded":"2024-03-12T17:08:12.997321+0000","last_fullsized":"2024-03-12T17:08:12.997321+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:44:36.258866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.833905+0000","last_change":"2024-03-12T17:08:10.209448+0000","last_active":"2024-03-12T17:08:11.833905+0000","last_peered":"2024-03-12T17:08:11.833905+0000","last_clean":"2024-03-12T17:08:11.833905+0000","last_became_active":"2024-03-12T17:08:10.208888+0000","last_became_peered":"2024-03-12T17:08:10.208888+0000","last_unstale":"2024-03-12T17:08:11.833905+0000","last_undegraded":"2024-03-12T17:08:11.833905+0000","last_fullsized":"2024-03-12T17:08:11.833905+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T21:29:58.170701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.695512+0000","last_change":"2024-03-12T17:08:10.722062+0000","last_active":"2024-03-12T17:08:11.695512+0000","last_peered":"2024-03-12T17:08:11.695512+0000","last_clean":"2024-03-12T17:08:11.695512+0000","last_became_active":"2024-03-12T17:08:10.721753+0000","last_became_peered":"2024-03-12T17:08:10.721753+0000","last_unstale":"2024-03-12T17:08:11.695512+0000","last_undegraded":"2024-03-12T17:08:11.695512+0000","last_fullsized":"2024-03-12T17:08:11.695512+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T01:33:01.223233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.694540+0000","last_change":"2024-03-12T17:08:10.219399+0000","last_active":"2024-03-12T17:08:11.694540+0000","last_peered":"2024-03-12T17:08:11.694540+0000","last_clean":"2024-03-12T17:08:11.694540+0000","last_became_active":"2024-03-12T17:08:10.219119+0000","last_became_peered":"2024-03-12T17:08:10.219119+0000","last_unstale":"2024-03-12T17:08:11.694540+0000","last_undegraded":"2024-03-12T17:08:11.694540+0000","last_fullsized":"2024-03-12T17:08:11.694540+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T17:39:57.420626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.999170+0000","last_change":"2024-03-12T17:08:10.735540+0000","last_active":"2024-03-12T17:08:12.999170+0000","last_peered":"2024-03-12T17:08:12.999170+0000","last_clean":"2024-03-12T17:08:12.999170+0000","last_became_active":"2024-03-12T17:08:10.735262+0000","last_became_peered":"2024-03-12T17:08:10.735262+0000","last_unstale":"2024-03-12T17:08:12.999170+0000","last_undegraded":"2024-03-12T17:08:12.999170+0000","last_fullsized":"2024-03-12T17:08:12.999170+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T19:23:32.915382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.690178+0000","last_change":"2024-03-12T17:08:10.212008+0000","last_active":"2024-03-12T17:08:11.690178+0000","last_peered":"2024-03-12T17:08:11.690178+0000","last_clean":"2024-03-12T17:08:11.690178+0000","last_became_active":"2024-03-12T17:08:10.211720+0000","last_became_peered":"2024-03-12T17:08:10.211720+0000","last_unstale":"2024-03-12T17:08:11.690178+0000","last_undegraded":"2024-03-12T17:08:11.690178+0000","last_fullsized":"2024-03-12T17:08:11.690178+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T22:06:31.032998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:11.696015+0000","last_change":"2024-03-12T17:08:10.718404+0000","last_active":"2024-03-12T17:08:11.696015+0000","last_peered":"2024-03-12T17:08:11.696015+0000","last_clean":"2024-03-12T17:08:11.696015+0000","last_became_active":"2024-03-12T17:08:10.715811+0000","last_became_peered":"2024-03-12T17:08:10.715811+0000","last_unstale":"2024-03-12T17:08:11.696015+0000","last_undegraded":"2024-03-12T17:08:11.696015+0000","last_fullsized":"2024-03-12T17:08:11.696015+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:10.170259+0000","last_clean_scrub_stamp":"2024-03-12T17:08:10.170259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T18:26:50.546066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-12T17:08:12.998948+0000","last_change":"2024-03-12T17:08:10.213481+0000","last_active":"2024-03-12T17:08:12.998948+0000","last_peered":"2024-03-12T17:08:12.998948+0000","last_clean":"2024-03-12T17:08:12.998948+0000","last_became_active":"2024-03-12T17:08:10.213191+0000","last_became_peered":"2024-03-12T17:08:10.213191+0000","last_unstale":"2024-03-12T17:08:12.998948+0000","last_undegraded":"2024-03-12T17:08:12.998948+0000","last_fullsized":"2024-03-12T17:08:12.998948+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:08:09.164933+0000","last_clean_scrub_stamp":"2024-03-12T17:08:09.164933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T03:20:32.413027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":179,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-12T17:08:10.716662+0000","last_change":"2024-03-12T17:06:21.714813+0000","last_active":"2024-03-12T17:08:10.716662+0000","last_peered":"2024-03-12T17:08:10.716662+0000","last_clean":"2024-03-12T17:08:10.716662+0000","last_became_active":"2024-03-12T17:06:21.712488+0000","last_became_peered":"2024-03-12T17:06:21.712488+0000","last_unstale":"2024-03-12T17:08:10.716662+0000","last_undegraded":"2024-03-12T17:08:10.716662+0000","last_fullsized":"2024-03-12T17:08:10.716662+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:19.815688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:19.815688+0000","last_clean_scrub_stamp":"2024-03-12T17:06:19.815688+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:38:54.300793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28064,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343776,"statfs":{"total":96636764160,"available":96608026624,"internally_reserved":0,"allocated":1060864,"data_stored":784160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:07:22 2024","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":1.095}]},{"osd":1,"last update":"Tue Mar 12 17:07:22 2024","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.845,"5min":1.845,"15min":1.845},"last":1.379},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.2889999999999999}]},{"osd":2,"last update":"Tue Mar 12 17:07:22 2024","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.121,"15min":1.121},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.841,"5min":1.841,"15min":1.841},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.861,"5min":1.861,"15min":1.861},"last":1.214}]},{"osd":3,"last update":"Tue Mar 12 17:07:22 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.4039999999999999,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.522}]},{"osd":4,"last update":"Tue Mar 12 17:07:22 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":3.5680000000000001,"5min":3.5680000000000001,"15min":3.5680000000000001},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.464,"5min":2.464,"15min":2.464},"last":1.6140000000000001}]},{"osd":5,"last update":"Tue Mar 12 17:07:22 2024","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.474},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.5310000000000001,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.157}]},{"osd":6,"last update":"Tue Mar 12 17:07:22 2024","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.401,"15min":1.401},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.587},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":4.625,"5min":4.625,"15min":4.625},"last":1.339}]}]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":466944,"data_stored":199360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.2170000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":0.876}]},{"osd":1,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.1120000000000001}]},{"osd":2,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.177,"5min":1.177,"15min":1.177},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":1.2969999999999999}]},{"osd":3,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.419},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.8380000000000001}]},{"osd":4,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.5030000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":2.2410000000000001},{"interface":"front","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.772}]},{"osd":5,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.9670000000000001},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":2.1320000000000001,"5min":2.1320000000000001,"15min":2.1320000000000001},"last":2.1459999999999999}]},{"osd":7,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.8999999999999999},{"interface":"front","average":{"1min":1.466,"5min":1.466,"15min":1.466},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.6560000000000001,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":2.4049999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":198890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":3.6709999999999998,"5min":3.6709999999999998,"15min":3.6709999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":3.706,"5min":3.706,"15min":3.706},"last":0.55600000000000005}]},{"osd":1,"last update":"Tue Mar 12 17:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":3.0390000000000001,"5min":3.0390000000000001,"15min":3.0390000000000001},"last":0.442},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.56599999999999995}]},{"osd":2,"last update":"Tue Mar 12 17:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.498},{"interface":"front","average":{"1min":1.264,"5min":1.264,"15min":1.264},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.53500000000000003}]},{"osd":3,"last update":"Tue Mar 12 17:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":3.7440000000000002,"5min":3.7440000000000002,"15min":3.7440000000000002},"last":0.38600000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.294,"15min":1.294},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":0.58599999999999997}]},{"osd":4,"last update":"Tue Mar 12 17:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.893,"5min":1.893,"15min":1.893},"last":0.34499999999999997}]},{"osd":6,"last update":"Tue Mar 12 17:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":8.2080000000000002,"5min":8.2080000000000002,"15min":8.2080000000000002},"last":0.41099999999999998},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.51400000000000001}]},{"osd":7,"last update":"Tue Mar 12 17:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":1.859,"5min":1.859,"15min":1.859},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":8.0139999999999993,"5min":8.0139999999999993,"15min":8.0139999999999993},"last":0.46600000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":199382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.724,"5min":1.724,"15min":1.724},"last":1.0489999999999999}]},{"osd":1,"last update":"Tue Mar 12 17:07:17 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":2.1309999999999998,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.639}]},{"osd":2,"last update":"Tue Mar 12 17:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.224,"15min":1.224},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":1.982,"5min":1.982,"15min":1.982},"last":1.141},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.1640000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":0.97499999999999998}]},{"osd":3,"last update":"Tue Mar 12 17:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.177,"15min":1.177},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":1.399},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.444}]},{"osd":5,"last update":"Tue Mar 12 17:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":2.2589999999999999,"5min":2.2589999999999999,"15min":2.2589999999999999},"last":1.762},{"interface":"front","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":1.5069999999999999}]},{"osd":6,"last update":"Tue Mar 12 17:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.385,"15min":1.385},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.3559999999999999,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":1.831}]},{"osd":7,"last update":"Tue Mar 12 17:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.425,"15min":1.425},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.0619999999999998,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":1.601},{"interface":"front","average":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":3.5720000000000001,"5min":3.5720000000000001,"15min":3.5720000000000001},"last":1.6879999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31592,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340248,"statfs":{"total":96636764160,"available":96604413952,"internally_reserved":0,"allocated":479232,"data_stored":200376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":5.0039999999999996,"5min":5.0039999999999996,"15min":5.0039999999999996},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.0449999999999999,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":1.1639999999999999}]},{"osd":1,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":4.9080000000000004,"5min":4.9080000000000004,"15min":4.9080000000000004},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.2410000000000001,"5min":3.2410000000000001,"15min":3.2410000000000001},"last":1.355}]},{"osd":2,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":3.4990000000000001,"5min":3.4990000000000001,"15min":3.4990000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":5.1870000000000003,"5min":5.1870000000000003,"15min":5.1870000000000003},"last":1.6679999999999999}]},{"osd":4,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":9.9299999999999997,"5min":9.9299999999999997,"15min":9.9299999999999997},"last":1.008},{"interface":"front","average":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":5.2450000000000001,"5min":5.2450000000000001,"15min":5.2450000000000001},"last":1.5740000000000001}]},{"osd":5,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.369,"15min":1.369},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.9550000000000001,"5min":2.9550000000000001,"15min":2.9550000000000001},"last":1.048},{"interface":"front","average":{"1min":1.589,"5min":1.589,"15min":1.589},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":3.089,"5min":3.089,"15min":3.089},"last":1.26}]},{"osd":6,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":3.3170000000000002,"5min":3.3170000000000002,"15min":3.3170000000000002},"last":1.732},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":2.722,"5min":2.722,"15min":2.722},"last":1.1439999999999999}]},{"osd":7,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.7559999999999998,"5min":2.7559999999999998,"15min":2.7559999999999998},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":3.1389999999999998,"5min":3.1389999999999998,"15min":3.1389999999999998},"last":1.4019999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":466944,"data_stored":198848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:07:18 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.39400000000000002},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":3.3879999999999999,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":0.94099999999999995}]},{"osd":1,"last update":"Tue Mar 12 17:07:18 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.7639999999999998,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":0.86499999999999999}]},{"osd":3,"last update":"Tue Mar 12 17:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.46200000000000002}]},{"osd":4,"last update":"Tue Mar 12 17:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.26,"15min":1.26},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":2.9569999999999999,"5min":2.9569999999999999,"15min":2.9569999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.46,"5min":1.46,"15min":1.46},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":4.4640000000000004,"5min":4.4640000000000004,"15min":4.4640000000000004},"last":1.093}]},{"osd":5,"last update":"Tue Mar 12 17:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.8069999999999999,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":1.018}]},{"osd":6,"last update":"Tue Mar 12 17:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.502,"15min":1.502},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":3.5179999999999998,"5min":3.5179999999999998,"15min":3.5179999999999998},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.492,"5min":1.492,"15min":1.492},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.464,"5min":2.464,"15min":2.464},"last":0.92900000000000005}]},{"osd":7,"last update":"Tue Mar 12 17:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":3.6110000000000002,"5min":3.6110000000000002,"15min":3.6110000000000002},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.571,"5min":1.571,"15min":1.571},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.8730000000000002,"5min":2.8730000000000002,"15min":2.8730000000000002},"last":0.64800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":454656,"data_stored":192248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":1.026},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":3.6419999999999999,"5min":3.6419999999999999,"15min":3.6419999999999999},"last":2.1459999999999999}]},{"osd":2,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.5049999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.84699999999999998}]},{"osd":3,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":3.9580000000000002,"5min":3.9580000000000002,"15min":3.9580000000000002},"last":1.119},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.8,"5min":1.8,"15min":1.8},"last":1.1579999999999999}]},{"osd":4,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.694,"5min":2.694,"15min":2.694},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":3.5150000000000001,"5min":3.5150000000000001,"15min":3.5150000000000001},"last":1.2430000000000001}]},{"osd":5,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":1.23},{"interface":"front","average":{"1min":1.536,"5min":1.536,"15min":1.536},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.7709999999999999,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":1.873}]},{"osd":6,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":4.1269999999999998,"5min":4.1269999999999998,"15min":4.1269999999999998},"last":1.391},{"interface":"front","average":{"1min":1.49,"5min":1.49,"15min":1.49},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":3.7850000000000001,"5min":3.7850000000000001,"15min":3.7850000000000001},"last":1.0800000000000001}]},{"osd":7,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":4.0030000000000001,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.357},{"interface":"front","average":{"1min":1.623,"5min":1.623,"15min":1.623},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.8660000000000001,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.9930000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1060864,"data_stored":789180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.71299999999999997}]},{"osd":2,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":0.67600000000000005},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":0.63100000000000001}]},{"osd":3,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":1.139},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.091}]},{"osd":4,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.403,"15min":1.403},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":3.0499999999999998,"5min":3.0499999999999998,"15min":3.0499999999999998},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":2.8319999999999999,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":1.2470000000000001}]},{"osd":5,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":2.2850000000000001,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.45}]},{"osd":6,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"min":{"1min":0.999,"5min":0.999,"15min":0.999},"max":{"1min":2.597,"5min":2.597,"15min":2.597},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"min":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"max":{"1min":2.452,"5min":2.452,"15min":2.452},"last":1.038}]},{"osd":7,"last update":"Tue Mar 12 17:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"min":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"max":{"1min":2.7240000000000002,"5min":2.7240000000000002,"15min":2.7240000000000002},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.617,"5min":1.617,"15min":1.617},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.6760000000000002,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":1.2789999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-12T17:08:16.153 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-12T17:08:16.153 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-12T17:08:16.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.435+0000 7f4452200640 1 Processor -- start 2024-03-12T17:08:16.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.435+0000 7f4452200640 1 -- start start 2024-03-12T17:08:16.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.436+0000 7f4452200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f444c0cc200 0x7f444c0cab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:16.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.436+0000 7f4452200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f444c0cb070 0x7f444c15a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:16.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.436+0000 7f4452200640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f444c0c8c90 con 0x7f444c159cd0 2024-03-12T17:08:16.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.436+0000 7f4452200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f444c0cc6c0 con 0x7f444c0cb070 2024-03-12T17:08:16.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.436+0000 7f4452200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f444c0cc9a0 con 0x7f444c0cc200 2024-03-12T17:08:16.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.437+0000 7f4450e00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f444c159cd0 0x7f444c0ad260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:34646/0 (socket says 172.21.15.52:34646) 2024-03-12T17:08:16.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.437+0000 7f4450e00640 1 -- 172.21.15.52:0/1013081893 learned_addr learned my addr 172.21.15.52:0/1013081893 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:08:16.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.437+0000 7f4451800640 1 --2- 172.21.15.52:0/1013081893 >> v2:172.21.15.52:3300/0 conn(0x7f444c0cb070 0x7f444c15a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:16.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.437+0000 7f444b400640 1 -- 172.21.15.52:0/1013081893 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3749921840 0 0) 0x7f444c0c8c90 con 0x7f444c159cd0 2024-03-12T17:08:16.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.437+0000 7f444b400640 1 -- 172.21.15.52:0/1013081893 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4434003520 con 0x7f444c159cd0 2024-03-12T17:08:16.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.437+0000 7f4451800640 1 -- 172.21.15.52:0/1013081893 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f444c0cc200 msgr2=0x7f444c0cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:16.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.437+0000 7f444be00640 1 --2- 172.21.15.52:0/1013081893 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f444c0cc200 0x7f444c0cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:16.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.437+0000 7f4451800640 1 --2- 172.21.15.52:0/1013081893 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f444c0cc200 0x7f444c0cab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:16.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.437+0000 7f4451800640 1 -- 172.21.15.52:0/1013081893 >> v1:172.21.15.52:6789/0 conn(0x7f444c159cd0 legacy=0x7f444c0ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:16.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.437+0000 7f4451800640 1 -- 172.21.15.52:0/1013081893 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444c0ccc80 con 0x7f444c0cb070 2024-03-12T17:08:16.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.437+0000 7f444be00640 1 --2- 172.21.15.52:0/1013081893 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f444c0cc200 0x7f444c0cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:08:16.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.439+0000 7f4451800640 1 --2- 172.21.15.52:0/1013081893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f444c0cb070 0x7f444c15a0b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4444009d90 tx=0x7f444402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7f38d920de1b48 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:16.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.439+0000 7f444b400640 1 -- 172.21.15.52:0/1013081893 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444402cc50 con 0x7f444c0cb070 2024-03-12T17:08:16.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.439+0000 7f444b400640 1 -- 172.21.15.52:0/1013081893 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444402cdb0 con 0x7f444c0cb070 2024-03-12T17:08:16.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.439+0000 7f444b400640 1 -- 172.21.15.52:0/1013081893 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4444034640 con 0x7f444c0cb070 2024-03-12T17:08:16.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.440+0000 7f4452200640 1 -- 172.21.15.52:0/1013081893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f444c0cb070 msgr2=0x7f444c15a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:16.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.440+0000 7f4452200640 1 --2- 172.21.15.52:0/1013081893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f444c0cb070 0x7f444c15a0b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4444009d90 tx=0x7f444402bea0 comp rx=0 tx=0).stop 2024-03-12T17:08:16.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.440+0000 7f4452200640 1 -- 172.21.15.52:0/1013081893 shutdown_connections 2024-03-12T17:08:16.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.440+0000 7f4452200640 1 --2- 172.21.15.52:0/1013081893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f444c0cb070 0x7f444c15a0b0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:16.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.440+0000 7f4452200640 1 --2- 172.21.15.52:0/1013081893 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f444c0cc200 0x7f444c0cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:16.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.440+0000 7f4452200640 1 -- 172.21.15.52:0/1013081893 >> 172.21.15.52:0/1013081893 conn(0x7f444c0b01b0 msgr2=0x7f444c0b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:16.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.440+0000 7f4452200640 1 -- 172.21.15.52:0/1013081893 shutdown_connections 2024-03-12T17:08:16.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.440+0000 7f4452200640 1 -- 172.21.15.52:0/1013081893 wait complete. 2024-03-12T17:08:16.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.442+0000 7f4452200640 1 Processor -- start 2024-03-12T17:08:16.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.442+0000 7f4452200640 1 -- start start 2024-03-12T17:08:16.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.444+0000 7f4452200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f444c0cb070 0x7f444c0c78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:16.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.444+0000 7f4452200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f444c0cc200 0x7f444c09aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:16.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.444+0000 7f4452200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f444c159cd0 0x7f444c09b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:16.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.444+0000 7f4452200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c15a0b0 con 0x7f444c0cc200 2024-03-12T17:08:16.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.444+0000 7f4452200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c09b720 con 0x7f444c159cd0 2024-03-12T17:08:16.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.444+0000 7f4452200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f444c09b910 con 0x7f444c0cb070 2024-03-12T17:08:16.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.444+0000 7f444be00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f444c0cc200 0x7f444c09aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:16.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.444+0000 7f444be00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f444c0cc200 0x7f444c09aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36944/0 (socket says 172.21.15.52:36944) 2024-03-12T17:08:16.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.444+0000 7f444be00640 1 -- 172.21.15.52:0/1891894275 learned_addr learned my addr 172.21.15.52:0/1891894275 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:16.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.444+0000 7f4450e00640 1 --2- 172.21.15.52:0/1891894275 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f444c0cb070 0x7f444c0c78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:16.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.444+0000 7f4451800640 1 --2- 172.21.15.52:0/1891894275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f444c159cd0 0x7f444c09b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:16.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.445+0000 7f444be00640 1 -- 172.21.15.52:0/1891894275 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f444c0cb070 msgr2=0x7f444c0c78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:16.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.445+0000 7f444be00640 1 --2- 172.21.15.52:0/1891894275 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f444c0cb070 0x7f444c0c78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:16.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.445+0000 7f444be00640 1 -- 172.21.15.52:0/1891894275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f444c159cd0 msgr2=0x7f444c09b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:16.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.445+0000 7f444be00640 1 --2- 172.21.15.52:0/1891894275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f444c159cd0 0x7f444c09b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:16.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.445+0000 7f444be00640 1 -- 172.21.15.52:0/1891894275 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4444009a70 con 0x7f444c0cc200 2024-03-12T17:08:16.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.445+0000 7f4450e00640 1 --2- 172.21.15.52:0/1891894275 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f444c0cb070 0x7f444c0c78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:08:16.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.445+0000 7f4451800640 1 --2- 172.21.15.52:0/1891894275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f444c159cd0 0x7f444c09b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:08:16.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.445+0000 7f444be00640 1 --2- 172.21.15.52:0/1891894275 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f444c0cc200 0x7f444c09aca0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f443800c7f0 tx=0x7f443800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:16.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.445+0000 7f4448c00640 1 -- 172.21.15.52:0/1891894275 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4438007a30 con 0x7f444c0cc200 2024-03-12T17:08:16.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.445+0000 7f4448c00640 1 -- 172.21.15.52:0/1891894275 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4438007b90 con 0x7f444c0cc200 2024-03-12T17:08:16.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.445+0000 7f4452200640 1 -- 172.21.15.52:0/1891894275 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f444c09bbf0 con 0x7f444c0cc200 2024-03-12T17:08:16.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.445+0000 7f4448c00640 1 -- 172.21.15.52:0/1891894275 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443800ce40 con 0x7f444c0cc200 2024-03-12T17:08:16.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.446+0000 7f4452200640 1 -- 172.21.15.52:0/1891894275 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f444c09c1b0 con 0x7f444c0cc200 2024-03-12T17:08:16.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.447+0000 7f4452200640 1 -- 172.21.15.52:0/1891894275 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f444c15a1f0 con 0x7f444c0cc200 2024-03-12T17:08:16.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.448+0000 7f4448c00640 1 -- 172.21.15.52:0/1891894275 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f44380113f0 con 0x7f444c0cc200 2024-03-12T17:08:16.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.449+0000 7f4448c00640 1 --2- 172.21.15.52:0/1891894275 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f442c06ac00 0x7f442c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:16.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.449+0000 7f4448c00640 1 -- 172.21.15.52:0/1891894275 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f44380896f0 con 0x7f444c0cc200 2024-03-12T17:08:16.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.450+0000 7f4450e00640 1 --2- 172.21.15.52:0/1891894275 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f442c06ac00 0x7f442c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:16.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.451+0000 7f4450e00640 1 --2- 172.21.15.52:0/1891894275 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f442c06ac00 0x7f442c06d0c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4440007fd0 tx=0x7f4440009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:16.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.459+0000 7f4448c00640 1 -- 172.21.15.52:0/1891894275 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f444c15a1f0 con 0x7f444c0cc200 2024-03-12T17:08:16.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.730+0000 7f4452200640 1 -- 172.21.15.52:0/1891894275 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f444c15a1f0 con 0x7f444c0cc200 2024-03-12T17:08:16.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.735+0000 7f4448c00640 1 -- 172.21.15.52:0/1891894275 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 190+0+0 (secure 0 0 0) 0x7f444c15a1f0 con 0x7f444c0cc200 2024-03-12T17:08:16.729 INFO:teuthology.orchestra.run.smithi052.stderr:updated caps for client.0 2024-03-12T17:08:16.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.738+0000 7f4452200640 1 -- 172.21.15.52:0/1891894275 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f442c06ac00 msgr2=0x7f442c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:16.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.738+0000 7f4452200640 1 --2- 172.21.15.52:0/1891894275 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f442c06ac00 0x7f442c06d0c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4440007fd0 tx=0x7f4440009040 comp rx=0 tx=0).stop 2024-03-12T17:08:16.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.738+0000 7f4452200640 1 -- 172.21.15.52:0/1891894275 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f444c0cc200 msgr2=0x7f444c09aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:16.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.738+0000 7f4452200640 1 --2- 172.21.15.52:0/1891894275 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f444c0cc200 0x7f444c09aca0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f443800c7f0 tx=0x7f443800cbf0 comp rx=0 tx=0).stop 2024-03-12T17:08:16.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.738+0000 7f4452200640 1 -- 172.21.15.52:0/1891894275 shutdown_connections 2024-03-12T17:08:16.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.738+0000 7f4452200640 1 --2- 172.21.15.52:0/1891894275 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f442c06ac00 0x7f442c06d0c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:16.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.738+0000 7f4452200640 1 --2- 172.21.15.52:0/1891894275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f444c159cd0 0x7f444c09b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:16.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.738+0000 7f4452200640 1 --2- 172.21.15.52:0/1891894275 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f444c0cc200 0x7f444c09aca0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:16.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.738+0000 7f4452200640 1 --2- 172.21.15.52:0/1891894275 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f444c0cb070 0x7f444c0c78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:16.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.738+0000 7f4452200640 1 -- 172.21.15.52:0/1891894275 >> 172.21.15.52:0/1891894275 conn(0x7f444c0b01b0 msgr2=0x7f444c0ce950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:16.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.738+0000 7f4452200640 1 -- 172.21.15.52:0/1891894275 shutdown_connections 2024-03-12T17:08:16.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.738+0000 7f4452200640 1 -- 172.21.15.52:0/1891894275 wait complete. 2024-03-12T17:08:16.751 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T17:08:16.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.970+0000 7fe1c7200640 1 Processor -- start 2024-03-12T17:08:16.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.970+0000 7fe1c7200640 1 -- start start 2024-03-12T17:08:16.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.972+0000 7fe1c7200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe1c8143460 0x7fe1c815aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:16.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.972+0000 7fe1c7200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe1c8143930 0x7fe1c80cd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:16.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.972+0000 7fe1c7200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe1c80cdc50 0x7fe1c815e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:16.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.972+0000 7fe1c7200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1c80cc6b0 con 0x7fe1c8143930 2024-03-12T17:08:16.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.972+0000 7fe1c7200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1c80cc950 con 0x7fe1c8143460 2024-03-12T17:08:16.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.972+0000 7fe1c7200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c80ccc30 con 0x7fe1c80cdc50 2024-03-12T17:08:16.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.972+0000 7fe1bfe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe1c8143930 0x7fe1c80cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:16.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.972+0000 7fe1bfe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe1c8143930 0x7fe1c80cd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36946/0 (socket says 172.21.15.52:36946) 2024-03-12T17:08:16.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.972+0000 7fe1bfe00640 1 -- 172.21.15.52:0/998325862 learned_addr learned my addr 172.21.15.52:0/998325862 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:16.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.973+0000 7fe1bfe00640 1 -- 172.21.15.52:0/998325862 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe1c80cdc50 msgr2=0x7fe1c815e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:16.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.973+0000 7fe1bfe00640 1 --2- 172.21.15.52:0/998325862 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe1c80cdc50 0x7fe1c815e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:16.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.973+0000 7fe1bfe00640 1 -- 172.21.15.52:0/998325862 >> v2:172.21.15.102:3300/0 conn(0x7fe1c8143460 msgr2=0x7fe1c815aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:16.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.973+0000 7fe1c5e00640 1 --2- 172.21.15.52:0/998325862 >> v2:172.21.15.102:3300/0 conn(0x7fe1c8143460 0x7fe1c815aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:16.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.973+0000 7fe1bfe00640 1 --2- 172.21.15.52:0/998325862 >> v2:172.21.15.102:3300/0 conn(0x7fe1c8143460 0x7fe1c815aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:16.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.973+0000 7fe1bfe00640 1 -- 172.21.15.52:0/998325862 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c80ccf10 con 0x7fe1c8143930 2024-03-12T17:08:16.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.974+0000 7fe1bfe00640 1 --2- 172.21.15.52:0/998325862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1c8143930 0x7fe1c80cd710 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe1c806e410 tx=0x7fe1b002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e400961434fb59ca server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:16.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.974+0000 7fe1c5400640 1 -- 172.21.15.52:0/998325862 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0030860 con 0x7fe1c8143930 2024-03-12T17:08:16.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.975+0000 7fe1c5400640 1 -- 172.21.15.52:0/998325862 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b00309c0 con 0x7fe1c8143930 2024-03-12T17:08:16.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.975+0000 7fe1c5400640 1 -- 172.21.15.52:0/998325862 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0030d50 con 0x7fe1c8143930 2024-03-12T17:08:16.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.975+0000 7fe1c7200640 1 -- 172.21.15.52:0/998325862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1c8143930 msgr2=0x7fe1c80cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:16.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.975+0000 7fe1c7200640 1 --2- 172.21.15.52:0/998325862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1c8143930 0x7fe1c80cd710 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe1c806e410 tx=0x7fe1b002b9a0 comp rx=0 tx=0).stop 2024-03-12T17:08:16.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.975+0000 7fe1c7200640 1 -- 172.21.15.52:0/998325862 shutdown_connections 2024-03-12T17:08:16.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.975+0000 7fe1c7200640 1 --2- 172.21.15.52:0/998325862 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe1c80cdc50 0x7fe1c815e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:16.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.975+0000 7fe1c7200640 1 --2- 172.21.15.52:0/998325862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1c8143930 0x7fe1c80cd710 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:16.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.975+0000 7fe1c7200640 1 --2- 172.21.15.52:0/998325862 >> v2:172.21.15.102:3300/0 conn(0x7fe1c8143460 0x7fe1c815aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:16.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.975+0000 7fe1c7200640 1 -- 172.21.15.52:0/998325862 >> 172.21.15.52:0/998325862 conn(0x7fe1c8065df0 msgr2=0x7fe1c8069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:16.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.976+0000 7fe1c7200640 1 -- 172.21.15.52:0/998325862 shutdown_connections 2024-03-12T17:08:16.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.976+0000 7fe1c7200640 1 -- 172.21.15.52:0/998325862 wait complete. 2024-03-12T17:08:16.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.977+0000 7fe1c7200640 1 Processor -- start 2024-03-12T17:08:16.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.977+0000 7fe1c7200640 1 -- start start 2024-03-12T17:08:16.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.978+0000 7fe1c7200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1c80cdc50 0x7fe1c81367d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:16.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.978+0000 7fe1c7200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1c8143460 0x7fe1c8134ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:16.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.979+0000 7fe1c7200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe1c8143930 0x7fe1c8135000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:16.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.979+0000 7fe1c7200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c8135540 con 0x7fe1c80cdc50 2024-03-12T17:08:16.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.979+0000 7fe1c7200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c8136340 con 0x7fe1c8143460 2024-03-12T17:08:16.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.979+0000 7fe1c7200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c806d060 con 0x7fe1c8143930 2024-03-12T17:08:16.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.979+0000 7fe1c5e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1c80cdc50 0x7fe1c81367d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:16.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.979+0000 7fe1c5e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1c80cdc50 0x7fe1c81367d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36960/0 (socket says 172.21.15.52:36960) 2024-03-12T17:08:16.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.979+0000 7fe1c5e00640 1 -- 172.21.15.52:0/1839403417 learned_addr learned my addr 172.21.15.52:0/1839403417 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:16.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.979+0000 7fe1bfe00640 1 --2- 172.21.15.52:0/1839403417 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1c8143460 0x7fe1c8134ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:16.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.979+0000 7fe1c6800640 1 --2- 172.21.15.52:0/1839403417 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe1c8143930 0x7fe1c8135000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:16.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.979+0000 7fe1c5e00640 1 -- 172.21.15.52:0/1839403417 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe1c8143930 msgr2=0x7fe1c8135000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:16.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.979+0000 7fe1c5e00640 1 --2- 172.21.15.52:0/1839403417 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe1c8143930 0x7fe1c8135000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:16.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.979+0000 7fe1c5e00640 1 -- 172.21.15.52:0/1839403417 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1c8143460 msgr2=0x7fe1c8134ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:16.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.979+0000 7fe1c5e00640 1 --2- 172.21.15.52:0/1839403417 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1c8143460 0x7fe1c8134ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:16.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.979+0000 7fe1c5e00640 1 -- 172.21.15.52:0/1839403417 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b00094d0 con 0x7fe1c80cdc50 2024-03-12T17:08:16.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.980+0000 7fe1c5e00640 1 --2- 172.21.15.52:0/1839403417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1c80cdc50 0x7fe1c81367d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe1b800c820 tx=0x7fe1b800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:16.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.980+0000 7fe1be000640 1 -- 172.21.15.52:0/1839403417 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8004220 con 0x7fe1c80cdc50 2024-03-12T17:08:16.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.980+0000 7fe1be000640 1 -- 172.21.15.52:0/1839403417 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b80043a0 con 0x7fe1c80cdc50 2024-03-12T17:08:16.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.980+0000 7fe1be000640 1 -- 172.21.15.52:0/1839403417 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8007a70 con 0x7fe1c80cdc50 2024-03-12T17:08:16.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.980+0000 7fe1c7200640 1 -- 172.21.15.52:0/1839403417 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1c8135730 con 0x7fe1c80cdc50 2024-03-12T17:08:16.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.980+0000 7fe1c7200640 1 -- 172.21.15.52:0/1839403417 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c8135cf0 con 0x7fe1c80cdc50 2024-03-12T17:08:16.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.981+0000 7fe1c7200640 1 -- 172.21.15.52:0/1839403417 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1c804e030 con 0x7fe1c80cdc50 2024-03-12T17:08:16.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.982+0000 7fe1be000640 1 -- 172.21.15.52:0/1839403417 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe1b8007bd0 con 0x7fe1c80cdc50 2024-03-12T17:08:16.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.983+0000 7fe1be000640 1 --2- 172.21.15.52:0/1839403417 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe1a406ac30 0x7fe1a406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:16.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.983+0000 7fe1be000640 1 -- 172.21.15.52:0/1839403417 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe1b80892b0 con 0x7fe1c80cdc50 2024-03-12T17:08:16.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.984+0000 7fe1bfe00640 1 --2- 172.21.15.52:0/1839403417 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe1a406ac30 0x7fe1a406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:16.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.985+0000 7fe1bfe00640 1 --2- 172.21.15.52:0/1839403417 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe1a406ac30 0x7fe1a406d0f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0036000 tx=0x7fe1b003b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:16.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:16.988+0000 7fe1be000640 1 -- 172.21.15.52:0/1839403417 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fe1b80542b0 con 0x7fe1c80cdc50 2024-03-12T17:08:17.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.233+0000 7fe1c7200640 1 -- 172.21.15.52:0/1839403417 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1c815d9d0 con 0x7fe1c80cdc50 2024-03-12T17:08:17.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.234+0000 7fe1be000640 1 -- 172.21.15.52:0/1839403417 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4734 (secure 0 0 0) 0x7fe1b80578d0 con 0x7fe1c80cdc50 2024-03-12T17:08:17.228 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:08:17.228 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6836/285561331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":285561331},{"type":"v1","addr":"172.21.15.102:6836","nonce":285561331}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4909,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1312095655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6835","nonce":1312095655},{"type":"v1","addr":"172.21.15.102:6837","nonce":1312095655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5337,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2451989690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2451989690},{"type":"v1","addr":"172.21.15.52:6833","nonce":2451989690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-12T17:08:10.679436+0000","modified":"2024-03-12T17:08:14.140830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5343},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5343":{"gid":5343,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.52:6835/543567396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":543567396},{"type":"v1","addr":"172.21.15.52:6835","nonce":543567396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-12T17:08:17.228 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 17 2024-03-12T17:08:17.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.237+0000 7fe1c7200640 1 -- 172.21.15.52:0/1839403417 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe1a406ac30 msgr2=0x7fe1a406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:17.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.237+0000 7fe1c7200640 1 --2- 172.21.15.52:0/1839403417 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe1a406ac30 0x7fe1a406d0f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0036000 tx=0x7fe1b003b040 comp rx=0 tx=0).stop 2024-03-12T17:08:17.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.237+0000 7fe1c7200640 1 -- 172.21.15.52:0/1839403417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1c80cdc50 msgr2=0x7fe1c81367d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:17.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.237+0000 7fe1c7200640 1 --2- 172.21.15.52:0/1839403417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1c80cdc50 0x7fe1c81367d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe1b800c820 tx=0x7fe1b800cce0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.237+0000 7fe1c7200640 1 -- 172.21.15.52:0/1839403417 shutdown_connections 2024-03-12T17:08:17.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.237+0000 7fe1c7200640 1 --2- 172.21.15.52:0/1839403417 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe1a406ac30 0x7fe1a406d0f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.237+0000 7fe1c7200640 1 --2- 172.21.15.52:0/1839403417 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe1c8143930 0x7fe1c8135000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.237+0000 7fe1c7200640 1 --2- 172.21.15.52:0/1839403417 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1c8143460 0x7fe1c8134ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.237+0000 7fe1c7200640 1 --2- 172.21.15.52:0/1839403417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1c80cdc50 0x7fe1c81367d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.237+0000 7fe1c7200640 1 -- 172.21.15.52:0/1839403417 >> 172.21.15.52:0/1839403417 conn(0x7fe1c8065df0 msgr2=0x7fe1c80af180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:17.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.237+0000 7fe1c7200640 1 -- 172.21.15.52:0/1839403417 shutdown_connections 2024-03-12T17:08:17.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.237+0000 7fe1c7200640 1 -- 172.21.15.52:0/1839403417 wait complete. 2024-03-12T17:08:17.654 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-12T17:08:10.679436+0000', 'modified': '2024-03-12T17:08:14.140830+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5343}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5343': {'gid': 5343, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.52:6835/543567396', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6834', 'nonce': 543567396}, {'type': 'v1', 'addr': '172.21.15.52:6835', 'nonce': 543567396}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-12T17:08:17.654 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-12T17:08:17.654 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-12T17:08:17.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.890+0000 7fb96ce00640 1 Processor -- start 2024-03-12T17:08:17.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.890+0000 7fb96ce00640 1 -- start start 2024-03-12T17:08:17.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.891+0000 7fb96ce00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb968146070 0x7fb9680b2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:17.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.892+0000 7fb96ce00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb9680707e0 0x7fb96806cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:17.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.892+0000 7fb96ce00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9680b2f70 con 0x7fb96815cd00 2024-03-12T17:08:17.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.892+0000 7fb96ce00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb96806d130 con 0x7fb9680707e0 2024-03-12T17:08:17.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.892+0000 7fb96ce00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9680b3250 con 0x7fb968146070 2024-03-12T17:08:17.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.892+0000 7fb967e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb9680707e0 0x7fb96806cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:17.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.892+0000 7fb966a00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fb96815cd00 0x7fb9680700d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:34660/0 (socket says 172.21.15.52:34660) 2024-03-12T17:08:17.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.892+0000 7fb966a00640 1 -- 172.21.15.52:0/1131386969 learned_addr learned my addr 172.21.15.52:0/1131386969 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:08:17.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.892+0000 7fb967400640 1 --2- 172.21.15.52:0/1131386969 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb968146070 0x7fb9680b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:17.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.892+0000 7fb967e00640 1 -- 172.21.15.52:0/1131386969 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb968146070 msgr2=0x7fb9680b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:17.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.892+0000 7fb967e00640 1 --2- 172.21.15.52:0/1131386969 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb968146070 0x7fb9680b2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.892+0000 7fb967e00640 1 -- 172.21.15.52:0/1131386969 >> v1:172.21.15.52:6789/0 conn(0x7fb96815cd00 legacy=0x7fb9680700d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:17.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.892+0000 7fb966000640 1 -- 172.21.15.52:0/1131386969 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3509890352 0 0) 0x7fb9680b2f70 con 0x7fb96815cd00 2024-03-12T17:08:17.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.893+0000 7fb967e00640 1 -- 172.21.15.52:0/1131386969 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9680b3530 con 0x7fb9680707e0 2024-03-12T17:08:17.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.893+0000 7fb967400640 1 --2- 172.21.15.52:0/1131386969 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb968146070 0x7fb9680b2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:08:17.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.894+0000 7fb967e00640 1 --2- 172.21.15.52:0/1131386969 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9680707e0 0x7fb96806cbf0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb960009d90 tx=0x7fb96002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea6de86101b291a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:17.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.894+0000 7fb966000640 1 -- 172.21.15.52:0/1131386969 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96002cc60 con 0x7fb9680707e0 2024-03-12T17:08:17.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.894+0000 7fb966000640 1 -- 172.21.15.52:0/1131386969 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb96002cdc0 con 0x7fb9680707e0 2024-03-12T17:08:17.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.894+0000 7fb966000640 1 -- 172.21.15.52:0/1131386969 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb960034660 con 0x7fb9680707e0 2024-03-12T17:08:17.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.894+0000 7fb96ce00640 1 -- 172.21.15.52:0/1131386969 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9680707e0 msgr2=0x7fb96806cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:17.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.894+0000 7fb96ce00640 1 --2- 172.21.15.52:0/1131386969 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9680707e0 0x7fb96806cbf0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb960009d90 tx=0x7fb96002beb0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.895+0000 7fb96ce00640 1 -- 172.21.15.52:0/1131386969 shutdown_connections 2024-03-12T17:08:17.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.895+0000 7fb96ce00640 1 --2- 172.21.15.52:0/1131386969 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9680707e0 0x7fb96806cbf0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.895+0000 7fb96ce00640 1 --2- 172.21.15.52:0/1131386969 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb968146070 0x7fb9680b2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.895+0000 7fb96ce00640 1 -- 172.21.15.52:0/1131386969 >> 172.21.15.52:0/1131386969 conn(0x7fb968065df0 msgr2=0x7fb968069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:17.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.895+0000 7fb96ce00640 1 -- 172.21.15.52:0/1131386969 shutdown_connections 2024-03-12T17:08:17.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.895+0000 7fb96ce00640 1 -- 172.21.15.52:0/1131386969 wait complete. 2024-03-12T17:08:17.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.896+0000 7fb96ce00640 1 Processor -- start 2024-03-12T17:08:17.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.897+0000 7fb96ce00640 1 -- start start 2024-03-12T17:08:17.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.898+0000 7fb96ce00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb9680707e0 0x7fb96814f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:17.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.898+0000 7fb96ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb968146070 0x7fb96813c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:17.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.898+0000 7fb96ce00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb96815cd00 0x7fb96813cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:17.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.898+0000 7fb96ce00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96814f120 con 0x7fb96815cd00 2024-03-12T17:08:17.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.898+0000 7fb96ce00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96814e510 con 0x7fb968146070 2024-03-12T17:08:17.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.898+0000 7fb96ce00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb96814e7f0 con 0x7fb9680707e0 2024-03-12T17:08:17.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.898+0000 7fb967e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb96815cd00 0x7fb96813cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:17.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.898+0000 7fb967e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb96815cd00 0x7fb96813cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36978/0 (socket says 172.21.15.52:36978) 2024-03-12T17:08:17.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.898+0000 7fb967e00640 1 -- 172.21.15.52:0/1942242707 learned_addr learned my addr 172.21.15.52:0/1942242707 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:17.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.898+0000 7fb966a00640 1 --2- 172.21.15.52:0/1942242707 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb968146070 0x7fb96813c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:17.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.898+0000 7fb967400640 1 --2- 172.21.15.52:0/1942242707 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb9680707e0 0x7fb96814f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:17.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.899+0000 7fb967e00640 1 -- 172.21.15.52:0/1942242707 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb9680707e0 msgr2=0x7fb96814f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:17.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.899+0000 7fb967e00640 1 --2- 172.21.15.52:0/1942242707 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb9680707e0 0x7fb96814f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.899+0000 7fb967e00640 1 -- 172.21.15.52:0/1942242707 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb968146070 msgr2=0x7fb96813c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:17.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.899+0000 7fb967e00640 1 --2- 172.21.15.52:0/1942242707 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb968146070 0x7fb96813c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.899+0000 7fb967e00640 1 -- 172.21.15.52:0/1942242707 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb960009a70 con 0x7fb96815cd00 2024-03-12T17:08:17.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.899+0000 7fb967400640 1 --2- 172.21.15.52:0/1942242707 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb9680707e0 0x7fb96814f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:08:17.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.899+0000 7fb967e00640 1 --2- 172.21.15.52:0/1942242707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb96815cd00 0x7fb96813cc20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb960009d90 tx=0x7fb960004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:17.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.899+0000 7fb95f400640 1 -- 172.21.15.52:0/1942242707 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb960040650 con 0x7fb96815cd00 2024-03-12T17:08:17.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.899+0000 7fb95f400640 1 -- 172.21.15.52:0/1942242707 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9600407d0 con 0x7fb96815cd00 2024-03-12T17:08:17.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.899+0000 7fb95f400640 1 -- 172.21.15.52:0/1942242707 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb960040b80 con 0x7fb96815cd00 2024-03-12T17:08:17.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.899+0000 7fb96ce00640 1 -- 172.21.15.52:0/1942242707 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb96814ead0 con 0x7fb96815cd00 2024-03-12T17:08:17.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.899+0000 7fb96ce00640 1 -- 172.21.15.52:0/1942242707 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb96814faf0 con 0x7fb96815cd00 2024-03-12T17:08:17.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.902+0000 7fb95f400640 1 -- 172.21.15.52:0/1942242707 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb960040ce0 con 0x7fb96815cd00 2024-03-12T17:08:17.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.903+0000 7fb95f400640 1 --2- 172.21.15.52:0/1942242707 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb94806ac00 0x7fb94806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:17.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.904+0000 7fb95f400640 1 -- 172.21.15.52:0/1942242707 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb9600ac460 con 0x7fb96815cd00 2024-03-12T17:08:17.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.904+0000 7fb970ffb740 1 Processor -- start 2024-03-12T17:08:17.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.904+0000 7fb967400640 1 --2- 172.21.15.52:0/1942242707 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb94806ac00 0x7fb94806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:17.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.904+0000 7fb970ffb740 1 -- start start 2024-03-12T17:08:17.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.905+0000 7fb970ffb740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565be3a8200 0x5565be35f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:17.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.905+0000 7fb970ffb740 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5565be35e280 0x5565be35fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:17.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.905+0000 7fb970ffb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565be35a720 0x5565be38a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:17.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.905+0000 7fb970ffb740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565be335e40 con 0x5565be3a8200 2024-03-12T17:08:17.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.905+0000 7fb970ffb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565be2cb090 con 0x5565be35a720 2024-03-12T17:08:17.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.905+0000 7fb970ffb740 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x5565be2c8820 con 0x5565be35e280 2024-03-12T17:08:17.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.905+0000 7fb967400640 1 --2- 172.21.15.52:0/1942242707 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb94806ac00 0x7fb94806d0c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb958006fd0 tx=0x7fb958008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:17.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.905+0000 7fb967400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565be3a8200 0x5565be35f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:17.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.906+0000 7fb967e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565be35a720 0x5565be38a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:17.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.906+0000 7fb967400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565be3a8200 0x5565be35f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36988/0 (socket says 172.21.15.52:36988) 2024-03-12T17:08:17.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.906+0000 7fb967400640 1 -- 172.21.15.52:0/981150921 learned_addr learned my addr 172.21.15.52:0/981150921 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:17.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.906+0000 7fb966a00640 1 --2- 172.21.15.52:0/981150921 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5565be35e280 0x5565be35fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:17.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.906+0000 7fb967400640 1 -- 172.21.15.52:0/981150921 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5565be35e280 msgr2=0x5565be35fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:17.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.906+0000 7fb967400640 1 --2- 172.21.15.52:0/981150921 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5565be35e280 0x5565be35fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.906+0000 7fb967400640 1 -- 172.21.15.52:0/981150921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565be35a720 msgr2=0x5565be38a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:17.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.906+0000 7fb967400640 1 --2- 172.21.15.52:0/981150921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565be35a720 0x5565be38a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.906+0000 7fb967400640 1 -- 172.21.15.52:0/981150921 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565be28b780 con 0x5565be3a8200 2024-03-12T17:08:17.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.906+0000 7fb967400640 1 --2- 172.21.15.52:0/981150921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565be3a8200 0x5565be35f830 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb95800db90 tx=0x7fb958010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43fe2b123783f58b server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:17.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.907+0000 7fb953e00640 1 -- 172.21.15.52:0/981150921 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95800b050 con 0x5565be3a8200 2024-03-12T17:08:17.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.907+0000 7fb953e00640 1 -- 172.21.15.52:0/981150921 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb958017070 con 0x5565be3a8200 2024-03-12T17:08:17.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.907+0000 7fb953e00640 1 -- 172.21.15.52:0/981150921 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb958012dc0 con 0x5565be3a8200 2024-03-12T17:08:17.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.907+0000 7fb970ffb740 1 -- 172.21.15.52:0/981150921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565be3a8200 msgr2=0x5565be35f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:17.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.907+0000 7fb970ffb740 1 --2- 172.21.15.52:0/981150921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565be3a8200 0x5565be35f830 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb95800db90 tx=0x7fb958010000 comp rx=0 tx=0).stop 2024-03-12T17:08:17.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.908+0000 7fb970ffb740 1 -- 172.21.15.52:0/981150921 shutdown_connections 2024-03-12T17:08:17.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.908+0000 7fb970ffb740 1 --2- 172.21.15.52:0/981150921 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5565be35e280 0x5565be35fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.908+0000 7fb970ffb740 1 --2- 172.21.15.52:0/981150921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565be35a720 0x5565be38a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.908+0000 7fb970ffb740 1 --2- 172.21.15.52:0/981150921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565be3a8200 0x5565be35f830 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.908+0000 7fb970ffb740 1 -- 172.21.15.52:0/981150921 >> 172.21.15.52:0/981150921 conn(0x5565be35c990 msgr2=0x5565be3a5510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:17.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.908+0000 7fb970ffb740 1 -- 172.21.15.52:0/981150921 shutdown_connections 2024-03-12T17:08:17.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.908+0000 7fb970ffb740 1 -- 172.21.15.52:0/981150921 wait complete. 2024-03-12T17:08:17.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.908+0000 7fb970ffb740 1 -- start start 2024-03-12T17:08:17.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.909+0000 7fb970ffb740 1 Processor -- start 2024-03-12T17:08:17.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.910+0000 7fb970ffb740 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5565be35e280 0x5565be3ad480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:17.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.910+0000 7fb970ffb740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565be3a8200 0x5565be382070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:17.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.910+0000 7fb970ffb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565be3a6710 0x5565be3a6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:17.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.910+0000 7fb970ffb740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565be2c4110 con 0x5565be3a8200 2024-03-12T17:08:17.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.910+0000 7fb970ffb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565be2658c0 con 0x5565be3a6710 2024-03-12T17:08:17.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.910+0000 7fb970ffb740 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x5565be265560 con 0x5565be35e280 2024-03-12T17:08:17.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.910+0000 7fb970ffb740 10 client.0 _ll_register_callbacks cb 0x5565be2bd8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T17:08:17.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.910+0000 7fb966a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565be3a8200 0x5565be382070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:17.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.910+0000 7fb966a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565be3a8200 0x5565be382070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37004/0 (socket says 172.21.15.52:37004) 2024-03-12T17:08:17.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.910+0000 7fb966a00640 1 -- 172.21.15.52:0/2685485566 learned_addr learned my addr 172.21.15.52:0/2685485566 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:17.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.910+0000 7fb967400640 1 --2- 172.21.15.52:0/2685485566 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5565be35e280 0x5565be3ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:17.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.910+0000 7fb967e00640 1 --2- 172.21.15.52:0/2685485566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565be3a6710 0x5565be3a6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:17.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.910+0000 7fb966a00640 1 -- 172.21.15.52:0/2685485566 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5565be35e280 msgr2=0x5565be3ad480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:17.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.910+0000 7fb966a00640 1 --2- 172.21.15.52:0/2685485566 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5565be35e280 0x5565be3ad480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.910+0000 7fb966a00640 1 -- 172.21.15.52:0/2685485566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565be3a6710 msgr2=0x5565be3a6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:17.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.910+0000 7fb966a00640 1 --2- 172.21.15.52:0/2685485566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565be3a6710 0x5565be3a6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.910+0000 7fb966a00640 1 -- 172.21.15.52:0/2685485566 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb95800d840 con 0x5565be3a8200 2024-03-12T17:08:17.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.911+0000 7fb966a00640 1 --2- 172.21.15.52:0/2685485566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565be3a8200 0x5565be382070 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb954005f10 tx=0x7fb954001520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894d287b2c06fcef server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:17.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.911+0000 7fb950c00640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-03-12T17:08:17.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.911+0000 7fb950c00640 1 -- 172.21.15.52:0/2685485566 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb954015470 con 0x5565be3a8200 2024-03-12T17:08:17.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.911+0000 7fb950c00640 1 -- 172.21.15.52:0/2685485566 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb9540155d0 con 0x5565be3a8200 2024-03-12T17:08:17.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.911+0000 7fb970ffb740 1 -- 172.21.15.52:0/2685485566 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5565be2ee660 con 0x5565be3a8200 2024-03-12T17:08:17.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.911+0000 7fb950c00640 1 -- 172.21.15.52:0/2685485566 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb954015790 con 0x5565be3a8200 2024-03-12T17:08:17.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.911+0000 7fb970ffb740 10 client.5520 fetch_fsmap learned FSMap version 17 2024-03-12T17:08:17.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.911+0000 7fb970ffb740 1 -- 172.21.15.52:0/2685485566 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x5565be2c2790 con 0x5565be3a8200 2024-03-12T17:08:17.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.912+0000 7fb950c00640 1 -- 172.21.15.52:0/2685485566 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5565be2ee660 con 0x5565be3a8200 2024-03-12T17:08:17.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.912+0000 7fb950c00640 1 -- 172.21.15.52:0/2685485566 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 17) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fb9540158f0 con 0x5565be3a8200 2024-03-12T17:08:17.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.912+0000 7fb970ffb740 10 client.5520 fetch_fsmap finished waiting for FSMap version 17 2024-03-12T17:08:17.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.912+0000 7fb970ffb740 10 client.5520 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-12T17:08:17.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.912+0000 7fb970ffb740 20 client.5520 populate_metadata read hostname 'smithi052' 2024-03-12T17:08:17.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.912+0000 7fb970ffb740 1 --2- 172.21.15.52:0/2685485566 >> [v2:172.21.15.52:6834/543567396,v1:172.21.15.52:6835/543567396] conn(0x5565be36ea90 0x5565be36ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:17.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.912+0000 7fb970ffb740 4 client.5520 mds_command: new command op to 5343 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-12T17:08:17.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.912+0000 7fb970ffb740 1 -- 172.21.15.52:0/2685485566 --> [v2:172.21.15.52:6834/543567396,v1:172.21.15.52:6835/543567396] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565be2b6e50 con 0x5565be36ea90 2024-03-12T17:08:17.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.912+0000 7fb967400640 1 --2- 172.21.15.52:0/2685485566 >> [v2:172.21.15.52:6834/543567396,v1:172.21.15.52:6835/543567396] conn(0x5565be36ea90 0x5565be36ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:17.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.913+0000 7fb967400640 1 --2- 172.21.15.52:0/2685485566 >> [v2:172.21.15.52:6834/543567396,v1:172.21.15.52:6835/543567396] conn(0x5565be36ea90 0x5565be36ee70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6102570c3d61b8c server_cookie=a0ef503f7b40217a in_seq=0 out_seq=0 2024-03-12T17:08:17.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.913+0000 7fb950c00640 10 client.5520 ms_handle_connect on v2:172.21.15.52:6834/543567396 2024-03-12T17:08:17.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.913+0000 7fb950c00640 1 -- 172.21.15.52:0/2685485566 <== mds.0 v2:172.21.15.52:6834/543567396 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5565be2b6e50 con 0x5565be36ea90 2024-03-12T17:08:17.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.913+0000 7fb950c00640 10 client.5520 handle_command_reply: tid=0 2024-03-12T17:08:17.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.914+0000 7fb970ffb740 1 client.5520 shutdown 2024-03-12T17:08:17.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.914+0000 7fb970ffb740 1 -- 172.21.15.52:0/2685485566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565be3a8200 msgr2=0x5565be382070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:17.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.914+0000 7fb970ffb740 1 --2- 172.21.15.52:0/2685485566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565be3a8200 0x5565be382070 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb954005f10 tx=0x7fb954001520 comp rx=0 tx=0).stop 2024-03-12T17:08:17.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.914+0000 7fb970ffb740 1 -- 172.21.15.52:0/2685485566 shutdown_connections 2024-03-12T17:08:17.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.914+0000 7fb970ffb740 1 --2- 172.21.15.52:0/2685485566 >> [v2:172.21.15.52:6834/543567396,v1:172.21.15.52:6835/543567396] conn(0x5565be36ea90 0x5565be36ee70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.914+0000 7fb970ffb740 1 --2- 172.21.15.52:0/2685485566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565be3a6710 0x5565be3a6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.914+0000 7fb970ffb740 1 --2- 172.21.15.52:0/2685485566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565be3a8200 0x5565be382070 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.914+0000 7fb970ffb740 1 --2- 172.21.15.52:0/2685485566 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5565be35e280 0x5565be3ad480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.914+0000 7fb950c00640 0 client.5520 ms_handle_reset on v2:172.21.15.52:6834/543567396 2024-03-12T17:08:17.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.914+0000 7fb970ffb740 1 -- 172.21.15.52:0/2685485566 >> 172.21.15.52:0/2685485566 conn(0x5565be35a720 msgr2=0x5565be35ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:17.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.914+0000 7fb970ffb740 1 -- 172.21.15.52:0/2685485566 shutdown_connections 2024-03-12T17:08:17.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.914+0000 7fb970ffb740 1 -- 172.21.15.52:0/2685485566 wait complete. 2024-03-12T17:08:17.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.914+0000 7fb970ffb740 20 client.5520 trim_cache size 0 max 16384 2024-03-12T17:08:17.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.931+0000 7fb970ffb740 1 Processor -- start 2024-03-12T17:08:17.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.931+0000 7fb970ffb740 1 -- start start 2024-03-12T17:08:17.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.932+0000 7fb970ffb740 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5565be35e280 0x5565be3a4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:17.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.932+0000 7fb970ffb740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565be3a6710 0x5565be38e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:17.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.932+0000 7fb970ffb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565be3a8200 0x5565be359820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:17.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.932+0000 7fb970ffb740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565be02bf20 con 0x5565be3a6710 2024-03-12T17:08:17.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.932+0000 7fb970ffb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565be35f1a0 con 0x5565be3a8200 2024-03-12T17:08:17.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.932+0000 7fb970ffb740 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x5565be38ea20 con 0x5565be35e280 2024-03-12T17:08:17.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.932+0000 7fb966a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565be3a6710 0x5565be38e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:17.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.932+0000 7fb966a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565be3a6710 0x5565be38e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37020/0 (socket says 172.21.15.52:37020) 2024-03-12T17:08:17.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.932+0000 7fb966a00640 1 -- 172.21.15.52:0/250325890 learned_addr learned my addr 172.21.15.52:0/250325890 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:17.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.933+0000 7fb967400640 1 --2- 172.21.15.52:0/250325890 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5565be35e280 0x5565be3a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:17.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.933+0000 7fb967e00640 1 --2- 172.21.15.52:0/250325890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565be3a8200 0x5565be359820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:17.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.933+0000 7fb966a00640 1 -- 172.21.15.52:0/250325890 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5565be35e280 msgr2=0x5565be3a4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:17.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.933+0000 7fb966a00640 1 --2- 172.21.15.52:0/250325890 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5565be35e280 0x5565be3a4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.933+0000 7fb966a00640 1 -- 172.21.15.52:0/250325890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565be3a8200 msgr2=0x5565be359820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:17.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.933+0000 7fb966a00640 1 --2- 172.21.15.52:0/250325890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565be3a8200 0x5565be359820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.933+0000 7fb966a00640 1 -- 172.21.15.52:0/250325890 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565be264ad0 con 0x5565be3a6710 2024-03-12T17:08:17.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.933+0000 7fb966a00640 1 --2- 172.21.15.52:0/250325890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565be3a6710 0x5565be38e4e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb954005870 tx=0x7fb95400e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fca96fadbd277d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:17.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.934+0000 7fb95cc00640 1 -- 172.21.15.52:0/250325890 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb954015db0 con 0x5565be3a6710 2024-03-12T17:08:17.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.934+0000 7fb95cc00640 1 -- 172.21.15.52:0/250325890 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb95400b070 con 0x5565be3a6710 2024-03-12T17:08:17.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.934+0000 7fb95cc00640 1 -- 172.21.15.52:0/250325890 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb954015670 con 0x5565be3a6710 2024-03-12T17:08:17.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.934+0000 7fb970ffb740 1 -- 172.21.15.52:0/250325890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565be3a6710 msgr2=0x5565be38e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:17.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.934+0000 7fb970ffb740 1 --2- 172.21.15.52:0/250325890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565be3a6710 0x5565be38e4e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb954005870 tx=0x7fb95400e000 comp rx=0 tx=0).stop 2024-03-12T17:08:17.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.934+0000 7fb970ffb740 1 -- 172.21.15.52:0/250325890 shutdown_connections 2024-03-12T17:08:17.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.934+0000 7fb970ffb740 1 --2- 172.21.15.52:0/250325890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565be3a8200 0x5565be359820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.934+0000 7fb970ffb740 1 --2- 172.21.15.52:0/250325890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565be3a6710 0x5565be38e4e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.934+0000 7fb970ffb740 1 --2- 172.21.15.52:0/250325890 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5565be35e280 0x5565be3a4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.934+0000 7fb970ffb740 1 -- 172.21.15.52:0/250325890 >> 172.21.15.52:0/250325890 conn(0x5565be35a720 msgr2=0x5565be3c7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:17.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.934+0000 7fb970ffb740 1 -- 172.21.15.52:0/250325890 shutdown_connections 2024-03-12T17:08:17.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.934+0000 7fb970ffb740 1 -- 172.21.15.52:0/250325890 wait complete. 2024-03-12T17:08:17.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.935+0000 7fb970ffb740 1 -- start start 2024-03-12T17:08:17.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.935+0000 7fb970ffb740 1 Processor -- start 2024-03-12T17:08:17.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.936+0000 7fb970ffb740 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5565be35e280 0x5565be410b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:17.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.936+0000 7fb970ffb740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565be3a6710 0x5565be36be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:17.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.936+0000 7fb970ffb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565be3a8200 0x5565be3bf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:17.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.936+0000 7fb970ffb740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565be36c900 con 0x5565be3a6710 2024-03-12T17:08:17.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.936+0000 7fb970ffb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565be359ec0 con 0x5565be3a8200 2024-03-12T17:08:17.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.936+0000 7fb970ffb740 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x5565be35a110 con 0x5565be35e280 2024-03-12T17:08:17.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.936+0000 7fb970ffb740 10 client.0 _ll_register_callbacks cb 0x7fb9540071d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-12T17:08:17.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.936+0000 7fb966a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565be3a6710 0x5565be36be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:17.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.937+0000 7fb966a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565be3a6710 0x5565be36be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37036/0 (socket says 172.21.15.52:37036) 2024-03-12T17:08:17.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.937+0000 7fb966a00640 1 -- 172.21.15.52:0/1441904563 learned_addr learned my addr 172.21.15.52:0/1441904563 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:17.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.937+0000 7fb967400640 1 --2- 172.21.15.52:0/1441904563 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5565be35e280 0x5565be410b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:17.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.937+0000 7fb967e00640 1 --2- 172.21.15.52:0/1441904563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565be3a8200 0x5565be3bf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:17.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.937+0000 7fb966a00640 1 -- 172.21.15.52:0/1441904563 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5565be35e280 msgr2=0x5565be410b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:17.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.937+0000 7fb966a00640 1 --2- 172.21.15.52:0/1441904563 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5565be35e280 0x5565be410b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.937+0000 7fb966a00640 1 -- 172.21.15.52:0/1441904563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565be3a8200 msgr2=0x5565be3bf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:17.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.937+0000 7fb966a00640 1 --2- 172.21.15.52:0/1441904563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565be3a8200 0x5565be3bf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.937+0000 7fb966a00640 1 -- 172.21.15.52:0/1441904563 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565be2c2790 con 0x5565be3a6710 2024-03-12T17:08:17.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.937+0000 7fb966a00640 1 --2- 172.21.15.52:0/1441904563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565be3a6710 0x5565be36be80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb954005870 tx=0x7fb954001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf25671ea4d29a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:17.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.937+0000 7fb950c00640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-03-12T17:08:17.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.938+0000 7fb950c00640 1 -- 172.21.15.52:0/1441904563 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb954026800 con 0x5565be3a6710 2024-03-12T17:08:17.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.938+0000 7fb950c00640 1 -- 172.21.15.52:0/1441904563 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb954026960 con 0x5565be3a6710 2024-03-12T17:08:17.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.938+0000 7fb970ffb740 1 -- 172.21.15.52:0/1441904563 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb958017070 con 0x5565be3a6710 2024-03-12T17:08:17.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.938+0000 7fb950c00640 1 -- 172.21.15.52:0/1441904563 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb954026b20 con 0x5565be3a6710 2024-03-12T17:08:17.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.938+0000 7fb970ffb740 10 client.5526 fetch_fsmap learned FSMap version 17 2024-03-12T17:08:17.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.938+0000 7fb970ffb740 1 -- 172.21.15.52:0/1441904563 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x5565be289fe0 con 0x5565be3a6710 2024-03-12T17:08:17.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.938+0000 7fb950c00640 1 -- 172.21.15.52:0/1441904563 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb958017070 con 0x5565be3a6710 2024-03-12T17:08:17.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.938+0000 7fb950c00640 1 -- 172.21.15.52:0/1441904563 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 17) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fb954026c80 con 0x5565be3a6710 2024-03-12T17:08:17.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.938+0000 7fb970ffb740 10 client.5526 fetch_fsmap finished waiting for FSMap version 17 2024-03-12T17:08:17.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.938+0000 7fb970ffb740 10 client.5526 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-12T17:08:17.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.938+0000 7fb970ffb740 20 client.5526 populate_metadata read hostname 'smithi052' 2024-03-12T17:08:17.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.938+0000 7fb970ffb740 1 --2- 172.21.15.52:0/1441904563 >> [v2:172.21.15.52:6834/543567396,v1:172.21.15.52:6835/543567396] conn(0x5565be3a6af0 0x5565be3a6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:17.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.938+0000 7fb970ffb740 4 client.5526 mds_command: new command op to 5343 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-12T17:08:17.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.938+0000 7fb970ffb740 1 -- 172.21.15.52:0/1441904563 --> [v2:172.21.15.52:6834/543567396,v1:172.21.15.52:6835/543567396] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5565bdfb83c0 con 0x5565be3a6af0 2024-03-12T17:08:17.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.939+0000 7fb967400640 1 --2- 172.21.15.52:0/1441904563 >> [v2:172.21.15.52:6834/543567396,v1:172.21.15.52:6835/543567396] conn(0x5565be3a6af0 0x5565be3a6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:17.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.939+0000 7fb967400640 1 --2- 172.21.15.52:0/1441904563 >> [v2:172.21.15.52:6834/543567396,v1:172.21.15.52:6835/543567396] conn(0x5565be3a6af0 0x5565be3a6ed0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d09bed271770e8be server_cookie=6e477871f34ce051 in_seq=0 out_seq=0 2024-03-12T17:08:17.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.939+0000 7fb950c00640 10 client.5526 ms_handle_connect on v2:172.21.15.52:6834/543567396 2024-03-12T17:08:17.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.940+0000 7fb950c00640 1 -- 172.21.15.52:0/1441904563 <== mds.0 v2:172.21.15.52:6834/543567396 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5565bdfb83c0 con 0x5565be3a6af0 2024-03-12T17:08:17.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.940+0000 7fb950c00640 10 client.5526 handle_command_reply: tid=0 2024-03-12T17:08:17.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.940+0000 7fb970ffb740 1 client.5526 shutdown 2024-03-12T17:08:17.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.940+0000 7fb970ffb740 1 -- 172.21.15.52:0/1441904563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565be3a6710 msgr2=0x5565be36be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:17.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.940+0000 7fb970ffb740 1 --2- 172.21.15.52:0/1441904563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565be3a6710 0x5565be36be80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb954005870 tx=0x7fb954001080 comp rx=0 tx=0).stop 2024-03-12T17:08:17.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.940+0000 7fb970ffb740 1 -- 172.21.15.52:0/1441904563 shutdown_connections 2024-03-12T17:08:17.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.940+0000 7fb970ffb740 1 --2- 172.21.15.52:0/1441904563 >> [v2:172.21.15.52:6834/543567396,v1:172.21.15.52:6835/543567396] conn(0x5565be3a6af0 0x5565be3a6ed0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.940+0000 7fb970ffb740 1 --2- 172.21.15.52:0/1441904563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565be3a8200 0x5565be3bf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.940+0000 7fb970ffb740 1 --2- 172.21.15.52:0/1441904563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565be3a6710 0x5565be36be80 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.940+0000 7fb970ffb740 1 --2- 172.21.15.52:0/1441904563 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5565be35e280 0x5565be410b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.940+0000 7fb950c00640 0 client.5526 ms_handle_reset on v2:172.21.15.52:6834/543567396 2024-03-12T17:08:17.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.940+0000 7fb970ffb740 1 -- 172.21.15.52:0/1441904563 >> 172.21.15.52:0/1441904563 conn(0x5565be35a720 msgr2=0x5565be355f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:17.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.940+0000 7fb970ffb740 1 -- 172.21.15.52:0/1441904563 shutdown_connections 2024-03-12T17:08:17.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.940+0000 7fb970ffb740 1 -- 172.21.15.52:0/1441904563 wait complete. 2024-03-12T17:08:17.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.940+0000 7fb970ffb740 20 client.5526 trim_cache size 0 max 16384 2024-03-12T17:08:17.938 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-03-12T17:08:17.938 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3", 2024-03-12T17:08:17.939 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 0, 2024-03-12T17:08:17.939 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 5343, 2024-03-12T17:08:17.939 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-03-12T17:08:17.939 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-03-12T17:08:17.939 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-03-12T17:08:17.939 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 7.2556519589999997, 2024-03-12T17:08:17.939 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 17, 2024-03-12T17:08:17.939 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 40, 2024-03-12T17:08:17.939 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 40, 2024-03-12T17:08:17.939 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 15.470344917 2024-03-12T17:08:17.940 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-03-12T17:08:17.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.941+0000 7fb96ce00640 1 -- 172.21.15.52:0/1942242707 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb94806ac00 msgr2=0x7fb94806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:17.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.941+0000 7fb96ce00640 1 --2- 172.21.15.52:0/1942242707 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb94806ac00 0x7fb94806d0c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb958006fd0 tx=0x7fb958008040 comp rx=0 tx=0).stop 2024-03-12T17:08:17.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.941+0000 7fb96ce00640 1 -- 172.21.15.52:0/1942242707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb96815cd00 msgr2=0x7fb96813cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:17.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.941+0000 7fb96ce00640 1 --2- 172.21.15.52:0/1942242707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb96815cd00 0x7fb96813cc20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb960009d90 tx=0x7fb960004a40 comp rx=0 tx=0).stop 2024-03-12T17:08:17.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.941+0000 7fb96ce00640 1 -- 172.21.15.52:0/1942242707 shutdown_connections 2024-03-12T17:08:17.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.941+0000 7fb96ce00640 1 --2- 172.21.15.52:0/1942242707 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb94806ac00 0x7fb94806d0c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.941+0000 7fb96ce00640 1 --2- 172.21.15.52:0/1942242707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb96815cd00 0x7fb96813cc20 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.941+0000 7fb96ce00640 1 --2- 172.21.15.52:0/1942242707 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb968146070 0x7fb96813c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.941+0000 7fb96ce00640 1 --2- 172.21.15.52:0/1942242707 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb9680707e0 0x7fb96814f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:17.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.941+0000 7fb96ce00640 1 -- 172.21.15.52:0/1942242707 >> 172.21.15.52:0/1942242707 conn(0x7fb968065df0 msgr2=0x7fb96806c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:17.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.941+0000 7fb96ce00640 1 -- 172.21.15.52:0/1942242707 shutdown_connections 2024-03-12T17:08:17.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:17.941+0000 7fb96ce00640 1 -- 172.21.15.52:0/1942242707 wait complete. 2024-03-12T17:08:17.952 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-12T17:08:18.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.173+0000 7fac2ec00640 1 Processor -- start 2024-03-12T17:08:18.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.173+0000 7fac2ec00640 1 -- start start 2024-03-12T17:08:18.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.175+0000 7fac2ec00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fac2815a940 0x7fac28099450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:18.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.175+0000 7fac2ec00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac280cc320 0x7fac280c4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:18.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.175+0000 7fac2ec00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac280cba30 con 0x7fac280cbec0 2024-03-12T17:08:18.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.175+0000 7fac2ec00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac2815ad20 con 0x7fac2815a940 2024-03-12T17:08:18.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.175+0000 7fac2ec00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac280cb140 con 0x7fac280cc320 2024-03-12T17:08:18.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.176+0000 7fac27e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fac2815a940 0x7fac28099450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:18.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.176+0000 7fac2cc00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac280cc320 0x7fac280c4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:18.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.176+0000 7fac2cc00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac280cc320 0x7fac280c4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.52:43694/0 (socket says 172.21.15.52:43694) 2024-03-12T17:08:18.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.176+0000 7fac2cc00640 1 -- 172.21.15.52:0/1525331665 learned_addr learned my addr 172.21.15.52:0/1525331665 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:18.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.177+0000 7fac27e00640 1 -- 172.21.15.52:0/1525331665 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac280cc320 msgr2=0x7fac280c4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:18.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.177+0000 7fac27e00640 1 --2- 172.21.15.52:0/1525331665 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac280cc320 0x7fac280c4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:18.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.177+0000 7fac27e00640 1 -- 172.21.15.52:0/1525331665 >> v1:172.21.15.102:6789/0 conn(0x7fac280cbec0 legacy=0x7fac280c4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:18.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.177+0000 7fac27e00640 1 -- 172.21.15.52:0/1525331665 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac28099ae0 con 0x7fac2815a940 2024-03-12T17:08:18.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.177+0000 7fac2cc00640 1 --2- 172.21.15.52:0/1525331665 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac280cc320 0x7fac280c4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T17:08:18.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.178+0000 7fac27e00640 1 --2- 172.21.15.52:0/1525331665 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac2815a940 0x7fac28099450 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fac18009d90 tx=0x7fac1802bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dcfc8377186b3f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:18.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.179+0000 7fac27400640 1 -- 172.21.15.52:0/1525331665 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1802cc50 con 0x7fac2815a940 2024-03-12T17:08:18.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.179+0000 7fac27400640 1 -- 172.21.15.52:0/1525331665 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1802cdb0 con 0x7fac2815a940 2024-03-12T17:08:18.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.179+0000 7fac27400640 1 -- 172.21.15.52:0/1525331665 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac18034660 con 0x7fac2815a940 2024-03-12T17:08:18.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.180+0000 7fac2ec00640 1 -- 172.21.15.52:0/1525331665 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac2815a940 msgr2=0x7fac28099450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:18.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.180+0000 7fac2ec00640 1 --2- 172.21.15.52:0/1525331665 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac2815a940 0x7fac28099450 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fac18009d90 tx=0x7fac1802bea0 comp rx=0 tx=0).stop 2024-03-12T17:08:18.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.180+0000 7fac2ec00640 1 -- 172.21.15.52:0/1525331665 shutdown_connections 2024-03-12T17:08:18.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.180+0000 7fac2ec00640 1 --2- 172.21.15.52:0/1525331665 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac280cc320 0x7fac280c4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:18.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.180+0000 7fac2ec00640 1 --2- 172.21.15.52:0/1525331665 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac2815a940 0x7fac28099450 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:18.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.180+0000 7fac2ec00640 1 -- 172.21.15.52:0/1525331665 >> 172.21.15.52:0/1525331665 conn(0x7fac280b2120 msgr2=0x7fac280b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:18.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.180+0000 7fac2ec00640 1 -- 172.21.15.52:0/1525331665 shutdown_connections 2024-03-12T17:08:18.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.180+0000 7fac2ec00640 1 -- 172.21.15.52:0/1525331665 wait complete. 2024-03-12T17:08:18.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.182+0000 7fac2ec00640 1 Processor -- start 2024-03-12T17:08:18.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.183+0000 7fac2ec00640 1 -- start start 2024-03-12T17:08:18.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.184+0000 7fac2ec00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac280cbec0 0x7fac280ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:18.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.184+0000 7fac2ec00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac2815a940 0x7fac280b4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:18.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.184+0000 7fac2ec00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac281580a0 0x7fac281f5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:18.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.184+0000 7fac2ec00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac280ca550 con 0x7fac281580a0 2024-03-12T17:08:18.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.184+0000 7fac2ec00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2815af90 con 0x7fac280cbec0 2024-03-12T17:08:18.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.184+0000 7fac2ec00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac2815b270 con 0x7fac2815a940 2024-03-12T17:08:18.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.184+0000 7fac2cc00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac281580a0 0x7fac281f5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:18.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.184+0000 7fac2cc00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac281580a0 0x7fac281f5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37044/0 (socket says 172.21.15.52:37044) 2024-03-12T17:08:18.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.184+0000 7fac2cc00640 1 -- 172.21.15.52:0/1305964214 learned_addr learned my addr 172.21.15.52:0/1305964214 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:18.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.185+0000 7fac27e00640 1 --2- 172.21.15.52:0/1305964214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac280cbec0 0x7fac280ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:18.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.185+0000 7fac1f400640 1 --2- 172.21.15.52:0/1305964214 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac2815a940 0x7fac280b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:18.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.185+0000 7fac2cc00640 1 -- 172.21.15.52:0/1305964214 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac2815a940 msgr2=0x7fac280b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:18.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.185+0000 7fac2cc00640 1 --2- 172.21.15.52:0/1305964214 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac2815a940 0x7fac280b4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:18.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.185+0000 7fac2cc00640 1 -- 172.21.15.52:0/1305964214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac280cbec0 msgr2=0x7fac280ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:18.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.185+0000 7fac2cc00640 1 --2- 172.21.15.52:0/1305964214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac280cbec0 0x7fac280ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:18.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.185+0000 7fac2cc00640 1 -- 172.21.15.52:0/1305964214 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac200094d0 con 0x7fac281580a0 2024-03-12T17:08:18.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.185+0000 7fac2cc00640 1 --2- 172.21.15.52:0/1305964214 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac281580a0 0x7fac281f5bb0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fac2000e980 tx=0x7fac2000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:18.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.185+0000 7fac24c00640 1 -- 172.21.15.52:0/1305964214 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2000ca00 con 0x7fac281580a0 2024-03-12T17:08:18.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.185+0000 7fac24c00640 1 -- 172.21.15.52:0/1305964214 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac2000cb80 con 0x7fac281580a0 2024-03-12T17:08:18.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.185+0000 7fac24c00640 1 -- 172.21.15.52:0/1305964214 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac20013470 con 0x7fac281580a0 2024-03-12T17:08:18.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.185+0000 7fac2ec00640 1 -- 172.21.15.52:0/1305964214 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac18009a70 con 0x7fac281580a0 2024-03-12T17:08:18.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.186+0000 7fac2ec00640 1 -- 172.21.15.52:0/1305964214 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac2815df20 con 0x7fac281580a0 2024-03-12T17:08:18.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.187+0000 7fac2ec00640 1 -- 172.21.15.52:0/1305964214 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac28158960 con 0x7fac281580a0 2024-03-12T17:08:18.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.188+0000 7fac24c00640 1 -- 172.21.15.52:0/1305964214 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fac200135d0 con 0x7fac281580a0 2024-03-12T17:08:18.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.189+0000 7fac24c00640 1 --2- 172.21.15.52:0/1305964214 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fac0406ac00 0x7fac0406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:18.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.189+0000 7fac24c00640 1 -- 172.21.15.52:0/1305964214 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fac20089810 con 0x7fac281580a0 2024-03-12T17:08:18.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.189+0000 7fac27e00640 1 --2- 172.21.15.52:0/1305964214 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fac0406ac00 0x7fac0406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:18.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.191+0000 7fac27e00640 1 --2- 172.21.15.52:0/1305964214 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fac0406ac00 0x7fac0406d0c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fac18036000 tx=0x7fac18030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:18.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.193+0000 7fac24c00640 1 -- 172.21.15.52:0/1305964214 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fac28158960 con 0x7fac281580a0 2024-03-12T17:08:18.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.435+0000 7fac2ec00640 1 -- 172.21.15.52:0/1305964214 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fac28158960 con 0x7fac281580a0 2024-03-12T17:08:18.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.436+0000 7fac24c00640 1 -- 172.21.15.52:0/1305964214 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7fac28158960 con 0x7fac281580a0 2024-03-12T17:08:18.430 INFO:teuthology.orchestra.run.smithi052.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-12T17:08:18.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.439+0000 7fac2ec00640 1 -- 172.21.15.52:0/1305964214 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fac0406ac00 msgr2=0x7fac0406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:18.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.439+0000 7fac2ec00640 1 --2- 172.21.15.52:0/1305964214 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fac0406ac00 0x7fac0406d0c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fac18036000 tx=0x7fac18030000 comp rx=0 tx=0).stop 2024-03-12T17:08:18.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.439+0000 7fac2ec00640 1 -- 172.21.15.52:0/1305964214 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac281580a0 msgr2=0x7fac281f5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:18.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.439+0000 7fac2ec00640 1 --2- 172.21.15.52:0/1305964214 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac281580a0 0x7fac281f5bb0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fac2000e980 tx=0x7fac2000ee40 comp rx=0 tx=0).stop 2024-03-12T17:08:18.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.439+0000 7fac2ec00640 1 -- 172.21.15.52:0/1305964214 shutdown_connections 2024-03-12T17:08:18.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.439+0000 7fac2ec00640 1 --2- 172.21.15.52:0/1305964214 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fac0406ac00 0x7fac0406d0c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:18.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.439+0000 7fac2ec00640 1 --2- 172.21.15.52:0/1305964214 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fac2815a940 0x7fac280b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:18.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.439+0000 7fac2ec00640 1 --2- 172.21.15.52:0/1305964214 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac281580a0 0x7fac281f5bb0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:18.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.439+0000 7fac2ec00640 1 --2- 172.21.15.52:0/1305964214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac280cbec0 0x7fac280ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:18.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.439+0000 7fac2ec00640 1 -- 172.21.15.52:0/1305964214 >> 172.21.15.52:0/1305964214 conn(0x7fac280b2120 msgr2=0x7fac28098ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:18.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.440+0000 7fac2ec00640 1 -- 172.21.15.52:0/1305964214 shutdown_connections 2024-03-12T17:08:18.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:18.440+0000 7fac2ec00640 1 -- 172.21.15.52:0/1305964214 wait complete. 2024-03-12T17:08:18.452 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-12T17:08:18.452 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-12T17:08:18.453 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi052.front.sepia.ceph.com 2024-03-12T17:08:18.453 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-12T17:08:18.453 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-12T17:08:18.453 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-12T17:08:18.453 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-12T17:08:18.453 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-12T17:08:18.453 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-12T17:08:18.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:08:18.454 DEBUG:teuthology.orchestra.run.smithi052:> ip addr 2024-03-12T17:08:18.507 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-12T17:08:18.507 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-12T17:08:18.507 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-03-12T17:08:18.507 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:08:18.507 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-03-12T17:08:18.507 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:08:18.507 INFO:teuthology.orchestra.run.smithi052.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-12T17:08:18.507 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d2 brd ff:ff:ff:ff:ff:ff 2024-03-12T17:08:18.507 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f0 2024-03-12T17:08:18.507 INFO:teuthology.orchestra.run.smithi052.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-12T17:08:18.507 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d3 brd ff:ff:ff:ff:ff:ff 2024-03-12T17:08:18.508 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f1 2024-03-12T17:08:18.508 INFO:teuthology.orchestra.run.smithi052.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-12T17:08:18.508 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:de brd ff:ff:ff:ff:ff:ff 2024-03-12T17:08:18.508 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f0 2024-03-12T17:08:18.508 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.21.15.52/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-12T17:08:18.508 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft 42496sec preferred_lft 42496sec 2024-03-12T17:08:18.508 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::ec4:7aff:febd:15de/64 scope link noprefixroute 2024-03-12T17:08:18.508 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:08:18.508 INFO:teuthology.orchestra.run.smithi052.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-12T17:08:18.508 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:df brd ff:ff:ff:ff:ff:ff 2024-03-12T17:08:18.508 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f1 2024-03-12T17:08:18.508 INFO:teuthology.orchestra.run.smithi052.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-12T17:08:18.508 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 72:aa:0f:4e:37:01 brd ff:ff:ff:ff:ff:ff 2024-03-12T17:08:18.509 INFO:teuthology.orchestra.run.smithi052.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-12T17:08:18.509 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:08:18.509 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::1896:69ff:fe28:1273/64 scope link 2024-03-12T17:08:18.509 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:08:18.509 INFO:teuthology.orchestra.run.smithi052.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-12T17:08:18.509 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 72:aa:0f:4e:37:01 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-12T17:08:18.509 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::70aa:fff:fe4e:3701/64 scope link 2024-03-12T17:08:18.509 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:08:18.509 INFO:teuthology.orchestra.run.smithi052.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-12T17:08:18.509 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether b2:70:60:b7:b6:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-12T17:08:18.509 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::b070:60ff:feb7:b63c/64 scope link 2024-03-12T17:08:18.509 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-03-12T17:08:18.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:08:18.510 DEBUG:teuthology.orchestra.run.smithi052:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-12T17:08:18.605 INFO:teuthology.orchestra.run.smithi052.stdout:1 2024-03-12T17:08:18.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:08:18.609 DEBUG:teuthology.orchestra.run.smithi052:> route 2024-03-12T17:08:18.666 INFO:teuthology.orchestra.run.smithi052.stdout:Kernel IP routing table 2024-03-12T17:08:18.666 INFO:teuthology.orchestra.run.smithi052.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-12T17:08:18.667 INFO:teuthology.orchestra.run.smithi052.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-12T17:08:18.667 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-12T17:08:18.667 INFO:teuthology.orchestra.run.smithi052.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-12T17:08:18.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:08:18.667 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec bash -c ' 2024-03-12T17:08:18.667 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-03-12T17:08:18.667 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-12T17:08:18.667 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-12T17:08:18.667 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-12T17:08:18.668 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-03-12T17:08:18.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:08:18.856 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-03-12T17:08:18.910 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-12T17:08:18.911 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-12T17:08:18.912 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-12T17:08:18.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:08:18.912 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set brx.0 up 2024-03-12T17:08:19.009 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-12T17:08:19.009 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-12T17:08:19.064 INFO:teuthology.orchestra.run.smithi052.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-12T17:08:19.064 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-12T17:08:19.064 DEBUG:teuthology.orchestra.run.smithi052:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-12T17:08:19.118 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-12T17:08:19.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:08:19.119 DEBUG:teuthology.orchestra.run.smithi052:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-12T17:08:19.252 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-12T17:08:19.252 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-12T17:08:19.289 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-12T17:08:19.289 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-12T17:08:19.289 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: resolved to: "172.21.15.52:3300,172.21.15.102:3300,172.21.15.102:3301" 2024-03-12T17:08:19.289 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: trying mount with new device syntax: 0@524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3.cephfs=/ 2024-03-12T17:08:19.290 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.52:3300/172.21.15.102:3300/172.21.15.102:3301" will pass to kernel 2024-03-12T17:08:19.292 INFO:teuthology.orchestra.run.smithi052.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-12T17:08:19.292 INFO:teuthology.orchestra.run.smithi052.stdout: You just mounted a file system that supports labels which does not 2024-03-12T17:08:19.293 INFO:teuthology.orchestra.run.smithi052.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-12T17:08:19.293 INFO:teuthology.orchestra.run.smithi052.stdout: applications will generate AVC messages and not be allowed access to 2024-03-12T17:08:19.293 INFO:teuthology.orchestra.run.smithi052.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-12T17:08:19.298 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-12T17:08:19.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:08:19.298 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-12T17:08:19.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:08:19.383 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-12T17:08:19.455 INFO:teuthology.orchestra.run.smithi052.stdout:client5014 2024-03-12T17:08:19.455 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:08:19.455 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-12T17:08:19.512 DEBUG:teuthology.orchestra.run.smithi052:> mktemp -d --suffix=xfstests-dev 2024-03-12T17:08:19.562 INFO:teuthology.orchestra.run.smithi052.stdout:/tmp/tmp.lWf7ZzDCDzxfstests-dev 2024-03-12T17:08:19.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:08:19.563 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone https://git.ceph.com/xfstests-dev.git --depth 1 /tmp/tmp.lWf7ZzDCDzxfstests-dev) 2024-03-12T17:08:19.622 INFO:teuthology.orchestra.run.smithi052.stderr:Cloning into '/tmp/tmp.lWf7ZzDCDzxfstests-dev'... 2024-03-12T17:08:20.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:08:20.695 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test) 2024-03-12T17:08:20.762 DEBUG:teuthology.orchestra.run.smithi052:> mktemp -d --suffix=test 2024-03-12T17:08:20.813 INFO:teuthology.orchestra.run.smithi052.stdout:/tmp/tmp.RgkcaM9vDRtest 2024-03-12T17:08:20.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:08:20.814 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scratch) 2024-03-12T17:08:20.866 DEBUG:teuthology.orchestra.run.smithi052:> mktemp -d --suffix=scratch 2024-03-12T17:08:20.924 INFO:teuthology.orchestra.run.smithi052.stdout:/tmp/tmp.fpQ4ymYXq9scratch 2024-03-12T17:08:20.925 INFO:tasks.cephfs.xfstests_dev:distro and version detected is "centos" and "9". 2024-03-12T17:08:20.925 DEBUG:teuthology.orchestra.run.smithi052:> sudo yum install -y acl attr automake bc dbench dump e2fsprogs fio gawk gcc indent libtool lvm2 make psmisc quota sed xfsdump xfsprogs libacl-devel libattr-devel libaio-devel libuuid-devel xfsprogs-devel python3 sqlite xfsprogs-qa-devel 2024-03-12T17:08:21.820 INFO:teuthology.orchestra.run.smithi052.stdout:Last metadata expiration check: 0:04:45 ago on Tue 12 Mar 2024 05:03:36 PM UTC. 2024-03-12T17:08:21.928 INFO:teuthology.orchestra.run.smithi052.stdout:Package acl-2.3.1-4.el9.x86_64 is already installed. 2024-03-12T17:08:21.929 INFO:teuthology.orchestra.run.smithi052.stdout:Package attr-2.5.1-3.el9.x86_64 is already installed. 2024-03-12T17:08:21.930 INFO:teuthology.orchestra.run.smithi052.stdout:Package automake-1.16.2-8.el9.noarch is already installed. 2024-03-12T17:08:21.931 INFO:teuthology.orchestra.run.smithi052.stdout:Package bc-1.07.1-14.el9.x86_64 is already installed. 2024-03-12T17:08:21.933 INFO:teuthology.orchestra.run.smithi052.stdout:Package dbench-4.0-25.el9.x86_64 is already installed. 2024-03-12T17:08:21.935 INFO:teuthology.orchestra.run.smithi052.stdout:Package e2fsprogs-1.46.5-3.el9.x86_64 is already installed. 2024-03-12T17:08:21.937 INFO:teuthology.orchestra.run.smithi052.stdout:Package gawk-5.1.0-6.el9.x86_64 is already installed. 2024-03-12T17:08:21.938 INFO:teuthology.orchestra.run.smithi052.stdout:Package gcc-11.4.1-2.1.el9.x86_64 is already installed. 2024-03-12T17:08:21.940 INFO:teuthology.orchestra.run.smithi052.stdout:Package libtool-2.4.6-45.el9.x86_64 is already installed. 2024-03-12T17:08:21.941 INFO:teuthology.orchestra.run.smithi052.stdout:Package lvm2-9:2.03.21-3.el9.x86_64 is already installed. 2024-03-12T17:08:21.942 INFO:teuthology.orchestra.run.smithi052.stdout:Package make-1:4.3-7.el9.x86_64 is already installed. 2024-03-12T17:08:21.944 INFO:teuthology.orchestra.run.smithi052.stdout:Package psmisc-23.4-3.el9.x86_64 is already installed. 2024-03-12T17:08:21.945 INFO:teuthology.orchestra.run.smithi052.stdout:Package quota-1:4.06-6.el9.x86_64 is already installed. 2024-03-12T17:08:21.946 INFO:teuthology.orchestra.run.smithi052.stdout:Package sed-4.8-9.el9.x86_64 is already installed. 2024-03-12T17:08:21.947 INFO:teuthology.orchestra.run.smithi052.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-03-12T17:08:21.948 INFO:teuthology.orchestra.run.smithi052.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-03-12T17:08:21.950 INFO:teuthology.orchestra.run.smithi052.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-03-12T17:08:21.950 INFO:teuthology.orchestra.run.smithi052.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-12T17:08:21.951 INFO:teuthology.orchestra.run.smithi052.stdout:Package libaio-devel-0.3.111-13.el9.x86_64 is already installed. 2024-03-12T17:08:21.953 INFO:teuthology.orchestra.run.smithi052.stdout:Package libuuid-devel-2.37.4-18.el9.x86_64 is already installed. 2024-03-12T17:08:21.954 INFO:teuthology.orchestra.run.smithi052.stdout:Package xfsprogs-devel-6.3.0-1.el9.x86_64 is already installed. 2024-03-12T17:08:21.955 INFO:teuthology.orchestra.run.smithi052.stdout:Package python3-3.9.17-2.el9.x86_64 is already installed. 2024-03-12T17:08:21.960 INFO:teuthology.orchestra.run.smithi052.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-03-12T17:08:22.032 INFO:teuthology.orchestra.run.smithi052.stdout:Dependencies resolved. 2024-03-12T17:08:22.035 INFO:teuthology.orchestra.run.smithi052.stdout:================================================================================ 2024-03-12T17:08:22.035 INFO:teuthology.orchestra.run.smithi052.stdout: Package Arch Version Repository Size 2024-03-12T17:08:22.035 INFO:teuthology.orchestra.run.smithi052.stdout:================================================================================ 2024-03-12T17:08:22.035 INFO:teuthology.orchestra.run.smithi052.stdout:Installing: 2024-03-12T17:08:22.035 INFO:teuthology.orchestra.run.smithi052.stdout: dump x86_64 1:0.4-0.51.b47.el9 epel 88 k 2024-03-12T17:08:22.036 INFO:teuthology.orchestra.run.smithi052.stdout: fio x86_64 3.35-1.el9 CentOS-AppStream 6.2 M 2024-03-12T17:08:22.036 INFO:teuthology.orchestra.run.smithi052.stdout: indent x86_64 2.2.13-5.el9 epel 179 k 2024-03-12T17:08:22.036 INFO:teuthology.orchestra.run.smithi052.stdout: sqlite x86_64 3.34.1-7.el9 CentOS-AppStream 747 k 2024-03-12T17:08:22.036 INFO:teuthology.orchestra.run.smithi052.stdout:Upgrading: 2024-03-12T17:08:22.036 INFO:teuthology.orchestra.run.smithi052.stdout: cpp x86_64 11.4.1-3.el9 appstream 11 M 2024-03-12T17:08:22.036 INFO:teuthology.orchestra.run.smithi052.stdout: device-mapper x86_64 9:1.02.197-2.el9 baseos 141 k 2024-03-12T17:08:22.036 INFO:teuthology.orchestra.run.smithi052.stdout: device-mapper-event x86_64 9:1.02.197-2.el9 baseos 34 k 2024-03-12T17:08:22.036 INFO:teuthology.orchestra.run.smithi052.stdout: device-mapper-event-libs x86_64 9:1.02.197-2.el9 baseos 32 k 2024-03-12T17:08:22.036 INFO:teuthology.orchestra.run.smithi052.stdout: device-mapper-libs x86_64 9:1.02.197-2.el9 baseos 179 k 2024-03-12T17:08:22.036 INFO:teuthology.orchestra.run.smithi052.stdout: e2fsprogs x86_64 1.46.5-5.el9 baseos 1.0 M 2024-03-12T17:08:22.036 INFO:teuthology.orchestra.run.smithi052.stdout: e2fsprogs-libs x86_64 1.46.5-5.el9 baseos 222 k 2024-03-12T17:08:22.036 INFO:teuthology.orchestra.run.smithi052.stdout: gcc x86_64 11.4.1-3.el9 appstream 32 M 2024-03-12T17:08:22.036 INFO:teuthology.orchestra.run.smithi052.stdout: gcc-c++ x86_64 11.4.1-3.el9 appstream 13 M 2024-03-12T17:08:22.036 INFO:teuthology.orchestra.run.smithi052.stdout: gcc-plugin-annobin x86_64 11.4.1-3.el9 appstream 47 k 2024-03-12T17:08:22.037 INFO:teuthology.orchestra.run.smithi052.stdout: libcom_err x86_64 1.46.5-5.el9 baseos 26 k 2024-03-12T17:08:22.037 INFO:teuthology.orchestra.run.smithi052.stdout: libgcc x86_64 11.4.1-3.el9 baseos 92 k 2024-03-12T17:08:22.037 INFO:teuthology.orchestra.run.smithi052.stdout: libgomp x86_64 11.4.1-3.el9 baseos 268 k 2024-03-12T17:08:22.037 INFO:teuthology.orchestra.run.smithi052.stdout: libss x86_64 1.46.5-5.el9 baseos 32 k 2024-03-12T17:08:22.037 INFO:teuthology.orchestra.run.smithi052.stdout: libstdc++ x86_64 11.4.1-3.el9 baseos 743 k 2024-03-12T17:08:22.037 INFO:teuthology.orchestra.run.smithi052.stdout: libstdc++-devel x86_64 11.4.1-3.el9 appstream 2.4 M 2024-03-12T17:08:22.037 INFO:teuthology.orchestra.run.smithi052.stdout: lvm2 x86_64 9:2.03.23-2.el9 baseos 1.5 M 2024-03-12T17:08:22.037 INFO:teuthology.orchestra.run.smithi052.stdout: lvm2-libs x86_64 9:2.03.23-2.el9 baseos 1.0 M 2024-03-12T17:08:22.037 INFO:teuthology.orchestra.run.smithi052.stdout: make x86_64 1:4.3-8.el9 baseos 536 k 2024-03-12T17:08:22.037 INFO:teuthology.orchestra.run.smithi052.stdout: python-unversioned-command noarch 3.9.18-3.el9 CentOS-AppStream 8.3 k 2024-03-12T17:08:22.037 INFO:teuthology.orchestra.run.smithi052.stdout: python3 x86_64 3.9.18-3.el9 CentOS-BaseOS 25 k 2024-03-12T17:08:22.037 INFO:teuthology.orchestra.run.smithi052.stdout: python3-devel x86_64 3.9.18-3.el9 CentOS-AppStream 204 k 2024-03-12T17:08:22.037 INFO:teuthology.orchestra.run.smithi052.stdout: python3-libs x86_64 3.9.18-3.el9 CentOS-BaseOS 7.3 M 2024-03-12T17:08:22.037 INFO:teuthology.orchestra.run.smithi052.stdout: sqlite-libs x86_64 3.34.1-7.el9 CentOS-BaseOS 619 k 2024-03-12T17:08:22.038 INFO:teuthology.orchestra.run.smithi052.stdout:Installing dependencies: 2024-03-12T17:08:22.038 INFO:teuthology.orchestra.run.smithi052.stdout: libnbd x86_64 1.18.1-3.el9 appstream 166 k 2024-03-12T17:08:22.038 INFO:teuthology.orchestra.run.smithi052.stdout: rmt x86_64 2:1.6-6.el9 CentOS-BaseOS 45 k 2024-03-12T17:08:22.038 INFO:teuthology.orchestra.run.smithi052.stdout:Installing weak dependencies: 2024-03-12T17:08:22.038 INFO:teuthology.orchestra.run.smithi052.stdout: fio-engine-http x86_64 3.35-1.el9 CentOS-AppStream 17 k 2024-03-12T17:08:22.038 INFO:teuthology.orchestra.run.smithi052.stdout: fio-engine-libaio x86_64 3.35-1.el9 CentOS-AppStream 14 k 2024-03-12T17:08:22.038 INFO:teuthology.orchestra.run.smithi052.stdout: fio-engine-nbd x86_64 3.35-1.el9 CentOS-AppStream 13 k 2024-03-12T17:08:22.038 INFO:teuthology.orchestra.run.smithi052.stdout: fio-engine-rados x86_64 3.35-1.el9 CentOS-AppStream 15 k 2024-03-12T17:08:22.038 INFO:teuthology.orchestra.run.smithi052.stdout: fio-engine-rbd x86_64 3.35-1.el9 CentOS-AppStream 15 k 2024-03-12T17:08:22.038 INFO:teuthology.orchestra.run.smithi052.stdout: fio-engine-rdma x86_64 3.35-1.el9 CentOS-AppStream 20 k 2024-03-12T17:08:22.038 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:08:22.038 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction Summary 2024-03-12T17:08:22.038 INFO:teuthology.orchestra.run.smithi052.stdout:================================================================================ 2024-03-12T17:08:22.039 INFO:teuthology.orchestra.run.smithi052.stdout:Install 12 Packages 2024-03-12T17:08:22.039 INFO:teuthology.orchestra.run.smithi052.stdout:Upgrade 24 Packages 2024-03-12T17:08:22.039 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:08:22.039 INFO:teuthology.orchestra.run.smithi052.stdout:Total download size: 80 M 2024-03-12T17:08:22.039 INFO:teuthology.orchestra.run.smithi052.stdout:Downloading Packages: 2024-03-12T17:08:22.682 INFO:teuthology.orchestra.run.smithi052.stdout:(1/36): fio-engine-http-3.35-1.el9.x86_64.rpm 52 kB/s | 17 kB 00:00 2024-03-12T17:08:22.707 INFO:teuthology.orchestra.run.smithi052.stdout:(2/36): fio-engine-libaio-3.35-1.el9.x86_64.rpm 41 kB/s | 14 kB 00:00 2024-03-12T17:08:22.891 INFO:teuthology.orchestra.run.smithi052.stdout:(3/36): fio-engine-nbd-3.35-1.el9.x86_64.rpm 65 kB/s | 13 kB 00:00 2024-03-12T17:08:22.916 INFO:teuthology.orchestra.run.smithi052.stdout:(4/36): fio-engine-rados-3.35-1.el9.x86_64.rpm 71 kB/s | 15 kB 00:00 2024-03-12T17:08:23.091 INFO:teuthology.orchestra.run.smithi052.stdout:(5/36): fio-3.35-1.el9.x86_64.rpm 8.5 MB/s | 6.2 MB 00:00 2024-03-12T17:08:23.117 INFO:teuthology.orchestra.run.smithi052.stdout:(6/36): fio-engine-rbd-3.35-1.el9.x86_64.rpm 68 kB/s | 15 kB 00:00 2024-03-12T17:08:23.141 INFO:teuthology.orchestra.run.smithi052.stdout:(7/36): fio-engine-rdma-3.35-1.el9.x86_64.rpm 93 kB/s | 20 kB 00:00 2024-03-12T17:08:23.184 INFO:teuthology.orchestra.run.smithi052.stdout:(8/36): rmt-1.6-6.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-03-12T17:08:23.226 INFO:teuthology.orchestra.run.smithi052.stdout:(9/36): sqlite-3.34.1-7.el9.x86_64.rpm 8.9 MB/s | 747 kB 00:00 2024-03-12T17:08:23.342 INFO:teuthology.orchestra.run.smithi052.stdout:(10/36): libnbd-1.18.1-3.el9.x86_64.rpm 839 kB/s | 166 kB 00:00 2024-03-12T17:08:23.393 INFO:teuthology.orchestra.run.smithi052.stdout:(11/36): python-unversioned-command-3.9.18-3.el 168 kB/s | 8.3 kB 00:00 2024-03-12T17:08:23.443 INFO:teuthology.orchestra.run.smithi052.stdout:(12/36): python3-devel-3.9.18-3.el9.x86_64.rpm 4.0 MB/s | 204 kB 00:00 2024-03-12T17:08:23.477 INFO:teuthology.orchestra.run.smithi052.stdout:(13/36): python3-3.9.18-3.el9.x86_64.rpm 768 kB/s | 25 kB 00:00 2024-03-12T17:08:23.627 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-12T17:08:23.636 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-12T17:08:23.685 INFO:teuthology.orchestra.run.smithi052.stdout:(14/36): dump-0.4-0.51.b47.el9.x86_64.rpm 178 kB/s | 88 kB 00:00 2024-03-12T17:08:23.819 INFO:teuthology.orchestra.run.smithi052.stdout:(15/36): python3-libs-3.9.18-3.el9.x86_64.rpm 21 MB/s | 7.3 MB 00:00 2024-03-12T17:08:23.882 INFO:teuthology.orchestra.run.smithi052.stdout:(16/36): sqlite-libs-3.34.1-7.el9.x86_64.rpm 3.1 MB/s | 619 kB 00:00 2024-03-12T17:08:23.920 INFO:teuthology.orchestra.run.smithi052.stdout:(17/36): indent-2.2.13-5.el9.x86_64.rpm 259 kB/s | 179 kB 00:00 2024-03-12T17:08:23.987 INFO:teuthology.orchestra.run.smithi052.stdout:(18/36): device-mapper-event-1.02.197-2.el9.x86 332 kB/s | 34 kB 00:00 2024-03-12T17:08:24.012 INFO:teuthology.orchestra.run.smithi052.stdout:(19/36): device-mapper-event-libs-1.02.197-2.el 355 kB/s | 32 kB 00:00 2024-03-12T17:08:24.037 INFO:teuthology.orchestra.run.smithi052.stdout:(20/36): device-mapper-1.02.197-2.el9.x86_64.rp 665 kB/s | 141 kB 00:00 2024-03-12T17:08:24.088 INFO:teuthology.orchestra.run.smithi052.stdout:(21/36): device-mapper-libs-1.02.197-2.el9.x86_ 1.8 MB/s | 179 kB 00:00 2024-03-12T17:08:24.122 INFO:teuthology.orchestra.run.smithi052.stdout:(22/36): e2fsprogs-libs-1.46.5-5.el9.x86_64.rpm 2.6 MB/s | 222 kB 00:00 2024-03-12T17:08:24.147 INFO:teuthology.orchestra.run.smithi052.stdout:(23/36): libcom_err-1.46.5-5.el9.x86_64.rpm 457 kB/s | 26 kB 00:00 2024-03-12T17:08:24.172 INFO:teuthology.orchestra.run.smithi052.stdout:(24/36): libgcc-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 92 kB 00:00 2024-03-12T17:08:24.214 INFO:teuthology.orchestra.run.smithi052.stdout:(25/36): libgomp-11.4.1-3.el9.x86_64.rpm 4.0 MB/s | 268 kB 00:00 2024-03-12T17:08:24.256 INFO:teuthology.orchestra.run.smithi052.stdout:(26/36): e2fsprogs-1.46.5-5.el9.x86_64.rpm 4.1 MB/s | 1.0 MB 00:00 2024-03-12T17:08:24.281 INFO:teuthology.orchestra.run.smithi052.stdout:(27/36): libss-1.46.5-5.el9.x86_64.rpm 296 kB/s | 32 kB 00:00 2024-03-12T17:08:24.340 INFO:teuthology.orchestra.run.smithi052.stdout:(28/36): libstdc++-11.4.1-3.el9.x86_64.rpm 5.8 MB/s | 743 kB 00:00 2024-03-12T17:08:24.432 INFO:teuthology.orchestra.run.smithi052.stdout:(29/36): lvm2-2.03.23-2.el9.x86_64.rpm 8.8 MB/s | 1.5 MB 00:00 2024-03-12T17:08:24.474 INFO:teuthology.orchestra.run.smithi052.stdout:(30/36): lvm2-libs-2.03.23-2.el9.x86_64.rpm 5.2 MB/s | 1.0 MB 00:00 2024-03-12T17:08:24.516 INFO:teuthology.orchestra.run.smithi052.stdout:(31/36): make-4.3-8.el9.x86_64.rpm 3.0 MB/s | 536 kB 00:00 2024-03-12T17:08:25.600 INFO:teuthology.orchestra.run.smithi052.stdout:(32/36): cpp-11.4.1-3.el9.x86_64.rpm 9.1 MB/s | 11 MB 00:01 2024-03-12T17:08:25.658 INFO:teuthology.orchestra.run.smithi052.stdout:(33/36): gcc-plugin-annobin-11.4.1-3.el9.x86_64 824 kB/s | 47 kB 00:00 2024-03-12T17:08:25.950 INFO:teuthology.orchestra.run.smithi052.stdout:(34/36): libstdc++-devel-11.4.1-3.el9.x86_64.rp 8.3 MB/s | 2.4 MB 00:00 2024-03-12T17:08:26.160 INFO:teuthology.orchestra.run.smithi052.stdout:(35/36): gcc-c++-11.4.1-3.el9.x86_64.rpm 7.8 MB/s | 13 MB 00:01 2024-03-12T17:08:27.677 INFO:teuthology.orchestra.run.smithi052.stdout:(36/36): gcc-11.4.1-3.el9.x86_64.rpm 10 MB/s | 32 MB 00:03 2024-03-12T17:08:27.679 INFO:teuthology.orchestra.run.smithi052.stdout:-------------------------------------------------------------------------------- 2024-03-12T17:08:27.680 INFO:teuthology.orchestra.run.smithi052.stdout:Total 14 MB/s | 80 MB 00:05 2024-03-12T17:08:28.979 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction check 2024-03-12T17:08:29.339 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction check succeeded. 2024-03-12T17:08:29.339 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction test 2024-03-12T17:08:30.236 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction test succeeded. 2024-03-12T17:08:30.236 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction 2024-03-12T17:08:32.774 INFO:teuthology.orchestra.run.smithi052.stdout: Preparing : 1/1 2024-03-12T17:08:32.817 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : libgcc-11.4.1-3.el9.x86_64 1/60 2024-03-12T17:08:32.900 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libgcc-11.4.1-3.el9.x86_64 1/60 2024-03-12T17:08:32.997 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : libcom_err-1.46.5-5.el9.x86_64 2/60 2024-03-12T17:08:33.101 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : libstdc++-11.4.1-3.el9.x86_64 3/60 2024-03-12T17:08:33.174 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : e2fsprogs-libs-1.46.5-5.el9.x86_64 4/60 2024-03-12T17:08:33.273 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : device-mapper-libs-9:1.02.197-2.el9.x86_64 5/60 2024-03-12T17:08:33.323 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : device-mapper-9:1.02.197-2.el9.x86_64 6/60 2024-03-12T17:08:33.401 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : device-mapper-event-libs-9:1.02.197-2.el9.x86_64 7/60 2024-03-12T17:08:35.061 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : sqlite-libs-3.34.1-7.el9.x86_64 8/60 2024-03-12T17:08:35.149 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : python3-libs-3.9.18-3.el9.x86_64 9/60 2024-03-12T17:08:35.185 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : python3-3.9.18-3.el9.x86_64 10/60 2024-03-12T17:08:35.214 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : python-unversioned-command-3.9.18-3.el9.noarch 11/60 2024-03-12T17:08:35.251 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : device-mapper-event-9:1.02.197-2.el9.x86_64 12/60 2024-03-12T17:08:35.637 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: device-mapper-event-9:1.02.197-2.el9.x86_64 12/60 2024-03-12T17:08:36.395 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : lvm2-libs-9:2.03.23-2.el9.x86_64 13/60 2024-03-12T17:08:36.484 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : libstdc++-devel-11.4.1-3.el9.x86_64 14/60 2024-03-12T17:08:36.863 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : libss-1.46.5-5.el9.x86_64 15/60 2024-03-12T17:08:37.478 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : cpp-11.4.1-3.el9.x86_64 16/60 2024-03-12T17:08:37.529 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : make-1:4.3-8.el9.x86_64 17/60 2024-03-12T17:08:38.746 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : libgomp-11.4.1-3.el9.x86_64 18/60 2024-03-12T17:08:38.998 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : gcc-11.4.1-3.el9.x86_64 19/60 2024-03-12T17:08:39.097 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: gcc-11.4.1-3.el9.x86_64 19/60 2024-03-12T17:08:39.134 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libnbd-1.18.1-3.el9.x86_64 20/60 2024-03-12T17:08:39.168 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : fio-engine-http-3.35-1.el9.x86_64 21/60 2024-03-12T17:08:39.202 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : fio-engine-libaio-3.35-1.el9.x86_64 22/60 2024-03-12T17:08:39.236 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : fio-engine-nbd-3.35-1.el9.x86_64 23/60 2024-03-12T17:08:39.270 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : fio-engine-rados-3.35-1.el9.x86_64 24/60 2024-03-12T17:08:39.303 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : fio-engine-rbd-3.35-1.el9.x86_64 25/60 2024-03-12T17:08:39.570 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : fio-engine-rdma-3.35-1.el9.x86_64 26/60 2024-03-12T17:08:39.625 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : fio-3.35-1.el9.x86_64 27/60 2024-03-12T17:08:39.666 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : rmt-2:1.6-6.el9.x86_64 28/60 2024-03-12T17:08:40.112 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : dump-1:0.4-0.51.b47.el9.x86_64 29/60 2024-03-12T17:08:40.204 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : gcc-c++-11.4.1-3.el9.x86_64 30/60 2024-03-12T17:08:40.238 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : gcc-plugin-annobin-11.4.1-3.el9.x86_64 31/60 2024-03-12T17:08:40.435 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: gcc-plugin-annobin-11.4.1-3.el9.x86_64 31/60 2024-03-12T17:08:40.676 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : e2fsprogs-1.46.5-5.el9.x86_64 32/60 2024-03-12T17:08:40.677 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : lvm2-9:2.03.23-2.el9.x86_64 33/60 2024-03-12T17:08:40.677 INFO:teuthology.orchestra.run.smithi052.stdout:warning: /etc/lvm/lvm.conf created as /etc/lvm/lvm.conf.rpmnew 2024-03-12T17:08:40.677 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:08:41.171 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: lvm2-9:2.03.23-2.el9.x86_64 33/60 2024-03-12T17:08:41.233 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : python3-devel-3.9.18-3.el9.x86_64 34/60 2024-03-12T17:08:41.304 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : sqlite-3.34.1-7.el9.x86_64 35/60 2024-03-12T17:08:41.345 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : indent-2.2.13-5.el9.x86_64 36/60 2024-03-12T17:08:41.396 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : python3-devel-3.9.17-2.el9.x86_64 37/60 2024-03-12T17:08:41.436 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : e2fsprogs-1.46.5-3.el9.x86_64 38/60 2024-03-12T17:08:41.475 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : gcc-c++-11.4.1-2.1.el9.x86_64 39/60 2024-03-12T17:08:41.507 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : python3-3.9.17-2.el9.x86_64 40/60 2024-03-12T17:08:41.534 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : python-unversioned-command-3.9.17-2.el9.noarch 41/60 2024-03-12T17:08:41.534 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : gcc-plugin-annobin-11.4.1-2.1.el9.x86_64 42/60 2024-03-12T17:08:41.671 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: gcc-plugin-annobin-11.4.1-2.1.el9.x86_64 42/60 2024-03-12T17:08:41.797 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : gcc-11.4.1-2.1.el9.x86_64 43/60 2024-03-12T17:08:41.797 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: lvm2-9:2.03.21-3.el9.x86_64 44/60 2024-03-12T17:08:41.798 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : lvm2-9:2.03.21-3.el9.x86_64 44/60 2024-03-12T17:08:41.866 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: lvm2-9:2.03.21-3.el9.x86_64 44/60 2024-03-12T17:08:41.902 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : lvm2-libs-9:2.03.21-3.el9.x86_64 45/60 2024-03-12T17:08:41.930 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: device-mapper-event-9:1.02.195-3.el9.x86_64 46/60 2024-03-12T17:08:41.972 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : device-mapper-event-9:1.02.195-3.el9.x86_64 46/60 2024-03-12T17:08:42.003 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : device-mapper-event-libs-9:1.02.195-3.el9.x86_64 47/60 2024-03-12T17:08:42.037 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : e2fsprogs-libs-1.46.5-3.el9.x86_64 48/60 2024-03-12T17:08:42.115 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : libss-1.46.5-3.el9.x86_64 49/60 2024-03-12T17:08:42.181 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : libstdc++-devel-11.4.1-2.1.el9.x86_64 50/60 2024-03-12T17:08:42.214 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : libstdc++-11.4.1-2.1.el9.x86_64 51/60 2024-03-12T17:08:42.248 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : device-mapper-libs-9:1.02.195-3.el9.x86_64 52/60 2024-03-12T17:08:42.381 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : device-mapper-9:1.02.195-3.el9.x86_64 53/60 2024-03-12T17:08:42.450 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : python3-libs-3.9.17-2.el9.x86_64 54/60 2024-03-12T17:08:42.483 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : sqlite-libs-3.34.1-6.el9.x86_64 55/60 2024-03-12T17:08:42.484 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : libgcc-11.4.1-2.1.el9.x86_64 56/60 2024-03-12T17:08:42.611 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libgcc-11.4.1-2.1.el9.x86_64 56/60 2024-03-12T17:08:42.670 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : libcom_err-1.46.5-3.el9.x86_64 57/60 2024-03-12T17:08:42.711 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : cpp-11.4.1-2.1.el9.x86_64 58/60 2024-03-12T17:08:42.746 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : libgomp-11.4.1-2.1.el9.x86_64 59/60 2024-03-12T17:08:42.870 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : make-1:4.3-7.el9.x86_64 60/60 2024-03-12T17:08:45.086 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: make-1:4.3-7.el9.x86_64 60/60 2024-03-12T17:08:45.087 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : fio-3.35-1.el9.x86_64 1/60 2024-03-12T17:08:45.087 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : fio-engine-http-3.35-1.el9.x86_64 2/60 2024-03-12T17:08:45.087 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : fio-engine-libaio-3.35-1.el9.x86_64 3/60 2024-03-12T17:08:45.087 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : fio-engine-nbd-3.35-1.el9.x86_64 4/60 2024-03-12T17:08:45.087 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : fio-engine-rados-3.35-1.el9.x86_64 5/60 2024-03-12T17:08:45.087 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : fio-engine-rbd-3.35-1.el9.x86_64 6/60 2024-03-12T17:08:45.087 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : fio-engine-rdma-3.35-1.el9.x86_64 7/60 2024-03-12T17:08:45.087 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : sqlite-3.34.1-7.el9.x86_64 8/60 2024-03-12T17:08:45.087 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : rmt-2:1.6-6.el9.x86_64 9/60 2024-03-12T17:08:45.087 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libnbd-1.18.1-3.el9.x86_64 10/60 2024-03-12T17:08:45.087 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : dump-1:0.4-0.51.b47.el9.x86_64 11/60 2024-03-12T17:08:45.087 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : indent-2.2.13-5.el9.x86_64 12/60 2024-03-12T17:08:45.087 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python-unversioned-command-3.9.18-3.el9.noarch 13/60 2024-03-12T17:08:45.087 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python-unversioned-command-3.9.17-2.el9.noarch 14/60 2024-03-12T17:08:45.088 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 15/60 2024-03-12T17:08:45.088 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 16/60 2024-03-12T17:08:45.088 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-3.9.18-3.el9.x86_64 17/60 2024-03-12T17:08:45.088 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-3.9.17-2.el9.x86_64 18/60 2024-03-12T17:08:45.088 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-libs-3.9.18-3.el9.x86_64 19/60 2024-03-12T17:08:45.088 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-libs-3.9.17-2.el9.x86_64 20/60 2024-03-12T17:08:45.088 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : sqlite-libs-3.34.1-7.el9.x86_64 21/60 2024-03-12T17:08:45.088 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : sqlite-libs-3.34.1-6.el9.x86_64 22/60 2024-03-12T17:08:45.088 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : device-mapper-9:1.02.197-2.el9.x86_64 23/60 2024-03-12T17:08:45.088 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : device-mapper-9:1.02.195-3.el9.x86_64 24/60 2024-03-12T17:08:45.088 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : device-mapper-event-9:1.02.197-2.el9.x86_64 25/60 2024-03-12T17:08:45.088 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : device-mapper-event-9:1.02.195-3.el9.x86_64 26/60 2024-03-12T17:08:45.088 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : device-mapper-event-libs-9:1.02.197-2.el9.x86_64 27/60 2024-03-12T17:08:45.092 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : device-mapper-event-libs-9:1.02.195-3.el9.x86_64 28/60 2024-03-12T17:08:45.092 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : device-mapper-libs-9:1.02.197-2.el9.x86_64 29/60 2024-03-12T17:08:45.092 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : device-mapper-libs-9:1.02.195-3.el9.x86_64 30/60 2024-03-12T17:08:45.092 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : e2fsprogs-1.46.5-5.el9.x86_64 31/60 2024-03-12T17:08:45.092 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : e2fsprogs-1.46.5-3.el9.x86_64 32/60 2024-03-12T17:08:45.092 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : e2fsprogs-libs-1.46.5-5.el9.x86_64 33/60 2024-03-12T17:08:45.092 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : e2fsprogs-libs-1.46.5-3.el9.x86_64 34/60 2024-03-12T17:08:45.092 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libcom_err-1.46.5-5.el9.x86_64 35/60 2024-03-12T17:08:45.092 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libcom_err-1.46.5-3.el9.x86_64 36/60 2024-03-12T17:08:45.092 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libgcc-11.4.1-3.el9.x86_64 37/60 2024-03-12T17:08:45.093 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libgcc-11.4.1-2.1.el9.x86_64 38/60 2024-03-12T17:08:45.093 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libgomp-11.4.1-3.el9.x86_64 39/60 2024-03-12T17:08:45.093 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libgomp-11.4.1-2.1.el9.x86_64 40/60 2024-03-12T17:08:45.093 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libss-1.46.5-5.el9.x86_64 41/60 2024-03-12T17:08:45.093 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libss-1.46.5-3.el9.x86_64 42/60 2024-03-12T17:08:45.093 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libstdc++-11.4.1-3.el9.x86_64 43/60 2024-03-12T17:08:45.093 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libstdc++-11.4.1-2.1.el9.x86_64 44/60 2024-03-12T17:08:45.093 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : lvm2-9:2.03.23-2.el9.x86_64 45/60 2024-03-12T17:08:45.093 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : lvm2-9:2.03.21-3.el9.x86_64 46/60 2024-03-12T17:08:45.093 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : lvm2-libs-9:2.03.23-2.el9.x86_64 47/60 2024-03-12T17:08:45.093 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : lvm2-libs-9:2.03.21-3.el9.x86_64 48/60 2024-03-12T17:08:45.093 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : make-1:4.3-8.el9.x86_64 49/60 2024-03-12T17:08:45.093 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : make-1:4.3-7.el9.x86_64 50/60 2024-03-12T17:08:45.093 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cpp-11.4.1-3.el9.x86_64 51/60 2024-03-12T17:08:45.093 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cpp-11.4.1-2.1.el9.x86_64 52/60 2024-03-12T17:08:45.094 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : gcc-11.4.1-3.el9.x86_64 53/60 2024-03-12T17:08:45.094 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : gcc-11.4.1-2.1.el9.x86_64 54/60 2024-03-12T17:08:45.094 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : gcc-c++-11.4.1-3.el9.x86_64 55/60 2024-03-12T17:08:45.094 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : gcc-c++-11.4.1-2.1.el9.x86_64 56/60 2024-03-12T17:08:45.094 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : gcc-plugin-annobin-11.4.1-3.el9.x86_64 57/60 2024-03-12T17:08:45.094 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : gcc-plugin-annobin-11.4.1-2.1.el9.x86_64 58/60 2024-03-12T17:08:45.094 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libstdc++-devel-11.4.1-3.el9.x86_64 59/60 2024-03-12T17:08:46.169 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libstdc++-devel-11.4.1-2.1.el9.x86_64 60/60 2024-03-12T17:08:46.170 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:08:46.170 INFO:teuthology.orchestra.run.smithi052.stdout:Upgraded: 2024-03-12T17:08:46.170 INFO:teuthology.orchestra.run.smithi052.stdout: cpp-11.4.1-3.el9.x86_64 2024-03-12T17:08:46.170 INFO:teuthology.orchestra.run.smithi052.stdout: device-mapper-9:1.02.197-2.el9.x86_64 2024-03-12T17:08:46.170 INFO:teuthology.orchestra.run.smithi052.stdout: device-mapper-event-9:1.02.197-2.el9.x86_64 2024-03-12T17:08:46.170 INFO:teuthology.orchestra.run.smithi052.stdout: device-mapper-event-libs-9:1.02.197-2.el9.x86_64 2024-03-12T17:08:46.170 INFO:teuthology.orchestra.run.smithi052.stdout: device-mapper-libs-9:1.02.197-2.el9.x86_64 2024-03-12T17:08:46.170 INFO:teuthology.orchestra.run.smithi052.stdout: e2fsprogs-1.46.5-5.el9.x86_64 2024-03-12T17:08:46.170 INFO:teuthology.orchestra.run.smithi052.stdout: e2fsprogs-libs-1.46.5-5.el9.x86_64 2024-03-12T17:08:46.170 INFO:teuthology.orchestra.run.smithi052.stdout: gcc-11.4.1-3.el9.x86_64 2024-03-12T17:08:46.170 INFO:teuthology.orchestra.run.smithi052.stdout: gcc-c++-11.4.1-3.el9.x86_64 2024-03-12T17:08:46.171 INFO:teuthology.orchestra.run.smithi052.stdout: gcc-plugin-annobin-11.4.1-3.el9.x86_64 2024-03-12T17:08:46.171 INFO:teuthology.orchestra.run.smithi052.stdout: libcom_err-1.46.5-5.el9.x86_64 2024-03-12T17:08:46.171 INFO:teuthology.orchestra.run.smithi052.stdout: libgcc-11.4.1-3.el9.x86_64 2024-03-12T17:08:46.171 INFO:teuthology.orchestra.run.smithi052.stdout: libgomp-11.4.1-3.el9.x86_64 2024-03-12T17:08:46.171 INFO:teuthology.orchestra.run.smithi052.stdout: libss-1.46.5-5.el9.x86_64 2024-03-12T17:08:46.171 INFO:teuthology.orchestra.run.smithi052.stdout: libstdc++-11.4.1-3.el9.x86_64 2024-03-12T17:08:46.171 INFO:teuthology.orchestra.run.smithi052.stdout: libstdc++-devel-11.4.1-3.el9.x86_64 2024-03-12T17:08:46.171 INFO:teuthology.orchestra.run.smithi052.stdout: lvm2-9:2.03.23-2.el9.x86_64 2024-03-12T17:08:46.171 INFO:teuthology.orchestra.run.smithi052.stdout: lvm2-libs-9:2.03.23-2.el9.x86_64 2024-03-12T17:08:46.171 INFO:teuthology.orchestra.run.smithi052.stdout: make-1:4.3-8.el9.x86_64 2024-03-12T17:08:46.171 INFO:teuthology.orchestra.run.smithi052.stdout: python-unversioned-command-3.9.18-3.el9.noarch 2024-03-12T17:08:46.171 INFO:teuthology.orchestra.run.smithi052.stdout: python3-3.9.18-3.el9.x86_64 2024-03-12T17:08:46.171 INFO:teuthology.orchestra.run.smithi052.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-03-12T17:08:46.172 INFO:teuthology.orchestra.run.smithi052.stdout: python3-libs-3.9.18-3.el9.x86_64 2024-03-12T17:08:46.172 INFO:teuthology.orchestra.run.smithi052.stdout: sqlite-libs-3.34.1-7.el9.x86_64 2024-03-12T17:08:46.172 INFO:teuthology.orchestra.run.smithi052.stdout:Installed: 2024-03-12T17:08:46.172 INFO:teuthology.orchestra.run.smithi052.stdout: dump-1:0.4-0.51.b47.el9.x86_64 fio-3.35-1.el9.x86_64 2024-03-12T17:08:46.172 INFO:teuthology.orchestra.run.smithi052.stdout: fio-engine-http-3.35-1.el9.x86_64 fio-engine-libaio-3.35-1.el9.x86_64 2024-03-12T17:08:46.172 INFO:teuthology.orchestra.run.smithi052.stdout: fio-engine-nbd-3.35-1.el9.x86_64 fio-engine-rados-3.35-1.el9.x86_64 2024-03-12T17:08:46.172 INFO:teuthology.orchestra.run.smithi052.stdout: fio-engine-rbd-3.35-1.el9.x86_64 fio-engine-rdma-3.35-1.el9.x86_64 2024-03-12T17:08:46.172 INFO:teuthology.orchestra.run.smithi052.stdout: indent-2.2.13-5.el9.x86_64 libnbd-1.18.1-3.el9.x86_64 2024-03-12T17:08:46.172 INFO:teuthology.orchestra.run.smithi052.stdout: rmt-2:1.6-6.el9.x86_64 sqlite-3.34.1-7.el9.x86_64 2024-03-12T17:08:46.172 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:08:46.172 INFO:teuthology.orchestra.run.smithi052.stdout:Complete! 2024-03-12T17:08:46.839 DEBUG:teuthology.orchestra.run.smithi052:> sudo useradd -m fsgqa 2024-03-12T17:08:46.987 INFO:teuthology.orchestra.run.smithi052.stderr:useradd: user 'fsgqa' already exists 2024-03-12T17:08:46.992 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-03-12T17:08:46.993 DEBUG:teuthology.orchestra.run.smithi052:> sudo groupadd fsgqa 2024-03-12T17:08:47.081 INFO:teuthology.orchestra.run.smithi052.stderr:groupadd: group 'fsgqa' already exists 2024-03-12T17:08:47.084 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-03-12T17:08:47.085 DEBUG:teuthology.orchestra.run.smithi052:> sudo useradd fsgqa2 2024-03-12T17:08:47.960 DEBUG:teuthology.orchestra.run.smithi052:> sudo useradd 123456-fsgqa 2024-03-12T17:08:48.844 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json mon dump 2024-03-12T17:08:49.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.080+0000 7fb899600640 1 Processor -- start 2024-03-12T17:08:49.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.081+0000 7fb899600640 1 -- start start 2024-03-12T17:08:49.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.082+0000 7fb899600640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8941571d0 0x7fb8940b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:49.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.082+0000 7fb899600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb8940cae50 0x7fb8940cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:49.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.082+0000 7fb899600640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8941575b0 con 0x7fb8940cdc00 2024-03-12T17:08:49.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.082+0000 7fb899600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8940cb820 con 0x7fb8940cae50 2024-03-12T17:08:49.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.082+0000 7fb899600640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb894159f20 con 0x7fb8941571d0 2024-03-12T17:08:49.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.083+0000 7fb893400640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fb8940cdc00 0x7fb89415aaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:43072/0 (socket says 172.21.15.52:43072) 2024-03-12T17:08:49.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.083+0000 7fb893400640 1 -- 172.21.15.52:0/59840451 learned_addr learned my addr 172.21.15.52:0/59840451 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:08:49.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.083+0000 7fb898c00640 1 --2- 172.21.15.52:0/59840451 >> v2:172.21.15.102:3300/0 conn(0x7fb8940cae50 0x7fb8940cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:49.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.083+0000 7fb893e00640 1 --2- 172.21.15.52:0/59840451 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8941571d0 0x7fb8940b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:49.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.083+0000 7fb892a00640 1 -- 172.21.15.52:0/59840451 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3459436405 0 0) 0x7fb8941575b0 con 0x7fb8940cdc00 2024-03-12T17:08:49.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.083+0000 7fb892a00640 1 -- 172.21.15.52:0/59840451 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb878003520 con 0x7fb8940cdc00 2024-03-12T17:08:49.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.084+0000 7fb892a00640 1 -- 172.21.15.52:0/59840451 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 49185089 0 0) 0x7fb878003520 con 0x7fb8940cdc00 2024-03-12T17:08:49.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.084+0000 7fb892a00640 1 -- 172.21.15.52:0/59840451 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8941571d0 msgr2=0x7fb8940b35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:49.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.084+0000 7fb892a00640 1 --2- 172.21.15.52:0/59840451 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8941571d0 0x7fb8940b35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:49.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.084+0000 7fb892a00640 1 -- 172.21.15.52:0/59840451 >> v2:172.21.15.102:3300/0 conn(0x7fb8940cae50 msgr2=0x7fb8940cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:49.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.084+0000 7fb892a00640 1 --2- 172.21.15.52:0/59840451 >> v2:172.21.15.102:3300/0 conn(0x7fb8940cae50 0x7fb8940cb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:49.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.084+0000 7fb892a00640 1 -- 172.21.15.52:0/59840451 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb89415a200 con 0x7fb8940cdc00 2024-03-12T17:08:49.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.084+0000 7fb892a00640 1 -- 172.21.15.52:0/59840451 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7fb884001670 con 0x7fb8940cdc00 2024-03-12T17:08:49.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.084+0000 7fb892a00640 1 -- 172.21.15.52:0/59840451 >> v1:172.21.15.52:6789/0 conn(0x7fb8940cdc00 legacy=0x7fb89415aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:49.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.084+0000 7fb892a00640 1 --2- 172.21.15.52:0/59840451 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb878003970 0x7fb878003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:49.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.084+0000 7fb892a00640 1 --2- 172.21.15.52:0/59840451 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8941571d0 0x7fb8780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:49.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.084+0000 7fb892a00640 1 --2- 172.21.15.52:0/59840451 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb878004d00 0x7fb878005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:49.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.084+0000 7fb892a00640 1 -- 172.21.15.52:0/59840451 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8940cb820 con 0x7fb878003970 2024-03-12T17:08:49.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.084+0000 7fb892a00640 1 -- 172.21.15.52:0/59840451 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb894159f20 con 0x7fb878004d00 2024-03-12T17:08:49.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.084+0000 7fb892a00640 1 -- 172.21.15.52:0/59840451 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87800b980 con 0x7fb8941571d0 2024-03-12T17:08:49.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.084+0000 7fb893e00640 1 --2- 172.21.15.52:0/59840451 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb878003970 0x7fb878003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:49.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.085+0000 7fb898c00640 1 --2- 172.21.15.52:0/59840451 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb878004d00 0x7fb878005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:49.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.085+0000 7fb893400640 1 --2- 172.21.15.52:0/59840451 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8941571d0 0x7fb8780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:49.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.085+0000 7fb893e00640 1 -- 172.21.15.52:0/59840451 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8941571d0 msgr2=0x7fb8780047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:49.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.085+0000 7fb893e00640 1 --2- 172.21.15.52:0/59840451 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8941571d0 0x7fb8780047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:49.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.085+0000 7fb893e00640 1 -- 172.21.15.52:0/59840451 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb878004d00 msgr2=0x7fb878005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:49.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.085+0000 7fb893e00640 1 --2- 172.21.15.52:0/59840451 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb878004d00 0x7fb878005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:49.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.085+0000 7fb893e00640 1 -- 172.21.15.52:0/59840451 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb878003520 con 0x7fb878003970 2024-03-12T17:08:49.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.085+0000 7fb893400640 1 --2- 172.21.15.52:0/59840451 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8941571d0 0x7fb8780047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:08:49.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.086+0000 7fb893e00640 1 --2- 172.21.15.52:0/59840451 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb878003970 0x7fb878003d90 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb87c009ac0 tx=0x7fb8780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e71eef82238480c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:49.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.086+0000 7fb892a00640 1 -- 172.21.15.52:0/59840451 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87c02c6b0 con 0x7fb878003970 2024-03-12T17:08:49.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.086+0000 7fb892a00640 1 -- 172.21.15.52:0/59840451 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb87c02c810 con 0x7fb878003970 2024-03-12T17:08:49.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.086+0000 7fb892a00640 1 -- 172.21.15.52:0/59840451 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87c02cba0 con 0x7fb878003970 2024-03-12T17:08:49.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.087+0000 7fb899600640 1 -- 172.21.15.52:0/59840451 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb878003970 msgr2=0x7fb878003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:49.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.087+0000 7fb899600640 1 --2- 172.21.15.52:0/59840451 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb878003970 0x7fb878003d90 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb87c009ac0 tx=0x7fb8780057f0 comp rx=0 tx=0).stop 2024-03-12T17:08:49.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.087+0000 7fb898c00640 1 -- 172.21.15.52:0/59840451 reap_dead start 2024-03-12T17:08:49.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.087+0000 7fb899600640 1 -- 172.21.15.52:0/59840451 shutdown_connections 2024-03-12T17:08:49.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.087+0000 7fb899600640 1 -- 172.21.15.52:0/59840451 >> 172.21.15.52:0/59840451 conn(0x7fb894065df0 msgr2=0x7fb894069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:49.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.087+0000 7fb899600640 1 -- 172.21.15.52:0/59840451 shutdown_connections 2024-03-12T17:08:49.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.087+0000 7fb899600640 1 -- 172.21.15.52:0/59840451 wait complete. 2024-03-12T17:08:49.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.089+0000 7fb899600640 1 Processor -- start 2024-03-12T17:08:49.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.089+0000 7fb899600640 1 -- start start 2024-03-12T17:08:49.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.090+0000 7fb899600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb878003970 0x7fb8940aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:49.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.090+0000 7fb899600640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb894158090 0x7fb89414d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:49.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.090+0000 7fb899600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb894158510 0x7fb89414d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:49.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.090+0000 7fb899600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89414b990 con 0x7fb878003970 2024-03-12T17:08:49.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.090+0000 7fb899600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8940af640 con 0x7fb894158510 2024-03-12T17:08:49.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.090+0000 7fb899600640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8940af920 con 0x7fb894158090 2024-03-12T17:08:49.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.090+0000 7fb893e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb878003970 0x7fb8940aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:49.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.090+0000 7fb893e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb878003970 0x7fb8940aefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59374/0 (socket says 172.21.15.52:59374) 2024-03-12T17:08:49.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.090+0000 7fb893e00640 1 -- 172.21.15.52:0/3091982935 learned_addr learned my addr 172.21.15.52:0/3091982935 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:49.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.091+0000 7fb898c00640 1 --2- 172.21.15.52:0/3091982935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb894158510 0x7fb89414d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:49.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.091+0000 7fb893400640 1 --2- 172.21.15.52:0/3091982935 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb894158090 0x7fb89414d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:49.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.091+0000 7fb893e00640 1 -- 172.21.15.52:0/3091982935 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb894158090 msgr2=0x7fb89414d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:49.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.091+0000 7fb893e00640 1 --2- 172.21.15.52:0/3091982935 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb894158090 0x7fb89414d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:49.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.091+0000 7fb893e00640 1 -- 172.21.15.52:0/3091982935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb894158510 msgr2=0x7fb89414d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:49.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.091+0000 7fb893e00640 1 --2- 172.21.15.52:0/3091982935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb894158510 0x7fb89414d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:49.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.091+0000 7fb893e00640 1 -- 172.21.15.52:0/3091982935 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8940afc00 con 0x7fb878003970 2024-03-12T17:08:49.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.091+0000 7fb898c00640 1 --2- 172.21.15.52:0/3091982935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb894158510 0x7fb89414d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:08:49.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.091+0000 7fb893e00640 1 --2- 172.21.15.52:0/3091982935 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb878003970 0x7fb8940aefd0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb87c009ac0 tx=0x7fb87c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:49.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.091+0000 7fb88be00640 1 -- 172.21.15.52:0/3091982935 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87c003810 con 0x7fb878003970 2024-03-12T17:08:49.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.091+0000 7fb88be00640 1 -- 172.21.15.52:0/3091982935 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb87c003970 con 0x7fb878003970 2024-03-12T17:08:49.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.091+0000 7fb88be00640 1 -- 172.21.15.52:0/3091982935 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87c003d00 con 0x7fb878003970 2024-03-12T17:08:49.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.091+0000 7fb899600640 1 -- 172.21.15.52:0/3091982935 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8940afee0 con 0x7fb878003970 2024-03-12T17:08:49.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.092+0000 7fb899600640 1 -- 172.21.15.52:0/3091982935 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb89413a1a0 con 0x7fb878003970 2024-03-12T17:08:49.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.093+0000 7fb899600640 1 -- 172.21.15.52:0/3091982935 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8940b1e20 con 0x7fb878003970 2024-03-12T17:08:49.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.094+0000 7fb88be00640 1 -- 172.21.15.52:0/3091982935 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb87c040450 con 0x7fb878003970 2024-03-12T17:08:49.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.094+0000 7fb88be00640 1 --2- 172.21.15.52:0/3091982935 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb87006ac00 0x7fb87006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:49.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.095+0000 7fb88be00640 1 -- 172.21.15.52:0/3091982935 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb87c0ab450 con 0x7fb878003970 2024-03-12T17:08:49.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.095+0000 7fb893400640 1 --2- 172.21.15.52:0/3091982935 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb87006ac00 0x7fb87006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:49.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.096+0000 7fb893400640 1 --2- 172.21.15.52:0/3091982935 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb87006ac00 0x7fb87006d0c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb884000ff0 tx=0x7fb884009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:49.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.100+0000 7fb88be00640 1 -- 172.21.15.52:0/3091982935 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fb87c0764e0 con 0x7fb878003970 2024-03-12T17:08:49.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.372+0000 7fb899600640 1 -- 172.21.15.52:0/3091982935 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb89404e030 con 0x7fb878003970 2024-03-12T17:08:49.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.373+0000 7fb88be00640 1 -- 172.21.15.52:0/3091982935 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+1276 (secure 0 0 0) 0x7fb87c0044e0 con 0x7fb878003970 2024-03-12T17:08:49.367 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:08:49.367 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":1,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","modified":"2024-03-12T17:05:24.868997Z","created":"2024-03-12T17:05:24.868997Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:3300","nonce":0},{"type":"v1","addr":"172.21.15.52:6789","nonce":0}]},"addr":"172.21.15.52:6789/0","public_addr":"172.21.15.52:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3301","nonce":0},{"type":"v1","addr":"172.21.15.102:6790","nonce":0}]},"addr":"172.21.15.102:6790/0","public_addr":"172.21.15.102:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-03-12T17:08:49.368 INFO:teuthology.orchestra.run.smithi052.stderr:dumped monmap epoch 1 2024-03-12T17:08:49.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.376+0000 7fb899600640 1 -- 172.21.15.52:0/3091982935 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb87006ac00 msgr2=0x7fb87006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:49.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.376+0000 7fb899600640 1 --2- 172.21.15.52:0/3091982935 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb87006ac00 0x7fb87006d0c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb884000ff0 tx=0x7fb884009040 comp rx=0 tx=0).stop 2024-03-12T17:08:49.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.376+0000 7fb899600640 1 -- 172.21.15.52:0/3091982935 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb878003970 msgr2=0x7fb8940aefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:49.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.376+0000 7fb899600640 1 --2- 172.21.15.52:0/3091982935 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb878003970 0x7fb8940aefd0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb87c009ac0 tx=0x7fb87c004730 comp rx=0 tx=0).stop 2024-03-12T17:08:49.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.376+0000 7fb899600640 1 -- 172.21.15.52:0/3091982935 shutdown_connections 2024-03-12T17:08:49.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.376+0000 7fb899600640 1 --2- 172.21.15.52:0/3091982935 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb87006ac00 0x7fb87006d0c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:49.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.376+0000 7fb899600640 1 --2- 172.21.15.52:0/3091982935 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb894158090 0x7fb89414d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:49.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.376+0000 7fb899600640 1 --2- 172.21.15.52:0/3091982935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb894158510 0x7fb89414d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:49.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.376+0000 7fb899600640 1 --2- 172.21.15.52:0/3091982935 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb878003970 0x7fb8940aefd0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:49.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.376+0000 7fb899600640 1 -- 172.21.15.52:0/3091982935 >> 172.21.15.52:0/3091982935 conn(0x7fb894065df0 msgr2=0x7fb89415aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:49.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.376+0000 7fb899600640 1 -- 172.21.15.52:0/3091982935 shutdown_connections 2024-03-12T17:08:49.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.376+0000 7fb899600640 1 -- 172.21.15.52:0/3091982935 wait complete. 2024-03-12T17:08:49.390 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get-or-create client.admin 2024-03-12T17:08:49.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.618+0000 7f2053200640 1 Processor -- start 2024-03-12T17:08:49.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.618+0000 7f2053200640 1 -- start start 2024-03-12T17:08:49.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.620+0000 7f2053200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f20540cb480 0x7f20540cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:49.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.620+0000 7f2053200640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20540cd1b0 con 0x7f2054160ee0 2024-03-12T17:08:49.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.620+0000 7f2053200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20540cd490 con 0x7f20540cca00 2024-03-12T17:08:49.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.620+0000 7f2053200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20540b1ca0 con 0x7f20540cb480 2024-03-12T17:08:49.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.620+0000 7f2051e00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f2054160ee0 0x7f20540cad70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:43086/0 (socket says 172.21.15.52:43086) 2024-03-12T17:08:49.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.620+0000 7f2051e00640 1 -- 172.21.15.52:0/4043533728 learned_addr learned my addr 172.21.15.52:0/4043533728 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:08:49.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.621+0000 7f2051400640 1 -- 172.21.15.52:0/4043533728 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 559189485 0 0) 0x7f20540cd1b0 con 0x7f2054160ee0 2024-03-12T17:08:49.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.621+0000 7f2051400640 1 -- 172.21.15.52:0/4043533728 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2038003520 con 0x7f2054160ee0 2024-03-12T17:08:49.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.621+0000 7f204be00640 1 --2- 172.21.15.52:0/4043533728 >> v2:172.21.15.102:3300/0 conn(0x7f20540cb480 0x7f20540cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:49.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.621+0000 7f2051400640 1 -- 172.21.15.52:0/4043533728 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1047258383 0 0) 0x7f2038003520 con 0x7f2054160ee0 2024-03-12T17:08:49.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.621+0000 7f2051400640 1 -- 172.21.15.52:0/4043533728 >> v2:172.21.15.102:3300/0 conn(0x7f20540cb480 msgr2=0x7f20540cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:49.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.621+0000 7f2051400640 1 --2- 172.21.15.52:0/4043533728 >> v2:172.21.15.102:3300/0 conn(0x7f20540cb480 0x7f20540cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:49.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.621+0000 7f2051400640 1 -- 172.21.15.52:0/4043533728 >> v1:172.21.15.102:6789/0 conn(0x7f20540cca00 legacy=0x7f20540b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:49.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.622+0000 7f2051400640 1 -- 172.21.15.52:0/4043533728 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20540b1f80 con 0x7f2054160ee0 2024-03-12T17:08:49.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.622+0000 7f2051400640 1 -- 172.21.15.52:0/4043533728 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f2044001670 con 0x7f2054160ee0 2024-03-12T17:08:49.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.622+0000 7f2051400640 1 -- 172.21.15.52:0/4043533728 >> v1:172.21.15.52:6789/0 conn(0x7f2054160ee0 legacy=0x7f20540cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:49.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.622+0000 7f2051400640 1 --2- 172.21.15.52:0/4043533728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2038003970 0x7f2038003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:49.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.622+0000 7f2051400640 1 --2- 172.21.15.52:0/4043533728 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f20380047c0 0x7f2038004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:49.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.622+0000 7f2051400640 1 --2- 172.21.15.52:0/4043533728 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2038005180 0x7f203800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:49.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.622+0000 7f2051400640 1 -- 172.21.15.52:0/4043533728 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20540b1ca0 con 0x7f2038005180 2024-03-12T17:08:49.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.622+0000 7f2051400640 1 -- 172.21.15.52:0/4043533728 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203800c020 con 0x7f2038003970 2024-03-12T17:08:49.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.622+0000 7f2051400640 1 -- 172.21.15.52:0/4043533728 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203800c1e0 con 0x7f20380047c0 2024-03-12T17:08:49.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.622+0000 7f2052800640 1 --2- 172.21.15.52:0/4043533728 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2038005180 0x7f203800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:49.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.622+0000 7f2051e00640 1 --2- 172.21.15.52:0/4043533728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2038003970 0x7f2038003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:49.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.622+0000 7f2052800640 1 -- 172.21.15.52:0/4043533728 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f20380047c0 msgr2=0x7f2038004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:49.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.622+0000 7f2052800640 1 --2- 172.21.15.52:0/4043533728 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f20380047c0 0x7f2038004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:49.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.622+0000 7f2052800640 1 -- 172.21.15.52:0/4043533728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2038003970 msgr2=0x7f2038003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:08:49.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.622+0000 7f2052800640 1 --2- 172.21.15.52:0/4043533728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2038003970 0x7f2038003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:49.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.623+0000 7f2052800640 1 -- 172.21.15.52:0/4043533728 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2038003520 con 0x7f2038005180 2024-03-12T17:08:49.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.623+0000 7f2052800640 1 -- 172.21.15.52:0/4043533728 reap_dead start 2024-03-12T17:08:49.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.624+0000 7f2052800640 1 --2- 172.21.15.52:0/4043533728 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2038005180 0x7f203800b980 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f204c001e50 tx=0x7f20380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d39e25251a5456 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:49.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.624+0000 7f2051400640 1 -- 172.21.15.52:0/4043533728 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c003b70 con 0x7f2038005180 2024-03-12T17:08:49.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.624+0000 7f2051400640 1 -- 172.21.15.52:0/4043533728 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204c003d10 con 0x7f2038005180 2024-03-12T17:08:49.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.624+0000 7f2051400640 1 -- 172.21.15.52:0/4043533728 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c005310 con 0x7f2038005180 2024-03-12T17:08:49.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.624+0000 7f2053200640 1 -- 172.21.15.52:0/4043533728 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2038005180 msgr2=0x7f203800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:49.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.624+0000 7f2053200640 1 --2- 172.21.15.52:0/4043533728 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2038005180 0x7f203800b980 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f204c001e50 tx=0x7f20380057f0 comp rx=0 tx=0).stop 2024-03-12T17:08:49.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.625+0000 7f2053200640 1 -- 172.21.15.52:0/4043533728 shutdown_connections 2024-03-12T17:08:49.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.625+0000 7f2053200640 1 --2- 172.21.15.52:0/4043533728 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2038005180 0x7f203800b980 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:49.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.625+0000 7f2053200640 1 -- 172.21.15.52:0/4043533728 >> 172.21.15.52:0/4043533728 conn(0x7f2054065e20 msgr2=0x7f2054069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:49.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.625+0000 7f2053200640 1 -- 172.21.15.52:0/4043533728 shutdown_connections 2024-03-12T17:08:49.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.625+0000 7f2053200640 1 -- 172.21.15.52:0/4043533728 wait complete. 2024-03-12T17:08:49.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.626+0000 7f2053200640 1 Processor -- start 2024-03-12T17:08:49.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.627+0000 7f2053200640 1 -- start start 2024-03-12T17:08:49.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.627+0000 7f2053200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2038005180 0x7f205414fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:49.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.627+0000 7f2053200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f205415c6a0 0x7f2054152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:49.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.627+0000 7f2053200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f205413fd60 0x7f20541f5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:49.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.627+0000 7f2053200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2054153020 con 0x7f205413fd60 2024-03-12T17:08:49.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.627+0000 7f2053200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20541502f0 con 0x7f205415c6a0 2024-03-12T17:08:49.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.627+0000 7f2053200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205415b8e0 con 0x7f2038005180 2024-03-12T17:08:49.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.628+0000 7f2052800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f205413fd60 0x7f20541f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:49.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.628+0000 7f2052800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f205413fd60 0x7f20541f5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59396/0 (socket says 172.21.15.52:59396) 2024-03-12T17:08:49.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.628+0000 7f2052800640 1 -- 172.21.15.52:0/1757268508 learned_addr learned my addr 172.21.15.52:0/1757268508 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:08:49.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.628+0000 7f2051e00640 1 --2- 172.21.15.52:0/1757268508 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2038005180 0x7f205414fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:49.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.628+0000 7f204be00640 1 --2- 172.21.15.52:0/1757268508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f205415c6a0 0x7f2054152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:49.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.628+0000 7f2052800640 1 -- 172.21.15.52:0/1757268508 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2038005180 msgr2=0x7f205414fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:49.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.628+0000 7f2052800640 1 --2- 172.21.15.52:0/1757268508 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2038005180 0x7f205414fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:49.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.628+0000 7f2052800640 1 -- 172.21.15.52:0/1757268508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f205415c6a0 msgr2=0x7f2054152ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:49.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.628+0000 7f2052800640 1 --2- 172.21.15.52:0/1757268508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f205415c6a0 0x7f2054152ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:49.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.628+0000 7f2052800640 1 -- 172.21.15.52:0/1757268508 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f205415bbc0 con 0x7f205413fd60 2024-03-12T17:08:49.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.629+0000 7f2052800640 1 --2- 172.21.15.52:0/1757268508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f205413fd60 0x7f20541f5860 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f204c005040 tx=0x7f204c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:49.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.629+0000 7f204a000640 1 -- 172.21.15.52:0/1757268508 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c006d50 con 0x7f205413fd60 2024-03-12T17:08:49.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.629+0000 7f204a000640 1 -- 172.21.15.52:0/1757268508 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204c03d070 con 0x7f205413fd60 2024-03-12T17:08:49.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.629+0000 7f204a000640 1 -- 172.21.15.52:0/1757268508 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c007230 con 0x7f205413fd60 2024-03-12T17:08:49.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.629+0000 7f2053200640 1 -- 172.21.15.52:0/1757268508 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f205415bea0 con 0x7f205413fd60 2024-03-12T17:08:49.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.629+0000 7f2053200640 1 -- 172.21.15.52:0/1757268508 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20541401f0 con 0x7f205413fd60 2024-03-12T17:08:49.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.630+0000 7f2053200640 1 -- 172.21.15.52:0/1757268508 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f205415c3c0 con 0x7f205413fd60 2024-03-12T17:08:49.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.631+0000 7f204a000640 1 -- 172.21.15.52:0/1757268508 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f204c007390 con 0x7f205413fd60 2024-03-12T17:08:49.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.632+0000 7f204a000640 1 --2- 172.21.15.52:0/1757268508 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f203006ac30 0x7f203006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:08:49.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.632+0000 7f204a000640 1 -- 172.21.15.52:0/1757268508 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f204c0ab3a0 con 0x7f205413fd60 2024-03-12T17:08:49.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.633+0000 7f2051e00640 1 --2- 172.21.15.52:0/1757268508 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f203006ac30 0x7f203006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:08:49.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.634+0000 7f2051e00640 1 --2- 172.21.15.52:0/1757268508 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f203006ac30 0x7f203006d0f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2044001050 tx=0x7f2044009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:08:49.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.637+0000 7f204a000640 1 -- 172.21.15.52:0/1757268508 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f205415c3c0 con 0x7f205413fd60 2024-03-12T17:08:49.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.898+0000 7f2053200640 1 -- 172.21.15.52:0/1757268508 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.admin"} v 0) v1 -- 0x7f205415c3c0 con 0x7f205413fd60 2024-03-12T17:08:49.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.899+0000 7f204a000640 1 -- 172.21.15.52:0/1757268508 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.admin"}]=0 v9) v1 ==== 92+0+63 (secure 0 0 0) 0x7f205415c3c0 con 0x7f205413fd60 2024-03-12T17:08:49.893 INFO:teuthology.orchestra.run.smithi052.stdout:[client.admin] 2024-03-12T17:08:49.893 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBVi/Bl0DH4ERAAr0Oaf2K8P8SdXQjliAt2TA== 2024-03-12T17:08:49.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.901+0000 7f2053200640 1 -- 172.21.15.52:0/1757268508 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f203006ac30 msgr2=0x7f203006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:49.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.901+0000 7f2053200640 1 --2- 172.21.15.52:0/1757268508 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f203006ac30 0x7f203006d0f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2044001050 tx=0x7f2044009040 comp rx=0 tx=0).stop 2024-03-12T17:08:49.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.901+0000 7f2053200640 1 -- 172.21.15.52:0/1757268508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f205413fd60 msgr2=0x7f20541f5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:08:49.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.901+0000 7f2053200640 1 --2- 172.21.15.52:0/1757268508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f205413fd60 0x7f20541f5860 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f204c005040 tx=0x7f204c0014a0 comp rx=0 tx=0).stop 2024-03-12T17:08:49.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.901+0000 7f2053200640 1 -- 172.21.15.52:0/1757268508 shutdown_connections 2024-03-12T17:08:49.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.901+0000 7f2053200640 1 --2- 172.21.15.52:0/1757268508 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f203006ac30 0x7f203006d0f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:49.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.901+0000 7f2053200640 1 --2- 172.21.15.52:0/1757268508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f205413fd60 0x7f20541f5860 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:49.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.901+0000 7f2053200640 1 --2- 172.21.15.52:0/1757268508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f205415c6a0 0x7f2054152ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:49.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.901+0000 7f2053200640 1 --2- 172.21.15.52:0/1757268508 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2038005180 0x7f205414fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:08:49.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.901+0000 7f2053200640 1 -- 172.21.15.52:0/1757268508 >> 172.21.15.52:0/1757268508 conn(0x7f2054065e20 msgr2=0x7f2054070020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:08:49.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.902+0000 7f2053200640 1 -- 172.21.15.52:0/1757268508 shutdown_connections 2024-03-12T17:08:49.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:08:49.902+0000 7f2053200640 1 -- 172.21.15.52:0/1757268508 wait complete. 2024-03-12T17:08:49.916 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:08:49.916 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/tmp/tmp.lWf7ZzDCDzxfstests-dev/local.config 2024-03-12T17:08:50.010 INFO:tasks.cephfs.xfstests_dev:local.config's contents - export FSTYP=ceph export TEST_DEV=172.21.15.52:6789:/test export TEST_DIR=/tmp/tmp.RgkcaM9vDRtest export SCRATCH_DEV=172.21.15.52:6789:/scratch export SCRATCH_MNT=/tmp/tmp.fpQ4ymYXq9scratch export CEPHFS_MOUNT_OPTIONS="-o name=admin,secret=AQBVi/Bl0DH4ERAAr0Oaf2K8P8SdXQjliAt2TA==" 2024-03-12T17:08:50.011 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:08:50.011 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/tmp/tmp.lWf7ZzDCDzxfstests-dev/ceph.exclude 2024-03-12T17:08:50.151 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.lWf7ZzDCDzxfstests-dev && exec sudo make) 2024-03-12T17:08:50.305 INFO:teuthology.orchestra.run.smithi052.stdout:libtoolize -cfi 2024-03-12T17:08:51.006 INFO:teuthology.orchestra.run.smithi052.stdout:libtoolize: putting auxiliary files in '.'. 2024-03-12T17:08:51.006 INFO:teuthology.orchestra.run.smithi052.stdout:libtoolize: copying file './config.guess' 2024-03-12T17:08:51.020 INFO:teuthology.orchestra.run.smithi052.stdout:libtoolize: copying file './config.sub' 2024-03-12T17:08:51.036 INFO:teuthology.orchestra.run.smithi052.stdout:libtoolize: copying file './install-sh' 2024-03-12T17:08:51.052 INFO:teuthology.orchestra.run.smithi052.stdout:libtoolize: copying file './ltmain.sh' 2024-03-12T17:08:51.146 INFO:teuthology.orchestra.run.smithi052.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-03-12T17:08:51.146 INFO:teuthology.orchestra.run.smithi052.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-03-12T17:08:51.166 INFO:teuthology.orchestra.run.smithi052.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-03-12T17:08:51.183 INFO:teuthology.orchestra.run.smithi052.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-03-12T17:08:51.201 INFO:teuthology.orchestra.run.smithi052.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-03-12T17:08:51.220 INFO:teuthology.orchestra.run.smithi052.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-03-12T17:08:51.244 INFO:teuthology.orchestra.run.smithi052.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-03-12T17:08:51.249 INFO:teuthology.orchestra.run.smithi052.stdout:cp include/install-sh . 2024-03-12T17:08:51.253 INFO:teuthology.orchestra.run.smithi052.stdout:aclocal -I m4 2024-03-12T17:08:52.672 INFO:teuthology.orchestra.run.smithi052.stdout:autoheader 2024-03-12T17:08:53.096 INFO:teuthology.orchestra.run.smithi052.stdout:autoconf 2024-03-12T17:08:53.355 INFO:teuthology.orchestra.run.smithi052.stdout:./configure \ 2024-03-12T17:08:53.355 INFO:teuthology.orchestra.run.smithi052.stdout: --libexecdir=/usr/lib \ 2024-03-12T17:08:53.355 INFO:teuthology.orchestra.run.smithi052.stdout: --exec_prefix=/var/lib 2024-03-12T17:08:53.809 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-12T17:08:53.815 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-12T17:08:53.822 INFO:teuthology.orchestra.run.smithi052.stdout:checking build system type... x86_64-pc-linux-gnu 2024-03-12T17:08:53.823 INFO:teuthology.orchestra.run.smithi052.stdout:checking host system type... x86_64-pc-linux-gnu 2024-03-12T17:08:53.827 INFO:teuthology.orchestra.run.smithi052.stdout:checking how to print strings... printf 2024-03-12T17:08:53.827 INFO:teuthology.orchestra.run.smithi052.stdout:checking for gcc... gcc 2024-03-12T17:08:53.957 INFO:teuthology.orchestra.run.smithi052.stdout:checking whether the C compiler works... yes 2024-03-12T17:08:53.957 INFO:teuthology.orchestra.run.smithi052.stdout:checking for C compiler default output file name... a.out 2024-03-12T17:08:54.031 INFO:teuthology.orchestra.run.smithi052.stdout:checking for suffix of executables... 2024-03-12T17:08:54.134 INFO:teuthology.orchestra.run.smithi052.stdout:checking whether we are cross compiling... no 2024-03-12T17:08:54.187 INFO:teuthology.orchestra.run.smithi052.stdout:checking for suffix of object files... o 2024-03-12T17:08:54.235 INFO:teuthology.orchestra.run.smithi052.stdout:checking whether we are using the GNU C compiler... yes 2024-03-12T17:08:54.284 INFO:teuthology.orchestra.run.smithi052.stdout:checking whether gcc accepts -g... yes 2024-03-12T17:08:54.355 INFO:teuthology.orchestra.run.smithi052.stdout:checking for gcc option to accept ISO C89... none needed 2024-03-12T17:08:54.364 INFO:teuthology.orchestra.run.smithi052.stdout:checking for a sed that does not truncate output... /bin/sed 2024-03-12T17:08:54.371 INFO:teuthology.orchestra.run.smithi052.stdout:checking for grep that handles long lines and -e... /bin/grep 2024-03-12T17:08:54.376 INFO:teuthology.orchestra.run.smithi052.stdout:checking for egrep... /bin/grep -E 2024-03-12T17:08:54.381 INFO:teuthology.orchestra.run.smithi052.stdout:checking for fgrep... /bin/grep -F 2024-03-12T17:08:54.393 INFO:teuthology.orchestra.run.smithi052.stdout:checking for ld used by gcc... /bin/ld 2024-03-12T17:08:54.399 INFO:teuthology.orchestra.run.smithi052.stdout:checking if the linker (/bin/ld) is GNU ld... yes 2024-03-12T17:08:54.413 INFO:teuthology.orchestra.run.smithi052.stdout:checking for BSD- or MS-compatible name lister (nm)... /bin/nm -B 2024-03-12T17:08:55.496 INFO:teuthology.orchestra.run.smithi052.stdout:checking the name lister (/bin/nm -B) interface... BSD nm 2024-03-12T17:08:55.496 INFO:teuthology.orchestra.run.smithi052.stdout:checking whether ln -s works... yes 2024-03-12T17:08:55.517 INFO:teuthology.orchestra.run.smithi052.stdout:checking the maximum length of command line arguments... 1572864 2024-03-12T17:08:55.526 INFO:teuthology.orchestra.run.smithi052.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-03-12T17:08:55.526 INFO:teuthology.orchestra.run.smithi052.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-03-12T17:08:55.526 INFO:teuthology.orchestra.run.smithi052.stdout:checking for /bin/ld option to reload object files... -r 2024-03-12T17:08:55.527 INFO:teuthology.orchestra.run.smithi052.stdout:checking for objdump... objdump 2024-03-12T17:08:55.527 INFO:teuthology.orchestra.run.smithi052.stdout:checking how to recognize dependent libraries... pass_all 2024-03-12T17:08:55.528 INFO:teuthology.orchestra.run.smithi052.stdout:checking for dlltool... no 2024-03-12T17:08:55.528 INFO:teuthology.orchestra.run.smithi052.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-03-12T17:08:55.528 INFO:teuthology.orchestra.run.smithi052.stdout:checking for ar... ar 2024-03-12T17:08:55.644 INFO:teuthology.orchestra.run.smithi052.stdout:checking for archiver @FILE support... @ 2024-03-12T17:08:55.644 INFO:teuthology.orchestra.run.smithi052.stdout:checking for strip... strip 2024-03-12T17:08:55.645 INFO:teuthology.orchestra.run.smithi052.stdout:checking for ranlib... ranlib 2024-03-12T17:08:55.646 INFO:teuthology.orchestra.run.smithi052.stdout:checking for gawk... gawk 2024-03-12T17:08:55.811 INFO:teuthology.orchestra.run.smithi052.stdout:checking command to parse /bin/nm -B output from gcc object... ok 2024-03-12T17:08:55.815 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sysroot... no 2024-03-12T17:08:55.826 INFO:teuthology.orchestra.run.smithi052.stdout:checking for a working dd... /bin/dd 2024-03-12T17:08:55.837 INFO:teuthology.orchestra.run.smithi052.stdout:checking how to truncate binary pipes... /bin/dd bs=4096 count=1 2024-03-12T17:08:55.874 INFO:teuthology.orchestra.run.smithi052.stdout:checking for mt... no 2024-03-12T17:08:55.881 INFO:teuthology.orchestra.run.smithi052.stdout:checking if : is a manifest tool... no 2024-03-12T17:08:55.976 INFO:teuthology.orchestra.run.smithi052.stdout:checking how to run the C preprocessor... gcc -E 2024-03-12T17:08:56.330 INFO:teuthology.orchestra.run.smithi052.stdout:checking for ANSI C header files... yes 2024-03-12T17:08:56.385 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/types.h... yes 2024-03-12T17:08:56.465 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/stat.h... yes 2024-03-12T17:08:56.532 INFO:teuthology.orchestra.run.smithi052.stdout:checking for stdlib.h... yes 2024-03-12T17:08:56.594 INFO:teuthology.orchestra.run.smithi052.stdout:checking for string.h... yes 2024-03-12T17:08:56.668 INFO:teuthology.orchestra.run.smithi052.stdout:checking for memory.h... yes 2024-03-12T17:08:56.735 INFO:teuthology.orchestra.run.smithi052.stdout:checking for strings.h... yes 2024-03-12T17:08:56.820 INFO:teuthology.orchestra.run.smithi052.stdout:checking for inttypes.h... yes 2024-03-12T17:08:56.884 INFO:teuthology.orchestra.run.smithi052.stdout:checking for stdint.h... yes 2024-03-12T17:08:56.964 INFO:teuthology.orchestra.run.smithi052.stdout:checking for unistd.h... yes 2024-03-12T17:08:57.039 INFO:teuthology.orchestra.run.smithi052.stdout:checking for dlfcn.h... yes 2024-03-12T17:08:57.052 INFO:teuthology.orchestra.run.smithi052.stdout:checking for objdir... .libs 2024-03-12T17:08:57.244 INFO:teuthology.orchestra.run.smithi052.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-03-12T17:08:57.244 INFO:teuthology.orchestra.run.smithi052.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-03-12T17:08:57.304 INFO:teuthology.orchestra.run.smithi052.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-03-12T17:08:57.358 INFO:teuthology.orchestra.run.smithi052.stdout:checking if gcc static flag -static works... no 2024-03-12T17:08:57.432 INFO:teuthology.orchestra.run.smithi052.stdout:checking if gcc supports -c -o file.o... yes 2024-03-12T17:08:57.432 INFO:teuthology.orchestra.run.smithi052.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-03-12T17:08:57.454 INFO:teuthology.orchestra.run.smithi052.stdout:checking whether the gcc linker (/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-03-12T17:08:57.522 INFO:teuthology.orchestra.run.smithi052.stdout:checking whether -lc should be explicitly linked in... no 2024-03-12T17:08:57.660 INFO:teuthology.orchestra.run.smithi052.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-03-12T17:08:57.660 INFO:teuthology.orchestra.run.smithi052.stdout:checking how to hardcode library paths into programs... immediate 2024-03-12T17:08:57.665 INFO:teuthology.orchestra.run.smithi052.stdout:checking whether stripping libraries is possible... yes 2024-03-12T17:08:57.665 INFO:teuthology.orchestra.run.smithi052.stdout:checking if libtool supports shared libraries... yes 2024-03-12T17:08:57.665 INFO:teuthology.orchestra.run.smithi052.stdout:checking whether to build shared libraries... yes 2024-03-12T17:08:57.665 INFO:teuthology.orchestra.run.smithi052.stdout:checking whether to build static libraries... yes 2024-03-12T17:08:57.689 INFO:teuthology.orchestra.run.smithi052.stdout:checking for gcc... (cached) gcc 2024-03-12T17:08:57.754 INFO:teuthology.orchestra.run.smithi052.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-03-12T17:08:57.754 INFO:teuthology.orchestra.run.smithi052.stdout:checking whether gcc accepts -g... (cached) yes 2024-03-12T17:08:57.755 INFO:teuthology.orchestra.run.smithi052.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-03-12T17:08:57.756 INFO:teuthology.orchestra.run.smithi052.stdout:checking for gmake... /usr/bin/gmake 2024-03-12T17:08:57.757 INFO:teuthology.orchestra.run.smithi052.stderr:./configure: line 12082: test: $(SHELL): binary operator expected 2024-03-12T17:08:57.757 INFO:teuthology.orchestra.run.smithi052.stdout:checking for tar... /bin/tar 2024-03-12T17:08:57.758 INFO:teuthology.orchestra.run.smithi052.stdout:checking for gzip... /bin/gzip 2024-03-12T17:08:57.788 INFO:teuthology.orchestra.run.smithi052.stdout:checking whether gcc -MM is supported... yes 2024-03-12T17:08:57.789 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sort... /bin/sort 2024-03-12T17:08:57.789 INFO:teuthology.orchestra.run.smithi052.stdout:checking whether ln -s works... yes 2024-03-12T17:08:57.791 INFO:teuthology.orchestra.run.smithi052.stdout:checking for rpm... /bin/rpm 2024-03-12T17:08:57.804 INFO:teuthology.orchestra.run.smithi052.stdout:checking for rpmbuild... no 2024-03-12T17:08:57.881 INFO:teuthology.orchestra.run.smithi052.stdout:checking assert.h usability... yes 2024-03-12T17:08:57.910 INFO:teuthology.orchestra.run.smithi052.stdout:checking assert.h presence... yes 2024-03-12T17:08:57.911 INFO:teuthology.orchestra.run.smithi052.stdout:checking for assert.h... yes 2024-03-12T17:08:57.993 INFO:teuthology.orchestra.run.smithi052.stdout:checking bstring.h usability... no 2024-03-12T17:08:58.037 INFO:teuthology.orchestra.run.smithi052.stdout:checking bstring.h presence... no 2024-03-12T17:08:58.038 INFO:teuthology.orchestra.run.smithi052.stdout:checking for bstring.h... no 2024-03-12T17:08:58.113 INFO:teuthology.orchestra.run.smithi052.stdout:checking libgen.h usability... yes 2024-03-12T17:08:58.141 INFO:teuthology.orchestra.run.smithi052.stdout:checking libgen.h presence... yes 2024-03-12T17:08:58.141 INFO:teuthology.orchestra.run.smithi052.stdout:checking for libgen.h... yes 2024-03-12T17:08:58.231 INFO:teuthology.orchestra.run.smithi052.stdout:checking dirent.h usability... yes 2024-03-12T17:08:58.262 INFO:teuthology.orchestra.run.smithi052.stdout:checking dirent.h presence... yes 2024-03-12T17:08:58.262 INFO:teuthology.orchestra.run.smithi052.stdout:checking for dirent.h... yes 2024-03-12T17:08:58.372 INFO:teuthology.orchestra.run.smithi052.stdout:checking errno.h usability... yes 2024-03-12T17:08:58.400 INFO:teuthology.orchestra.run.smithi052.stdout:checking errno.h presence... yes 2024-03-12T17:08:58.401 INFO:teuthology.orchestra.run.smithi052.stdout:checking for errno.h... yes 2024-03-12T17:08:58.492 INFO:teuthology.orchestra.run.smithi052.stdout:checking malloc.h usability... yes 2024-03-12T17:08:58.525 INFO:teuthology.orchestra.run.smithi052.stdout:checking malloc.h presence... yes 2024-03-12T17:08:58.526 INFO:teuthology.orchestra.run.smithi052.stdout:checking for malloc.h... yes 2024-03-12T17:08:58.609 INFO:teuthology.orchestra.run.smithi052.stdout:checking uuid.h usability... no 2024-03-12T17:08:58.649 INFO:teuthology.orchestra.run.smithi052.stdout:checking uuid.h presence... no 2024-03-12T17:08:58.649 INFO:teuthology.orchestra.run.smithi052.stdout:checking for uuid.h... no 2024-03-12T17:08:58.744 INFO:teuthology.orchestra.run.smithi052.stdout:checking uuid/uuid.h usability... yes 2024-03-12T17:08:58.775 INFO:teuthology.orchestra.run.smithi052.stdout:checking uuid/uuid.h presence... yes 2024-03-12T17:08:58.775 INFO:teuthology.orchestra.run.smithi052.stdout:checking for uuid/uuid.h... yes 2024-03-12T17:08:58.854 INFO:teuthology.orchestra.run.smithi052.stdout:checking sys/uuid.h usability... no 2024-03-12T17:08:58.896 INFO:teuthology.orchestra.run.smithi052.stdout:checking sys/uuid.h presence... no 2024-03-12T17:08:58.896 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/uuid.h... no 2024-03-12T17:08:58.972 INFO:teuthology.orchestra.run.smithi052.stdout:checking sys/file.h usability... yes 2024-03-12T17:08:59.001 INFO:teuthology.orchestra.run.smithi052.stdout:checking sys/file.h presence... yes 2024-03-12T17:08:59.001 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/file.h... yes 2024-03-12T17:08:59.084 INFO:teuthology.orchestra.run.smithi052.stdout:checking sys/fcntl.h usability... yes 2024-03-12T17:08:59.115 INFO:teuthology.orchestra.run.smithi052.stdout:checking sys/fcntl.h presence... yes 2024-03-12T17:08:59.115 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/fcntl.h... yes 2024-03-12T17:08:59.199 INFO:teuthology.orchestra.run.smithi052.stdout:checking sys/syssgi.h usability... no 2024-03-12T17:08:59.243 INFO:teuthology.orchestra.run.smithi052.stdout:checking sys/syssgi.h presence... no 2024-03-12T17:08:59.243 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/syssgi.h... no 2024-03-12T17:08:59.343 INFO:teuthology.orchestra.run.smithi052.stdout:checking sys/param.h usability... yes 2024-03-12T17:08:59.376 INFO:teuthology.orchestra.run.smithi052.stdout:checking sys/param.h presence... yes 2024-03-12T17:08:59.376 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/param.h... yes 2024-03-12T17:08:59.399 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/stat.h... (cached) yes 2024-03-12T17:08:59.492 INFO:teuthology.orchestra.run.smithi052.stdout:checking sys/statvfs.h usability... yes 2024-03-12T17:08:59.521 INFO:teuthology.orchestra.run.smithi052.stdout:checking sys/statvfs.h presence... yes 2024-03-12T17:08:59.521 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/statvfs.h... yes 2024-03-12T17:08:59.601 INFO:teuthology.orchestra.run.smithi052.stdout:checking sys/time.h usability... yes 2024-03-12T17:08:59.633 INFO:teuthology.orchestra.run.smithi052.stdout:checking sys/time.h presence... yes 2024-03-12T17:08:59.634 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/time.h... yes 2024-03-12T17:08:59.727 INFO:teuthology.orchestra.run.smithi052.stdout:checking sys/ioctl.h usability... yes 2024-03-12T17:08:59.758 INFO:teuthology.orchestra.run.smithi052.stdout:checking sys/ioctl.h presence... yes 2024-03-12T17:08:59.758 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/ioctl.h... yes 2024-03-12T17:08:59.854 INFO:teuthology.orchestra.run.smithi052.stdout:checking sys/wait.h usability... yes 2024-03-12T17:08:59.886 INFO:teuthology.orchestra.run.smithi052.stdout:checking sys/wait.h presence... yes 2024-03-12T17:08:59.887 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/wait.h... yes 2024-03-12T17:08:59.905 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/types.h... (cached) yes 2024-03-12T17:08:59.922 INFO:teuthology.orchestra.run.smithi052.stdout:checking for strings.h... (cached) yes 2024-03-12T17:08:59.997 INFO:teuthology.orchestra.run.smithi052.stdout:checking err.h usability... yes 2024-03-12T17:09:00.028 INFO:teuthology.orchestra.run.smithi052.stdout:checking err.h presence... yes 2024-03-12T17:09:00.028 INFO:teuthology.orchestra.run.smithi052.stdout:checking for err.h... yes 2024-03-12T17:09:00.113 INFO:teuthology.orchestra.run.smithi052.stdout:checking linux/falloc.h usability... yes 2024-03-12T17:09:00.138 INFO:teuthology.orchestra.run.smithi052.stdout:checking linux/falloc.h presence... yes 2024-03-12T17:09:00.138 INFO:teuthology.orchestra.run.smithi052.stdout:checking for linux/falloc.h... yes 2024-03-12T17:09:00.217 INFO:teuthology.orchestra.run.smithi052.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-03-12T17:09:00.257 INFO:teuthology.orchestra.run.smithi052.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-03-12T17:09:00.258 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/fs/xfs_fsops.h... no 2024-03-12T17:09:00.332 INFO:teuthology.orchestra.run.smithi052.stdout:checking sys/fs/xfs_itable.h usability... no 2024-03-12T17:09:00.372 INFO:teuthology.orchestra.run.smithi052.stdout:checking sys/fs/xfs_itable.h presence... no 2024-03-12T17:09:00.373 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/fs/xfs_itable.h... no 2024-03-12T17:09:00.445 INFO:teuthology.orchestra.run.smithi052.stdout:checking xfs/platform_defs.h usability... no 2024-03-12T17:09:00.490 INFO:teuthology.orchestra.run.smithi052.stdout:checking xfs/platform_defs.h presence... no 2024-03-12T17:09:00.490 INFO:teuthology.orchestra.run.smithi052.stdout:checking for xfs/platform_defs.h... no 2024-03-12T17:09:00.566 INFO:teuthology.orchestra.run.smithi052.stdout:checking btrfs/ioctl.h usability... no 2024-03-12T17:09:00.610 INFO:teuthology.orchestra.run.smithi052.stdout:checking btrfs/ioctl.h presence... no 2024-03-12T17:09:00.610 INFO:teuthology.orchestra.run.smithi052.stdout:checking for btrfs/ioctl.h... no 2024-03-12T17:09:00.687 INFO:teuthology.orchestra.run.smithi052.stdout:checking cifs/ioctl.h usability... no 2024-03-12T17:09:00.730 INFO:teuthology.orchestra.run.smithi052.stdout:checking cifs/ioctl.h presence... no 2024-03-12T17:09:00.730 INFO:teuthology.orchestra.run.smithi052.stdout:checking for cifs/ioctl.h... no 2024-03-12T17:09:00.804 INFO:teuthology.orchestra.run.smithi052.stdout:checking sys/mman.h usability... yes 2024-03-12T17:09:00.835 INFO:teuthology.orchestra.run.smithi052.stdout:checking sys/mman.h presence... yes 2024-03-12T17:09:00.835 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/mman.h... yes 2024-03-12T17:09:00.893 INFO:teuthology.orchestra.run.smithi052.stdout:checking for xfs/xfs_log_format.h... no 2024-03-12T17:09:00.998 INFO:teuthology.orchestra.run.smithi052.stdout:checking for uuid_compare... no 2024-03-12T17:09:01.085 INFO:teuthology.orchestra.run.smithi052.stdout:checking for uuid_compare in -luuid... yes 2024-03-12T17:09:01.216 INFO:teuthology.orchestra.run.smithi052.stdout:checking for xfs/xfs.h... yes 2024-03-12T17:09:01.262 INFO:teuthology.orchestra.run.smithi052.stdout:checking for xfs/libxfs.h... no 2024-03-12T17:09:01.319 INFO:teuthology.orchestra.run.smithi052.stdout:checking whether xlog_assign_lsn is declared... no 2024-03-12T17:09:01.393 INFO:teuthology.orchestra.run.smithi052.stdout:checking for xfs/xqm.h... yes 2024-03-12T17:09:01.500 INFO:teuthology.orchestra.run.smithi052.stdout:checking xfsctl from xfs/xfs.h... ok 2024-03-12T17:09:01.562 INFO:teuthology.orchestra.run.smithi052.stdout:checking xfs/handle.h usability... yes 2024-03-12T17:09:01.589 INFO:teuthology.orchestra.run.smithi052.stdout:checking xfs/handle.h presence... yes 2024-03-12T17:09:01.589 INFO:teuthology.orchestra.run.smithi052.stdout:checking for xfs/handle.h... yes 2024-03-12T17:09:01.677 INFO:teuthology.orchestra.run.smithi052.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-03-12T17:09:01.745 INFO:teuthology.orchestra.run.smithi052.stdout:checking sys/acl.h usability... yes 2024-03-12T17:09:01.775 INFO:teuthology.orchestra.run.smithi052.stdout:checking sys/acl.h presence... yes 2024-03-12T17:09:01.775 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/acl.h... yes 2024-03-12T17:09:01.839 INFO:teuthology.orchestra.run.smithi052.stdout:checking acl/libacl.h usability... yes 2024-03-12T17:09:01.869 INFO:teuthology.orchestra.run.smithi052.stdout:checking acl/libacl.h presence... yes 2024-03-12T17:09:01.869 INFO:teuthology.orchestra.run.smithi052.stdout:checking for acl/libacl.h... yes 2024-03-12T17:09:01.955 INFO:teuthology.orchestra.run.smithi052.stdout:checking for acl_init in -lacl... yes 2024-03-12T17:09:02.022 INFO:teuthology.orchestra.run.smithi052.stdout:checking gdbm-ndbm.h usability... no 2024-03-12T17:09:02.063 INFO:teuthology.orchestra.run.smithi052.stdout:checking gdbm-ndbm.h presence... no 2024-03-12T17:09:02.063 INFO:teuthology.orchestra.run.smithi052.stdout:checking for gdbm-ndbm.h... no 2024-03-12T17:09:02.125 INFO:teuthology.orchestra.run.smithi052.stdout:checking gdbm/ndbm.h usability... no 2024-03-12T17:09:02.167 INFO:teuthology.orchestra.run.smithi052.stdout:checking gdbm/ndbm.h presence... no 2024-03-12T17:09:02.168 INFO:teuthology.orchestra.run.smithi052.stdout:checking for gdbm/ndbm.h... no 2024-03-12T17:09:02.235 INFO:teuthology.orchestra.run.smithi052.stdout:checking ndbm.h usability... no 2024-03-12T17:09:02.279 INFO:teuthology.orchestra.run.smithi052.stdout:checking ndbm.h presence... no 2024-03-12T17:09:02.279 INFO:teuthology.orchestra.run.smithi052.stdout:checking for ndbm.h... no 2024-03-12T17:09:02.349 INFO:teuthology.orchestra.run.smithi052.stdout:checking libaio.h usability... yes 2024-03-12T17:09:02.382 INFO:teuthology.orchestra.run.smithi052.stdout:checking libaio.h presence... yes 2024-03-12T17:09:02.382 INFO:teuthology.orchestra.run.smithi052.stdout:checking for libaio.h... yes 2024-03-12T17:09:02.453 INFO:teuthology.orchestra.run.smithi052.stdout:checking liburing.h usability... no 2024-03-12T17:09:02.496 INFO:teuthology.orchestra.run.smithi052.stdout:checking liburing.h presence... no 2024-03-12T17:09:02.497 INFO:teuthology.orchestra.run.smithi052.stdout:checking for liburing.h... no 2024-03-12T17:09:02.563 INFO:teuthology.orchestra.run.smithi052.stdout:checking sys/capability.h usability... no 2024-03-12T17:09:02.605 INFO:teuthology.orchestra.run.smithi052.stdout:checking sys/capability.h presence... no 2024-03-12T17:09:02.605 INFO:teuthology.orchestra.run.smithi052.stdout:checking for sys/capability.h... no 2024-03-12T17:09:02.671 INFO:teuthology.orchestra.run.smithi052.stdout:checking linux/fiemap.h usability... yes 2024-03-12T17:09:02.700 INFO:teuthology.orchestra.run.smithi052.stdout:checking linux/fiemap.h presence... yes 2024-03-12T17:09:02.700 INFO:teuthology.orchestra.run.smithi052.stdout:checking for linux/fiemap.h... yes 2024-03-12T17:09:02.794 INFO:teuthology.orchestra.run.smithi052.stdout:checking for fallocate... yes 2024-03-12T17:09:02.883 INFO:teuthology.orchestra.run.smithi052.stdout:checking for open_by_handle_at... yes 2024-03-12T17:09:02.952 INFO:teuthology.orchestra.run.smithi052.stdout:checking linux/fs.h usability... yes 2024-03-12T17:09:02.981 INFO:teuthology.orchestra.run.smithi052.stdout:checking linux/fs.h presence... yes 2024-03-12T17:09:02.981 INFO:teuthology.orchestra.run.smithi052.stdout:checking for linux/fs.h... yes 2024-03-12T17:09:03.047 INFO:teuthology.orchestra.run.smithi052.stdout:checking btrfsutil.h usability... no 2024-03-12T17:09:03.092 INFO:teuthology.orchestra.run.smithi052.stdout:checking btrfsutil.h presence... no 2024-03-12T17:09:03.093 INFO:teuthology.orchestra.run.smithi052.stdout:checking for btrfsutil.h... no 2024-03-12T17:09:03.204 INFO:teuthology.orchestra.run.smithi052.stdout:checking for copy_file_range... yes 2024-03-12T17:09:03.299 INFO:teuthology.orchestra.run.smithi052.stdout:checking for SEEK_DATA... yes 2024-03-12T17:09:03.417 INFO:teuthology.orchestra.run.smithi052.stdout:checking for BMV_OF_SHARED... yes 2024-03-12T17:09:03.510 INFO:teuthology.orchestra.run.smithi052.stdout:checking for nftw... yes 2024-03-12T17:09:03.602 INFO:teuthology.orchestra.run.smithi052.stdout:checking for RLIMIT_NOFILE... yes 2024-03-12T17:09:03.684 INFO:teuthology.orchestra.run.smithi052.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-03-12T17:09:03.775 INFO:teuthology.orchestra.run.smithi052.stdout:checking for FICLONE... yes 2024-03-12T17:09:03.862 INFO:teuthology.orchestra.run.smithi052.stdout:checking for renameat2... yes 2024-03-12T17:09:03.951 INFO:teuthology.orchestra.run.smithi052.stdout:checking for reallocarray... yes 2024-03-12T17:09:04.054 INFO:teuthology.orchestra.run.smithi052.stdout:checking for struct mount_attr... yes 2024-03-12T17:09:04.166 INFO:teuthology.orchestra.run.smithi052.stdout:checking for struct btrfs_qgroup_limit... yes 2024-03-12T17:09:04.280 INFO:teuthology.orchestra.run.smithi052.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-03-12T17:09:04.394 INFO:teuthology.orchestra.run.smithi052.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-03-12T17:09:04.503 INFO:teuthology.orchestra.run.smithi052.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-03-12T17:09:04.608 INFO:teuthology.orchestra.run.smithi052.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-03-12T17:09:04.709 INFO:teuthology.orchestra.run.smithi052.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-03-12T17:09:04.816 INFO:teuthology.orchestra.run.smithi052.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-03-12T17:09:04.893 INFO:teuthology.orchestra.run.smithi052.stdout:checking linux/btrfs.h usability... yes 2024-03-12T17:09:04.922 INFO:teuthology.orchestra.run.smithi052.stdout:checking linux/btrfs.h presence... yes 2024-03-12T17:09:04.922 INFO:teuthology.orchestra.run.smithi052.stdout:checking for linux/btrfs.h... yes 2024-03-12T17:09:05.018 INFO:teuthology.orchestra.run.smithi052.stdout:checking linux/btrfs_tree.h usability... yes 2024-03-12T17:09:05.046 INFO:teuthology.orchestra.run.smithi052.stdout:checking linux/btrfs_tree.h presence... yes 2024-03-12T17:09:05.046 INFO:teuthology.orchestra.run.smithi052.stdout:checking for linux/btrfs_tree.h... yes 2024-03-12T17:09:05.101 INFO:teuthology.orchestra.run.smithi052.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-03-12T17:09:05.174 INFO:teuthology.orchestra.run.smithi052.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-03-12T17:09:05.217 INFO:teuthology.orchestra.run.smithi052.stdout:configure: creating ./config.status 2024-03-12T17:09:06.506 INFO:teuthology.orchestra.run.smithi052.stdout:config.status: creating include/builddefs 2024-03-12T17:09:06.550 INFO:teuthology.orchestra.run.smithi052.stdout:config.status: creating include/config.h 2024-03-12T17:09:06.584 INFO:teuthology.orchestra.run.smithi052.stdout:config.status: executing libtool commands 2024-03-12T17:09:06.702 INFO:teuthology.orchestra.run.smithi052.stdout:Building include 2024-03-12T17:09:06.716 INFO:teuthology.orchestra.run.smithi052.stdout:Building lib 2024-03-12T17:09:06.726 INFO:teuthology.orchestra.run.smithi052.stdout: [LTDEP] 2024-03-12T17:09:07.068 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] dataascii.lo 2024-03-12T17:09:07.342 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] databin.lo 2024-03-12T17:09:07.668 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] datapid.lo 2024-03-12T17:09:07.937 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] file_lock.lo 2024-03-12T17:09:08.262 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] forker.lo 2024-03-12T17:09:08.557 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] pattern.lo 2024-03-12T17:09:08.835 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] open_flags.lo 2024-03-12T17:09:09.144 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] random_range.lo 2024-03-12T17:09:09.533 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] string_to_tokens.lo 2024-03-12T17:09:09.781 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] str_to_bytes.lo 2024-03-12T17:09:10.083 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] tlibio.lo 2024-03-12T17:09:10.518 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] write_log.lo 2024-03-12T17:09:10.938 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] random.lo 2024-03-12T17:09:11.206 INFO:teuthology.orchestra.run.smithi052.stdout: [LD] 2024-03-12T17:09:11.431 INFO:teuthology.orchestra.run.smithi052.stdout:Building ltp 2024-03-12T17:09:11.440 INFO:teuthology.orchestra.run.smithi052.stdout: [DEP] 2024-03-12T17:09:11.761 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] doio 2024-03-12T17:09:12.703 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] fsstress 2024-03-12T17:09:14.443 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] fsx 2024-03-12T17:09:15.442 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] growfiles 2024-03-12T17:09:16.160 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] iogen 2024-03-12T17:09:16.767 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aio-stress 2024-03-12T17:09:17.430 INFO:teuthology.orchestra.run.smithi052.stdout:Building src 2024-03-12T17:09:17.444 INFO:teuthology.orchestra.run.smithi052.stdout: [DEP] 2024-03-12T17:09:21.512 INFO:teuthology.orchestra.run.smithi052.stdout: [DEP] 2024-03-12T17:09:21.590 INFO:teuthology.orchestra.run.smithi052.stdout: [DEP] 2024-03-12T17:09:21.613 INFO:teuthology.orchestra.run.smithi052.stdout: [DEP] 2024-03-12T17:09:22.126 INFO:teuthology.orchestra.run.smithi052.stdout: [DEP] 2024-03-12T17:09:22.179 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] dirstress 2024-03-12T17:09:22.641 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] fill 2024-03-12T17:09:22.979 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] fill2 2024-03-12T17:09:23.439 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] getpagesize 2024-03-12T17:09:23.766 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] holes 2024-03-12T17:09:23.942 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-12T17:09:23.948 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-12T17:09:24.148 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] lstat64 2024-03-12T17:09:24.521 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] nametest 2024-03-12T17:09:24.964 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] permname 2024-03-12T17:09:25.351 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] randholes 2024-03-12T17:09:25.804 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] runas 2024-03-12T17:09:26.207 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] truncfile 2024-03-12T17:09:26.572 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] usemem 2024-03-12T17:09:26.913 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] mmapcat 2024-03-12T17:09:27.240 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] append_reader 2024-03-12T17:09:27.564 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] append_writer 2024-03-12T17:09:27.900 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] dirperf 2024-03-12T17:09:28.312 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] metaperf 2024-03-12T17:09:28.804 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] devzero 2024-03-12T17:09:29.150 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] feature 2024-03-12T17:09:29.554 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] alloc 2024-03-12T17:09:29.939 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] fault 2024-03-12T17:09:30.309 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] fstest 2024-03-12T17:09:30.735 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_access_root 2024-03-12T17:09:31.085 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] godown 2024-03-12T17:09:31.464 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] resvtest 2024-03-12T17:09:31.839 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] writemod 2024-03-12T17:09:32.149 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] writev_on_pagefault 2024-03-12T17:09:32.519 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] makeextents 2024-03-12T17:09:32.914 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] itrash 2024-03-12T17:09:33.289 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] rename 2024-03-12T17:09:33.637 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] multi_open_unlink 2024-03-12T17:09:34.000 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] unwritten_sync 2024-03-12T17:09:34.396 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] genhashnames 2024-03-12T17:09:34.791 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_holes 2024-03-12T17:09:35.131 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_mmap_writev 2024-03-12T17:09:35.478 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_truncate_cmtime 2024-03-12T17:09:35.799 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] dirhash_collide 2024-03-12T17:09:36.179 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_rename_overwrite 2024-03-12T17:09:36.534 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] holetest 2024-03-12T17:09:36.969 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_truncate_self 2024-03-12T17:09:37.309 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] af_unix 2024-03-12T17:09:37.644 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_mmap_stale_pmd 2024-03-12T17:09:37.977 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_mmap_cow_race 2024-03-12T17:09:38.343 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_mmap_fallocate 2024-03-12T17:09:38.654 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] fsync-err 2024-03-12T17:09:39.014 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_mmap_write_ro 2024-03-12T17:09:39.370 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_ext4_dax_journal_corruption 2024-03-12T17:09:39.722 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_ext4_dax_inline_corruption 2024-03-12T17:09:40.039 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_ofd_locks 2024-03-12T17:09:40.384 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_mmap_collision 2024-03-12T17:09:40.790 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] mmap-write-concurrent 2024-03-12T17:09:41.140 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_get_file_time 2024-03-12T17:09:41.504 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_create_short_dirs 2024-03-12T17:09:41.865 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_create_long_dirs 2024-03-12T17:09:42.238 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_enospc 2024-03-12T17:09:42.633 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_mmap_writev_overlap 2024-03-12T17:09:42.968 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] checkpoint_journal 2024-03-12T17:09:43.315 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] mmap-rw-fault 2024-03-12T17:09:43.682 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] allocstale 2024-03-12T17:09:44.049 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_mmap_cow_memory_failure 2024-03-12T17:09:44.413 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] fake-dump-rootino 2024-03-12T17:09:44.724 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] dio-buf-fault 2024-03-12T17:09:45.044 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] rewinddir-test 2024-03-12T17:09:45.400 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] readdir-while-renames 2024-03-12T17:09:45.787 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] xfsctl 2024-03-12T17:09:46.179 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] bstat 2024-03-12T17:09:46.571 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_mtab 2024-03-12T17:09:46.935 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] getdevicesize 2024-03-12T17:09:47.294 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] preallo_rw_pattern_reader 2024-03-12T17:09:47.618 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] preallo_rw_pattern_writer 2024-03-12T17:09:47.942 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] ftrunc 2024-03-12T17:09:48.292 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] trunc 2024-03-12T17:09:48.651 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] fs_perms 2024-03-12T17:09:49.020 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] testx 2024-03-12T17:09:49.344 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] looptest 2024-03-12T17:09:49.750 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] locktest 2024-03-12T17:09:50.267 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] unwritten_mmap 2024-03-12T17:09:50.631 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] bulkstat_unlink_test 2024-03-12T17:09:51.013 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] deduperace 2024-03-12T17:09:51.413 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] bulkstat_unlink_test_modified 2024-03-12T17:09:51.818 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_dir_offset 2024-03-12T17:09:52.166 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_futimens 2024-03-12T17:09:52.476 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_immutable 2024-03-12T17:09:53.361 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] stale_handle 2024-03-12T17:09:53.735 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] pwrite_mmap_blocked 2024-03-12T17:09:54.051 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_dir_offset2 2024-03-12T17:09:54.075 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-12T17:09:54.080 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-12T17:09:54.414 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] seek_sanity_test 2024-03-12T17:09:55.078 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] seek_copy_test 2024-03-12T17:09:55.450 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_readdir_1 2024-03-12T17:09:55.752 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_readdir_2 2024-03-12T17:09:56.102 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] fsync-tester 2024-03-12T17:09:56.533 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] nsexec 2024-03-12T17:09:56.902 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] cloner 2024-03-12T17:09:57.286 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] renameat2 2024-03-12T17:09:57.645 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_getcwd 2024-03-12T17:09:58.006 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] e4compact 2024-03-12T17:09:58.393 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] test-nextquota 2024-03-12T17:09:58.778 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] punch-alternating 2024-03-12T17:09:59.163 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] attr-list-by-handle-cursor-test 2024-03-12T17:09:59.545 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] listxattr 2024-03-12T17:09:59.890 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] dio-interleaved 2024-03-12T17:10:00.250 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_dir_type 2024-03-12T17:10:00.607 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] dio-invalidate-cache 2024-03-12T17:10:01.012 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] stat_test 2024-03-12T17:10:01.491 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_encrypted_d_revalidate 2024-03-12T17:10:01.864 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] attr_replace_test 2024-03-12T17:10:02.228 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] swapon 2024-03-12T17:10:02.596 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] mkswap 2024-03-12T17:10:02.932 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_attr_corruption 2024-03-12T17:10:03.273 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_open_tmpfiles 2024-03-12T17:10:03.664 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] fscrypt-crypt-util 2024-03-12T17:10:04.554 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] bulkstat_null_ocount 2024-03-12T17:10:04.865 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] splice-test 2024-03-12T17:10:05.223 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] chprojid_fail 2024-03-12T17:10:05.596 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] detached_mounts_propagation 2024-03-12T17:10:05.968 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] ext4_resize 2024-03-12T17:10:06.288 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_readdir_3 2024-03-12T17:10:06.687 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] splice2pipe 2024-03-12T17:10:07.035 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] uuid_ioctl 2024-03-12T17:10:07.378 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_snapshot_deleted_subvolume 2024-03-12T17:10:07.735 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] fiemap-tester 2024-03-12T17:10:08.214 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_stripealign 2024-03-12T17:10:08.568 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] open_by_handle 2024-03-12T17:10:09.040 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_mmap_dio 2024-03-12T17:10:09.399 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] xfsfind 2024-03-12T17:10:09.800 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] t_reflink_read_race 2024-03-12T17:10:10.202 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] fssum 2024-03-12T17:10:10.885 INFO:teuthology.orchestra.run.smithi052.stdout:Building log-writes 2024-03-12T17:10:10.901 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] replay-log 2024-03-12T17:10:11.437 INFO:teuthology.orchestra.run.smithi052.stdout:Building perf 2024-03-12T17:10:11.447 INFO:teuthology.orchestra.run.smithi052.stdout:Building aio-dio-regress 2024-03-12T17:10:11.462 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aiocp 2024-03-12T17:10:11.877 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aio-dio-append-write-fallocate-race 2024-03-12T17:10:12.241 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aio-dio-append-write-read-race 2024-03-12T17:10:12.597 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aio-dio-cow-race 2024-03-12T17:10:12.950 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aio-dio-cycle-write 2024-03-12T17:10:13.349 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aio-dio-eof-race 2024-03-12T17:10:13.738 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aio-dio-extend-stat 2024-03-12T17:10:14.097 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aio-dio-fcntl-race 2024-03-12T17:10:14.449 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aio-dio-hole-filling-race 2024-03-12T17:10:14.814 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aio-dio-invalidate-failure 2024-03-12T17:10:15.173 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aio-dio-invalidate-readahead 2024-03-12T17:10:15.510 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aiodio_sparse2 2024-03-12T17:10:15.940 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aio-dio-subblock-eof-read 2024-03-12T17:10:16.254 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aio-dio-write-verify 2024-03-12T17:10:16.636 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-03-12T17:10:16.972 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-03-12T17:10:17.332 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] aio-last-ref-held-by-io 2024-03-12T17:10:17.689 INFO:teuthology.orchestra.run.smithi052.stdout:Building vfs 2024-03-12T17:10:17.704 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] vfstest 2024-03-12T17:10:20.203 INFO:teuthology.orchestra.run.smithi052.stdout: [CC] mount-idmapped 2024-03-12T17:10:20.830 INFO:teuthology.orchestra.run.smithi052.stdout:Building m4 2024-03-12T17:10:20.841 INFO:teuthology.orchestra.run.smithi052.stdout:Building common 2024-03-12T17:10:20.852 INFO:teuthology.orchestra.run.smithi052.stdout:Building tests 2024-03-12T17:10:20.866 INFO:teuthology.orchestra.run.smithi052.stdout:Building btrfs 2024-03-12T17:10:20.884 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/btrfs/group.list 2024-03-12T17:10:21.117 INFO:teuthology.orchestra.run.smithi052.stdout:Building ceph 2024-03-12T17:10:21.132 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/ceph/group.list 2024-03-12T17:10:21.190 INFO:teuthology.orchestra.run.smithi052.stdout:Building cifs 2024-03-12T17:10:21.205 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/cifs/group.list 2024-03-12T17:10:21.253 INFO:teuthology.orchestra.run.smithi052.stdout:Building ext4 2024-03-12T17:10:21.268 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/ext4/group.list 2024-03-12T17:10:21.413 INFO:teuthology.orchestra.run.smithi052.stdout:Building f2fs 2024-03-12T17:10:21.429 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/f2fs/group.list 2024-03-12T17:10:21.498 INFO:teuthology.orchestra.run.smithi052.stdout:Building generic 2024-03-12T17:10:21.518 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/generic/group.list 2024-03-12T17:10:21.917 INFO:teuthology.orchestra.run.smithi052.stdout:Building nfs 2024-03-12T17:10:21.934 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/nfs/group.list 2024-03-12T17:10:21.990 INFO:teuthology.orchestra.run.smithi052.stdout:Building ocfs2 2024-03-12T17:10:22.005 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/ocfs2/group.list 2024-03-12T17:10:22.059 INFO:teuthology.orchestra.run.smithi052.stdout:Building overlay 2024-03-12T17:10:22.074 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/overlay/group.list 2024-03-12T17:10:22.239 INFO:teuthology.orchestra.run.smithi052.stdout:Building perf 2024-03-12T17:10:22.254 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/perf/group.list 2024-03-12T17:10:22.298 INFO:teuthology.orchestra.run.smithi052.stdout:Building selftest 2024-03-12T17:10:22.313 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/selftest/group.list 2024-03-12T17:10:22.358 INFO:teuthology.orchestra.run.smithi052.stdout:Building shared 2024-03-12T17:10:22.375 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/shared/group.list 2024-03-12T17:10:22.445 INFO:teuthology.orchestra.run.smithi052.stdout:Building tmpfs 2024-03-12T17:10:22.461 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/tmpfs/group.list 2024-03-12T17:10:22.515 INFO:teuthology.orchestra.run.smithi052.stdout:Building udf 2024-03-12T17:10:22.530 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/udf/group.list 2024-03-12T17:10:22.574 INFO:teuthology.orchestra.run.smithi052.stdout:Building xfs 2024-03-12T17:10:22.593 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/xfs/group.list 2024-03-12T17:10:22.997 DEBUG:teuthology.orchestra.run.smithi052:> (cd /tmp/tmp.lWf7ZzDCDzxfstests-dev && exec sudo make install) 2024-03-12T17:10:23.136 INFO:teuthology.orchestra.run.smithi052.stdout:Building include 2024-03-12T17:10:23.146 INFO:teuthology.orchestra.run.smithi052.stdout:Building lib 2024-03-12T17:10:23.157 INFO:teuthology.orchestra.run.smithi052.stdout:Building ltp 2024-03-12T17:10:23.169 INFO:teuthology.orchestra.run.smithi052.stdout:Building src 2024-03-12T17:10:23.253 INFO:teuthology.orchestra.run.smithi052.stdout:Building log-writes 2024-03-12T17:10:23.263 INFO:teuthology.orchestra.run.smithi052.stdout:Building perf 2024-03-12T17:10:23.273 INFO:teuthology.orchestra.run.smithi052.stdout:Building aio-dio-regress 2024-03-12T17:10:23.284 INFO:teuthology.orchestra.run.smithi052.stdout:Building vfs 2024-03-12T17:10:23.297 INFO:teuthology.orchestra.run.smithi052.stdout:Building m4 2024-03-12T17:10:23.307 INFO:teuthology.orchestra.run.smithi052.stdout:Building common 2024-03-12T17:10:23.317 INFO:teuthology.orchestra.run.smithi052.stdout:Building tests 2024-03-12T17:10:23.331 INFO:teuthology.orchestra.run.smithi052.stdout:Building btrfs 2024-03-12T17:10:23.347 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/btrfs/group.list 2024-03-12T17:10:23.574 INFO:teuthology.orchestra.run.smithi052.stdout:Building ceph 2024-03-12T17:10:23.590 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/ceph/group.list 2024-03-12T17:10:23.647 INFO:teuthology.orchestra.run.smithi052.stdout:Building cifs 2024-03-12T17:10:23.661 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/cifs/group.list 2024-03-12T17:10:23.710 INFO:teuthology.orchestra.run.smithi052.stdout:Building ext4 2024-03-12T17:10:23.726 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/ext4/group.list 2024-03-12T17:10:23.878 INFO:teuthology.orchestra.run.smithi052.stdout:Building f2fs 2024-03-12T17:10:23.893 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/f2fs/group.list 2024-03-12T17:10:23.961 INFO:teuthology.orchestra.run.smithi052.stdout:Building generic 2024-03-12T17:10:23.980 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/generic/group.list 2024-03-12T17:10:24.205 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-12T17:10:24.213 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-12T17:10:24.350 INFO:teuthology.orchestra.run.smithi052.stdout:Building nfs 2024-03-12T17:10:24.364 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/nfs/group.list 2024-03-12T17:10:24.418 INFO:teuthology.orchestra.run.smithi052.stdout:Building ocfs2 2024-03-12T17:10:24.432 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/ocfs2/group.list 2024-03-12T17:10:24.484 INFO:teuthology.orchestra.run.smithi052.stdout:Building overlay 2024-03-12T17:10:24.499 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/overlay/group.list 2024-03-12T17:10:24.649 INFO:teuthology.orchestra.run.smithi052.stdout:Building perf 2024-03-12T17:10:24.664 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/perf/group.list 2024-03-12T17:10:24.707 INFO:teuthology.orchestra.run.smithi052.stdout:Building selftest 2024-03-12T17:10:24.721 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/selftest/group.list 2024-03-12T17:10:24.768 INFO:teuthology.orchestra.run.smithi052.stdout:Building shared 2024-03-12T17:10:24.784 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/shared/group.list 2024-03-12T17:10:24.853 INFO:teuthology.orchestra.run.smithi052.stdout:Building tmpfs 2024-03-12T17:10:24.868 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/tmpfs/group.list 2024-03-12T17:10:24.918 INFO:teuthology.orchestra.run.smithi052.stdout:Building udf 2024-03-12T17:10:24.933 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/udf/group.list 2024-03-12T17:10:24.976 INFO:teuthology.orchestra.run.smithi052.stdout:Building xfs 2024-03-12T17:10:24.995 INFO:teuthology.orchestra.run.smithi052.stdout: [GROUP] /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/xfs/group.list 2024-03-12T17:10:25.384 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C include install 2024-03-12T17:10:25.388 INFO:teuthology.orchestra.run.smithi052.stdout:gmake[1]: Nothing to be done for 'install'. 2024-03-12T17:10:25.388 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C lib install 2024-03-12T17:10:25.393 INFO:teuthology.orchestra.run.smithi052.stdout:gmake[1]: Nothing to be done for 'install'. 2024-03-12T17:10:25.394 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C ltp install 2024-03-12T17:10:25.401 INFO:teuthology.orchestra.run.smithi052.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/ltp 2024-03-12T17:10:25.434 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 doio fsstress fsx growfiles iogen aio-stress /var/lib/xfstests/ltp 2024-03-12T17:10:25.667 INFO:teuthology.orchestra.run.smithi052.stdout:../install-sh -o root -g root -m 755 rwtest.sh /var/lib/xfstests/ltp 2024-03-12T17:10:25.701 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C src install 2024-03-12T17:10:25.748 INFO:teuthology.orchestra.run.smithi052.stdout:Building log-writes 2024-03-12T17:10:25.759 INFO:teuthology.orchestra.run.smithi052.stdout:Building perf 2024-03-12T17:10:25.771 INFO:teuthology.orchestra.run.smithi052.stdout:Building aio-dio-regress 2024-03-12T17:10:25.783 INFO:teuthology.orchestra.run.smithi052.stdout:Building vfs 2024-03-12T17:10:25.792 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake -C log-writes install 2024-03-12T17:10:25.796 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/log-writes 2024-03-12T17:10:25.828 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 replay-log /var/lib/xfstests/src/log-writes 2024-03-12T17:10:25.858 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake -C perf install 2024-03-12T17:10:25.862 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/perf 2024-03-12T17:10:25.896 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 FioCompare.py fio-insert-and-compare.py FioResultDecoder.py generate-schema.py ResultData.py fio-results.sql /var/lib/xfstests/src/perf 2024-03-12T17:10:25.983 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake -C aio-dio-regress install 2024-03-12T17:10:25.988 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/aio-dio-regress 2024-03-12T17:10:26.024 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 aiocp aio-dio-append-write-fallocate-race aio-dio-append-write-read-race aio-dio-cow-race aio-dio-cycle-write aio-dio-eof-race aio-dio-extend-stat aio-dio-fcntl-race aio-dio-hole-filling-race aio-dio-invalidate-failure aio-dio-invalidate-readahead aiodio_sparse2 aio-dio-subblock-eof-read aio-dio-write-verify aio-free-ring-with-bogus-nr-pages aio-io-setup-with-nonwritable-context-pointer aio-last-ref-held-by-io /var/lib/xfstests/src/aio-dio-regress 2024-03-12T17:10:26.225 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake -C vfs install 2024-03-12T17:10:26.228 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/vfs 2024-03-12T17:10:26.261 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 vfstest mount-idmapped /var/lib/xfstests/src/vfs 2024-03-12T17:10:26.306 INFO:teuthology.orchestra.run.smithi052.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src 2024-03-12T17:10:26.340 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dirstress fill fill2 getpagesize holes lstat64 nametest permname randholes runas truncfile usemem mmapcat append_reader append_writer dirperf metaperf devzero feature alloc fault fstest t_access_root godown resvtest writemod writev_on_pagefault makeextents itrash rename multi_open_unlink unwritten_sync genhashnames t_holes t_mmap_writev t_truncate_cmtime dirhash_collide t_rename_overwrite holetest t_truncate_self af_unix t_mmap_stale_pmd t_mmap_cow_race t_mmap_fallocate fsync-err t_mmap_write_ro t_ext4_dax_journal_corruption t_ext4_dax_inline_corruption t_ofd_locks t_mmap_collision mmap-write-concurrent t_get_file_time t_create_short_dirs t_create_long_dirs t_enospc t_mmap_writev_overlap checkpoint_journal mmap-rw-fault allocstale t_mmap_cow_memory_failure fake-dump-rootino dio-buf-fault rewinddir-test readdir-while-renames xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest locktest unwritten_mmap bulkstat_unlink_test deduperace bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner renameat2 t_getcwd e4compact test-nextquota punch-alternating attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type dio-invalidate-cache stat_test t_encrypted_d_revalidate attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe uuid_ioctl t_snapshot_deleted_subvolume fiemap-tester t_stripealign open_by_handle t_mmap_dio xfsfind t_reflink_read_race fssum /var/lib/xfstests/src 2024-03-12T17:10:30.971 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dmerror fill2attr fill2fs fill2fs_check scaleread.sh btrfs_crc32c_forged_name.py popdir.pl popattr.py soak_duration.awk /var/lib/xfstests/src 2024-03-12T17:10:31.339 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 644 dumpfile /var/lib/xfstests/src 2024-03-12T17:10:31.414 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C m4 install 2024-03-12T17:10:31.418 INFO:teuthology.orchestra.run.smithi052.stdout:gmake[1]: Nothing to be done for 'install'. 2024-03-12T17:10:31.418 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C common install 2024-03-12T17:10:31.422 INFO:teuthology.orchestra.run.smithi052.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/common 2024-03-12T17:10:31.451 INFO:teuthology.orchestra.run.smithi052.stdout:../install-sh -o root -g root -m 644 * /var/lib/xfstests/common 2024-03-12T17:10:32.022 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C tests install 2024-03-12T17:10:32.026 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/btrfs/ install 2024-03-12T17:10:32.033 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/btrfs 2024-03-12T17:10:32.068 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 304 305 306 307 308 309 310 /var/lib/xfstests/tests/btrfs 2024-03-12T17:10:35.474 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/btrfs 2024-03-12T17:10:35.509 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out /var/lib/xfstests/tests/btrfs 2024-03-12T17:10:39.108 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/ceph/ install 2024-03-12T17:10:39.112 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ceph 2024-03-12T17:10:39.148 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 /var/lib/xfstests/tests/ceph 2024-03-12T17:10:39.224 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ceph 2024-03-12T17:10:39.258 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out /var/lib/xfstests/tests/ceph 2024-03-12T17:10:39.337 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/cifs/ install 2024-03-12T17:10:39.341 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/cifs 2024-03-12T17:10:39.377 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/cifs 2024-03-12T17:10:39.411 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/cifs 2024-03-12T17:10:39.445 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/cifs 2024-03-12T17:10:39.480 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/ext4/ install 2024-03-12T17:10:39.484 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ext4 2024-03-12T17:10:39.517 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 271 301 302 303 304 305 306 307 308 /var/lib/xfstests/tests/ext4 2024-03-12T17:10:40.242 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ext4 2024-03-12T17:10:40.275 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 001.out.default 001.out.nodelalloc 001.out.nozero 001.cfg 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 271.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out /var/lib/xfstests/tests/ext4 2024-03-12T17:10:41.058 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/f2fs/ install 2024-03-12T17:10:41.062 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/f2fs 2024-03-12T17:10:41.098 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 001 002 /var/lib/xfstests/tests/f2fs 2024-03-12T17:10:41.142 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/f2fs 2024-03-12T17:10:41.177 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 001.out 002.out /var/lib/xfstests/tests/f2fs 2024-03-12T17:10:41.223 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/generic/ install 2024-03-12T17:10:41.233 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/generic 2024-03-12T17:10:41.269 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 /var/lib/xfstests/tests/generic 2024-03-12T17:10:49.706 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/generic 2024-03-12T17:10:49.738 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 035.out.default 035.out.nfs 050.out.default 050.out.nojournal 050.out.xfsquota 035.cfg 050.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 607.out 608.out 609.out 610.out 611.out 612.out 613.out 614.out 615.out 616.out 617.out 618.out 619.out 620.out 621.out 622.out 623.out 624.out 625.out 626.out 627.out 628.out 629.out 630.out 631.out 632.out 633.out 634.out 635.out 636.out 637.out 638.out 639.out 640.out 641.out 642.out 643.out 644.out 645.out 646.out 647.out 648.out 649.out 650.out 651.out 652.out 653.out 654.out 655.out 656.out 657.out 658.out 659.out 660.out 661.out 662.out 663.out 664.out 665.out 666.out 667.out 668.out 669.out 670.out 671.out 672.out 673.out 674.out 675.out 676.out 677.out 678.out 679.out 680.out 681.out 682.out 683.out 684.out 685.out 686.out 687.out 688.out 689.out 690.out 691.out 692.out 693.out 694.out 695.out 696.out 697.out 698.out 699.out 700.out 701.out 702.out 703.out 704.out 705.out 706.out 707.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out /var/lib/xfstests/tests/generic 2024-03-12T17:10:54.344 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-12T17:10:54.352 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-12T17:10:58.227 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/nfs/ install 2024-03-12T17:10:58.231 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/nfs 2024-03-12T17:10:58.265 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/nfs 2024-03-12T17:10:58.298 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/nfs 2024-03-12T17:10:58.332 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/nfs 2024-03-12T17:10:58.366 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/ocfs2/ install 2024-03-12T17:10:58.370 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ocfs2 2024-03-12T17:10:58.405 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/ocfs2 2024-03-12T17:10:58.439 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ocfs2 2024-03-12T17:10:58.473 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/ocfs2 2024-03-12T17:10:58.508 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/overlay/ install 2024-03-12T17:10:58.513 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/overlay 2024-03-12T17:10:58.547 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 /var/lib/xfstests/tests/overlay 2024-03-12T17:10:59.801 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/overlay 2024-03-12T17:10:59.838 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out /var/lib/xfstests/tests/overlay 2024-03-12T17:11:01.121 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/perf/ install 2024-03-12T17:11:01.125 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/perf 2024-03-12T17:11:01.159 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/perf 2024-03-12T17:11:01.190 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/perf 2024-03-12T17:11:01.222 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/perf 2024-03-12T17:11:01.257 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/selftest/ install 2024-03-12T17:11:01.260 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/selftest 2024-03-12T17:11:01.297 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 /var/lib/xfstests/tests/selftest 2024-03-12T17:11:01.387 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/selftest 2024-03-12T17:11:01.421 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out /var/lib/xfstests/tests/selftest 2024-03-12T17:11:01.502 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/shared/ install 2024-03-12T17:11:01.506 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/shared 2024-03-12T17:11:01.542 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 002 032 298 /var/lib/xfstests/tests/shared 2024-03-12T17:11:01.592 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/shared 2024-03-12T17:11:01.627 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 002.out 032.out 298.out /var/lib/xfstests/tests/shared 2024-03-12T17:11:01.684 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/tmpfs/ install 2024-03-12T17:11:01.688 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/tmpfs 2024-03-12T17:11:01.722 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/tmpfs 2024-03-12T17:11:01.756 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/tmpfs 2024-03-12T17:11:01.790 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/tmpfs 2024-03-12T17:11:01.825 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/udf/ install 2024-03-12T17:11:01.829 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/udf 2024-03-12T17:11:01.863 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 102 /var/lib/xfstests/tests/udf 2024-03-12T17:11:01.898 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/udf 2024-03-12T17:11:01.933 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 102.out /var/lib/xfstests/tests/udf 2024-03-12T17:11:01.968 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.lWf7ZzDCDzxfstests-dev/tests/xfs/ install 2024-03-12T17:11:01.978 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/xfs 2024-03-12T17:11:02.015 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 /var/lib/xfstests/tests/xfs 2024-03-12T17:11:10.127 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/xfs 2024-03-12T17:11:10.162 INFO:teuthology.orchestra.run.smithi052.stdout:../../install-sh -o root -g root -m 644 033.out.crc 033.out.default 071.out.32 071.out.64 096.out.external 096.out.internal 216.out.64mblog 216.out.classic 033.cfg 216.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out 740.out 741.out 742.out 743.out 744.out 745.out 746.out 747.out 748.out 749.out 750.out 751.out 752.out 753.out 754.out 755.out 756.out 757.out 758.out 759.out 760.out 761.out 762.out 763.out 764.out 765.out 766.out 767.out 768.out 769.out 770.out 771.out 772.out 773.out 774.out 775.out 776.out 777.out 778.out 779.out 780.out 781.out 782.out 783.out 784.out 785.out 786.out 787.out 788.out 789.out 790.out 791.out 792.out 793.out 794.out 795.out 796.out 797.out 798.out 799.out 800.out /var/lib/xfstests/tests/xfs 2024-03-12T17:11:18.286 INFO:teuthology.orchestra.run.smithi052.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests 2024-03-12T17:11:18.323 INFO:teuthology.orchestra.run.smithi052.stdout:./install-sh -o root -g root -m 755 -d /var/lib/xfstests 2024-03-12T17:11:18.357 INFO:teuthology.orchestra.run.smithi052.stdout:./install-sh -o root -g root -m 755 check /var/lib/xfstests 2024-03-12T17:11:18.401 INFO:teuthology.orchestra.run.smithi052.stdout:./install-sh -o root -g root -m 644 randomize.awk /var/lib/xfstests 2024-03-12T17:11:18.431 INFO:tasks.cephfs.test_acls:client is kernel mounted 2024-03-12T17:11:18.431 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-12T17:11:18.432 DEBUG:teuthology.orchestra.run.smithi052:running tests for ACLs from xfstests-dev> (cd /tmp/tmp.lWf7ZzDCDzxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/099) 2024-03-12T17:11:18.995 INFO:teuthology.orchestra.run.smithi052.stdout:FSTYP -- ceph 2024-03-12T17:11:19.007 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM -- Linux/x86_64 smithi052 6.8.0-rc7-gbf761b6efc7e #1 SMP PREEMPT_DYNAMIC Tue Mar 12 08:24:03 UTC 2024 2024-03-12T17:11:19.008 INFO:teuthology.orchestra.run.smithi052.stdout:MKFS_OPTIONS -- 172.21.15.52:6789:/scratch 2024-03-12T17:11:19.011 INFO:teuthology.orchestra.run.smithi052.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBVi/Bl0DH4ERAAr0Oaf2K8P8SdXQjliAt2TA== -o context=system_u:object_r:root_t:s0 172.21.15.52:6789:/scratch /tmp/tmp.fpQ4ymYXq9scratch 2024-03-12T17:11:19.011 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:11:21.348 INFO:teuthology.orchestra.run.smithi052.stdout:generic/099 2s 2024-03-12T17:11:21.367 INFO:teuthology.orchestra.run.smithi052.stdout:Ran: generic/099 2024-03-12T17:11:21.367 INFO:teuthology.orchestra.run.smithi052.stdout:Passed all 1 tests 2024-03-12T17:11:21.376 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:11:21.476 INFO:tasks.cephfs.test_acls:Command return value: 0 2024-03-12T17:11:21.476 INFO:tasks.cephfs.test_acls:Command stdout - FSTYP -- ceph PLATFORM -- Linux/x86_64 smithi052 6.8.0-rc7-gbf761b6efc7e #1 SMP PREEMPT_DYNAMIC Tue Mar 12 08:24:03 UTC 2024 MKFS_OPTIONS -- 172.21.15.52:6789:/scratch MOUNT_OPTIONS -- -o name=admin,secret=AQBVi/Bl0DH4ERAAr0Oaf2K8P8SdXQjliAt2TA== -o context=system_u:object_r:root_t:s0 172.21.15.52:6789:/scratch /tmp/tmp.fpQ4ymYXq9scratch generic/099 2s Ran: generic/099 Passed all 1 tests 2024-03-12T17:11:21.476 INFO:tasks.cephfs.test_acls:Command stderr - 2024-03-12T17:11:21.476 DEBUG:teuthology.orchestra.run.smithi052:> sudo userdel --force --remove fsgqa 2024-03-12T17:11:22.244 DEBUG:teuthology.orchestra.run.smithi052:> sudo userdel --force --remove 123456-fsgqa 2024-03-12T17:11:23.043 DEBUG:teuthology.orchestra.run.smithi052:> sudo groupdel fsgqa 2024-03-12T17:11:23.152 INFO:teuthology.orchestra.run.smithi052.stderr:groupdel: group 'fsgqa' does not exist 2024-03-12T17:11:23.157 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-03-12T17:11:23.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:11:23.158 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo stat /tmp/tmp.lWf7ZzDCDzxfstests-dev/results) 2024-03-12T17:11:23.255 INFO:teuthology.orchestra.run.smithi052.stdout: File: /tmp/tmp.lWf7ZzDCDzxfstests-dev/results 2024-03-12T17:11:23.256 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 4096 Blocks: 8 IO Block: 4096 directory 2024-03-12T17:11:23.256 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 801h/2049d Inode: 533543 Links: 3 2024-03-12T17:11:23.256 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T17:11:23.256 INFO:teuthology.orchestra.run.smithi052.stdout:Context: unconfined_u:object_r:user_tmp_t:s0 2024-03-12T17:11:23.256 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-03-12 17:11:18.944402892 +0000 2024-03-12T17:11:23.256 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-03-12 17:11:21.368370995 +0000 2024-03-12T17:11:23.256 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-03-12 17:11:21.368370995 +0000 2024-03-12T17:11:23.256 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-03-12 17:11:18.944402892 +0000 2024-03-12T17:11:23.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:11:23.260 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p /var/log/ceph/xfstests-dev-results) 2024-03-12T17:11:23.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:11:23.351 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp -r /tmp/tmp.lWf7ZzDCDzxfstests-dev/results /var/log/ceph/xfstests-dev-results) 2024-03-12T17:11:23.451 INFO:tasks.cephfs.xfstests_dev:results dir from xfstests-dev has been saved; it was copied from /tmp/tmp.lWf7ZzDCDzxfstests-dev to /var/log/ceph. 2024-03-12T17:11:23.451 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -rf /tmp/tmp.lWf7ZzDCDzxfstests-dev 2024-03-12T17:11:23.776 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-12T17:11:23.777 DEBUG:teuthology.orchestra.run.smithi052:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-12T17:11:23.899 DEBUG:teuthology.orchestra.run.smithi102:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-12T17:11:24.021 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-12T17:11:24.022 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:11:24.022 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-12T17:11:24.082 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:11:24.082 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-12T17:11:24.141 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:11:24.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.364+0000 7f244b200640 1 Processor -- start 2024-03-12T17:11:24.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.365+0000 7f244b200640 1 -- start start 2024-03-12T17:11:24.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.366+0000 7f244b200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f244c0cf5c0 0x7f244c0b8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:24.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.366+0000 7f244b200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f244c0d0c20 0x7f244c0d10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:24.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.366+0000 7f244b200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f244c15ce90 con 0x7f244c15cab0 2024-03-12T17:11:24.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.366+0000 7f244b200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f244c0cddf0 con 0x7f244c0cf5c0 2024-03-12T17:11:24.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.366+0000 7f244b200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f244c0cd570 con 0x7f244c0d0c20 2024-03-12T17:11:24.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.367+0000 7f2449400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f244c0cf5c0 0x7f244c0b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:24.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.367+0000 7f2449400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f244c0cf5c0 0x7f244c0b8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49500/0 (socket says 172.21.15.52:49500) 2024-03-12T17:11:24.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.367+0000 7f2449400640 1 -- 172.21.15.52:0/1464774203 learned_addr learned my addr 172.21.15.52:0/1464774203 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:24.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.367+0000 7f244a800640 1 --2- 172.21.15.52:0/1464774203 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f244c0d0c20 0x7f244c0d10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:24.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.367+0000 7f2449400640 1 -- 172.21.15.52:0/1464774203 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f244c0d0c20 msgr2=0x7f244c0d10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:24.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.367+0000 7f2449400640 1 --2- 172.21.15.52:0/1464774203 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f244c0d0c20 0x7f244c0d10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:24.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.367+0000 7f2449400640 1 -- 172.21.15.52:0/1464774203 >> v1:172.21.15.102:6789/0 conn(0x7f244c15cab0 legacy=0x7f244c0ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:24.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.367+0000 7f2449400640 1 -- 172.21.15.52:0/1464774203 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244c161370 con 0x7f244c0cf5c0 2024-03-12T17:11:24.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.368+0000 7f2449400640 1 --2- 172.21.15.52:0/1464774203 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f244c0cf5c0 0x7f244c0b8050 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2434009870 tx=0x7f243402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb11b678038438d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:24.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.369+0000 7f2448a00640 1 -- 172.21.15.52:0/1464774203 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2434030840 con 0x7f244c0cf5c0 2024-03-12T17:11:24.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.369+0000 7f2448a00640 1 -- 172.21.15.52:0/1464774203 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24340309a0 con 0x7f244c0cf5c0 2024-03-12T17:11:24.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.369+0000 7f2448a00640 1 -- 172.21.15.52:0/1464774203 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2434030d30 con 0x7f244c0cf5c0 2024-03-12T17:11:24.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.369+0000 7f244b200640 1 -- 172.21.15.52:0/1464774203 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f244c0cf5c0 msgr2=0x7f244c0b8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:24.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.369+0000 7f244b200640 1 --2- 172.21.15.52:0/1464774203 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f244c0cf5c0 0x7f244c0b8050 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2434009870 tx=0x7f243402b980 comp rx=0 tx=0).stop 2024-03-12T17:11:24.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.370+0000 7f244b200640 1 -- 172.21.15.52:0/1464774203 shutdown_connections 2024-03-12T17:11:24.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.370+0000 7f244b200640 1 --2- 172.21.15.52:0/1464774203 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f244c0d0c20 0x7f244c0d10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:24.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.370+0000 7f244b200640 1 --2- 172.21.15.52:0/1464774203 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f244c0cf5c0 0x7f244c0b8050 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:24.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.370+0000 7f244b200640 1 -- 172.21.15.52:0/1464774203 >> 172.21.15.52:0/1464774203 conn(0x7f244c06ae90 msgr2=0x7f244c06b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:24.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.370+0000 7f244b200640 1 -- 172.21.15.52:0/1464774203 shutdown_connections 2024-03-12T17:11:24.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.370+0000 7f244b200640 1 -- 172.21.15.52:0/1464774203 wait complete. 2024-03-12T17:11:24.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.371+0000 7f244b200640 1 Processor -- start 2024-03-12T17:11:24.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.372+0000 7f244b200640 1 -- start start 2024-03-12T17:11:24.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.373+0000 7f244b200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f244c0cf5c0 0x7f244c0c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:24.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.373+0000 7f244b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f244c0d0c20 0x7f244c15dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:24.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.373+0000 7f244b200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f244c15cab0 0x7f244c151d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:24.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.373+0000 7f244b200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244c0ca9f0 con 0x7f244c0cf5c0 2024-03-12T17:11:24.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.373+0000 7f244b200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244c15e180 con 0x7f244c0d0c20 2024-03-12T17:11:24.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.373+0000 7f244b200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f244c0c9d70 con 0x7f244c15cab0 2024-03-12T17:11:24.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.373+0000 7f2449e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f244c0cf5c0 0x7f244c0c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:24.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.373+0000 7f2449e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f244c0cf5c0 0x7f244c0c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49510/0 (socket says 172.21.15.52:49510) 2024-03-12T17:11:24.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.373+0000 7f2449e00640 1 -- 172.21.15.52:0/2720235101 learned_addr learned my addr 172.21.15.52:0/2720235101 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:24.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.373+0000 7f2449e00640 1 -- 172.21.15.52:0/2720235101 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f244c15cab0 msgr2=0x7f244c151d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:24.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.373+0000 7f2449e00640 1 --2- 172.21.15.52:0/2720235101 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f244c15cab0 0x7f244c151d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:24.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.373+0000 7f2449e00640 1 -- 172.21.15.52:0/2720235101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f244c0d0c20 msgr2=0x7f244c15dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:24.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.373+0000 7f2449400640 1 --2- 172.21.15.52:0/2720235101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f244c0d0c20 0x7f244c15dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:24.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.373+0000 7f2449e00640 1 --2- 172.21.15.52:0/2720235101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f244c0d0c20 0x7f244c15dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:24.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.374+0000 7f2449e00640 1 -- 172.21.15.52:0/2720235101 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24340094d0 con 0x7f244c0cf5c0 2024-03-12T17:11:24.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.374+0000 7f2449400640 1 --2- 172.21.15.52:0/2720235101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f244c0d0c20 0x7f244c15dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:11:24.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.374+0000 7f2449e00640 1 --2- 172.21.15.52:0/2720235101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f244c0cf5c0 0x7f244c0c9770 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f243c00c3c0 tx=0x7f243c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:24.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.374+0000 7f2442000640 1 -- 172.21.15.52:0/2720235101 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c0052d0 con 0x7f244c0cf5c0 2024-03-12T17:11:24.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.374+0000 7f2442000640 1 -- 172.21.15.52:0/2720235101 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243c005450 con 0x7f244c0cf5c0 2024-03-12T17:11:24.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.374+0000 7f2442000640 1 -- 172.21.15.52:0/2720235101 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c007260 con 0x7f244c0cf5c0 2024-03-12T17:11:24.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.374+0000 7f244b200640 1 -- 172.21.15.52:0/2720235101 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f244c0ca050 con 0x7f244c0cf5c0 2024-03-12T17:11:24.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.375+0000 7f244b200640 1 -- 172.21.15.52:0/2720235101 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f244c0ca610 con 0x7f244c0cf5c0 2024-03-12T17:11:24.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.376+0000 7f244b200640 1 -- 172.21.15.52:0/2720235101 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f244c04e030 con 0x7f244c0cf5c0 2024-03-12T17:11:24.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.377+0000 7f2442000640 1 -- 172.21.15.52:0/2720235101 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f243c0073c0 con 0x7f244c0cf5c0 2024-03-12T17:11:24.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.377+0000 7f2442000640 1 --2- 172.21.15.52:0/2720235101 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f242806ac00 0x7f242806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:24.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.378+0000 7f2442000640 1 -- 172.21.15.52:0/2720235101 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f243c088850 con 0x7f244c0cf5c0 2024-03-12T17:11:24.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.378+0000 7f2449400640 1 --2- 172.21.15.52:0/2720235101 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f242806ac00 0x7f242806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:24.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.379+0000 7f2449400640 1 --2- 172.21.15.52:0/2720235101 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f242806ac00 0x7f242806d0c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2434036000 tx=0x7f243403b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:24.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.382+0000 7f2442000640 1 -- 172.21.15.52:0/2720235101 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f243c053960 con 0x7f244c0cf5c0 2024-03-12T17:11:24.493 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-12T17:11:24.501 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-12T17:11:24.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.583+0000 7f244b200640 1 -- 172.21.15.52:0/2720235101 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f244c0d02d0 con 0x7f244c0cf5c0 2024-03-12T17:11:24.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.584+0000 7f2442000640 1 -- 172.21.15.52:0/2720235101 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f243c056f80 con 0x7f244c0cf5c0 2024-03-12T17:11:24.579 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:11:24.579 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":42,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:09:13.132892+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T17:06:19.218543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-12T17:08:09.114678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-12T17:08:09.691941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.733031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.775178+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.591522+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.820838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.829252+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.582481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.d","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:11:24.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.587+0000 7f244b200640 1 -- 172.21.15.52:0/2720235101 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f242806ac00 msgr2=0x7f242806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:24.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.587+0000 7f244b200640 1 --2- 172.21.15.52:0/2720235101 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f242806ac00 0x7f242806d0c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2434036000 tx=0x7f243403b040 comp rx=0 tx=0).stop 2024-03-12T17:11:24.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.587+0000 7f244b200640 1 -- 172.21.15.52:0/2720235101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f244c0cf5c0 msgr2=0x7f244c0c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:24.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.587+0000 7f244b200640 1 --2- 172.21.15.52:0/2720235101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f244c0cf5c0 0x7f244c0c9770 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f243c00c3c0 tx=0x7f243c00c880 comp rx=0 tx=0).stop 2024-03-12T17:11:24.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.587+0000 7f244b200640 1 -- 172.21.15.52:0/2720235101 shutdown_connections 2024-03-12T17:11:24.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.587+0000 7f244b200640 1 --2- 172.21.15.52:0/2720235101 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f242806ac00 0x7f242806d0c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:24.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.587+0000 7f244b200640 1 --2- 172.21.15.52:0/2720235101 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f244c15cab0 0x7f244c151d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:24.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.587+0000 7f244b200640 1 --2- 172.21.15.52:0/2720235101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f244c0d0c20 0x7f244c15dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:24.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.587+0000 7f244b200640 1 --2- 172.21.15.52:0/2720235101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f244c0cf5c0 0x7f244c0c9770 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:24.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.587+0000 7f244b200640 1 -- 172.21.15.52:0/2720235101 >> 172.21.15.52:0/2720235101 conn(0x7f244c06ae90 msgr2=0x7f244c0b7650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:24.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.587+0000 7f244b200640 1 -- 172.21.15.52:0/2720235101 shutdown_connections 2024-03-12T17:11:24.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.587+0000 7f244b200640 1 -- 172.21.15.52:0/2720235101 wait complete. 2024-03-12T17:11:24.602 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T17:06:19.218543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-12T17:08:09.114678+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-12T17:08:09.691941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-12T17:11:24.602 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T17:11:24.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.830+0000 7f95e2800640 1 Processor -- start 2024-03-12T17:11:24.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.831+0000 7f95e2800640 1 -- start start 2024-03-12T17:11:24.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.832+0000 7f95e2800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f95dc0b0a90 0x7f95dc165250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:24.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.832+0000 7f95e2800640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95dc0b0fd0 con 0x7f95dc15a630 2024-03-12T17:11:24.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.832+0000 7f95e2800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95dc0b12b0 con 0x7f95dc0cd5e0 2024-03-12T17:11:24.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.832+0000 7f95e2800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95dc0b1590 con 0x7f95dc0b0a90 2024-03-12T17:11:24.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.832+0000 7f95e1400640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f95dc15a630 0x7f95dc0cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:43524/0 (socket says 172.21.15.52:43524) 2024-03-12T17:11:24.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.832+0000 7f95e1400640 1 -- 172.21.15.52:0/250497231 learned_addr learned my addr 172.21.15.52:0/250497231 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:11:24.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.833+0000 7f95dbe00640 1 -- 172.21.15.52:0/250497231 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1420829406 0 0) 0x7f95dc0b0fd0 con 0x7f95dc15a630 2024-03-12T17:11:24.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.833+0000 7f95dbe00640 1 -- 172.21.15.52:0/250497231 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95c4003520 con 0x7f95dc15a630 2024-03-12T17:11:24.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.833+0000 7f95e1e00640 1 --2- 172.21.15.52:0/250497231 >> v2:172.21.15.102:3300/0 conn(0x7f95dc0b0a90 0x7f95dc165250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:24.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.833+0000 7f95dbe00640 1 -- 172.21.15.52:0/250497231 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3694853596 0 0) 0x7f95c4003520 con 0x7f95dc15a630 2024-03-12T17:11:24.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.833+0000 7f95dbe00640 1 -- 172.21.15.52:0/250497231 >> v2:172.21.15.102:3300/0 conn(0x7f95dc0b0a90 msgr2=0x7f95dc165250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:24.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.833+0000 7f95dbe00640 1 --2- 172.21.15.52:0/250497231 >> v2:172.21.15.102:3300/0 conn(0x7f95dc0b0a90 0x7f95dc165250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:24.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.833+0000 7f95dbe00640 1 -- 172.21.15.52:0/250497231 >> v1:172.21.15.102:6789/0 conn(0x7f95dc0cd5e0 legacy=0x7f95dc0b0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:24.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.833+0000 7f95dbe00640 1 -- 172.21.15.52:0/250497231 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95dc0b1870 con 0x7f95dc15a630 2024-03-12T17:11:24.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.833+0000 7f95dbe00640 1 -- 172.21.15.52:0/250497231 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f95d0001690 con 0x7f95dc15a630 2024-03-12T17:11:24.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.833+0000 7f95dbe00640 1 -- 172.21.15.52:0/250497231 >> v1:172.21.15.52:6789/0 conn(0x7f95dc15a630 legacy=0x7f95dc0cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:24.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.833+0000 7f95dbe00640 1 --2- 172.21.15.52:0/250497231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95c4003970 0x7f95c4003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:24.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.833+0000 7f95dbe00640 1 --2- 172.21.15.52:0/250497231 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95c40047c0 0x7f95c4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:24.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.833+0000 7f95dbe00640 1 --2- 172.21.15.52:0/250497231 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95c40052a0 0x7f95c400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:24.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.833+0000 7f95dbe00640 1 -- 172.21.15.52:0/250497231 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc0b1590 con 0x7f95c4003970 2024-03-12T17:11:24.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.833+0000 7f95dbe00640 1 -- 172.21.15.52:0/250497231 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c400c150 con 0x7f95c40047c0 2024-03-12T17:11:24.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.833+0000 7f95dbe00640 1 -- 172.21.15.52:0/250497231 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c400c310 con 0x7f95c40052a0 2024-03-12T17:11:24.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.834+0000 7f95e1400640 1 --2- 172.21.15.52:0/250497231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95c4003970 0x7f95c4003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:24.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.834+0000 7f95e1400640 1 -- 172.21.15.52:0/250497231 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95c40052a0 msgr2=0x7f95c400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:24.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.834+0000 7f95e1400640 1 --2- 172.21.15.52:0/250497231 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95c40052a0 0x7f95c400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:24.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.834+0000 7f95e1400640 1 -- 172.21.15.52:0/250497231 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95c40047c0 msgr2=0x7f95c4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:24.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.834+0000 7f95e1400640 1 --2- 172.21.15.52:0/250497231 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95c40047c0 0x7f95c4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:24.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.834+0000 7f95e1400640 1 -- 172.21.15.52:0/250497231 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c4003520 con 0x7f95c4003970 2024-03-12T17:11:24.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.834+0000 7f95e1e00640 1 -- 172.21.15.52:0/250497231 reap_dead start 2024-03-12T17:11:24.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.835+0000 7f95e1400640 1 --2- 172.21.15.52:0/250497231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95c4003970 0x7f95c4003d70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f95d0001eb0 tx=0x7f95c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=721193b777ad56cb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:24.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.836+0000 7f95dbe00640 1 -- 172.21.15.52:0/250497231 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d0005ad0 con 0x7f95c4003970 2024-03-12T17:11:24.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.836+0000 7f95dbe00640 1 -- 172.21.15.52:0/250497231 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d0005c50 con 0x7f95c4003970 2024-03-12T17:11:24.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.836+0000 7f95dbe00640 1 -- 172.21.15.52:0/250497231 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d002c870 con 0x7f95c4003970 2024-03-12T17:11:24.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.836+0000 7f95e2800640 1 -- 172.21.15.52:0/250497231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95c4003970 msgr2=0x7f95c4003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:24.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.836+0000 7f95e2800640 1 --2- 172.21.15.52:0/250497231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95c4003970 0x7f95c4003d70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f95d0001eb0 tx=0x7f95c40057f0 comp rx=0 tx=0).stop 2024-03-12T17:11:24.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.836+0000 7f95e2800640 1 -- 172.21.15.52:0/250497231 shutdown_connections 2024-03-12T17:11:24.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.836+0000 7f95e2800640 1 --2- 172.21.15.52:0/250497231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95c4003970 0x7f95c4003d70 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:24.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.836+0000 7f95e2800640 1 -- 172.21.15.52:0/250497231 >> 172.21.15.52:0/250497231 conn(0x7f95dc0d0540 msgr2=0x7f95dc0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:24.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.836+0000 7f95e2800640 1 -- 172.21.15.52:0/250497231 shutdown_connections 2024-03-12T17:11:24.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.836+0000 7f95e2800640 1 -- 172.21.15.52:0/250497231 wait complete. 2024-03-12T17:11:24.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.838+0000 7f95e2800640 1 Processor -- start 2024-03-12T17:11:24.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.838+0000 7f95e2800640 1 -- start start 2024-03-12T17:11:24.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.839+0000 7f95e2800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95c4003970 0x7f95dc15d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:24.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.839+0000 7f95e2800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95dc06c9b0 0x7f95dc0c6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:24.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.839+0000 7f95e2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95dc0ad230 0x7f95dc0ad6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:24.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.839+0000 7f95e2800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc0c4050 con 0x7f95c4003970 2024-03-12T17:11:24.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.839+0000 7f95e2800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc0adbf0 con 0x7f95dc0ad230 2024-03-12T17:11:24.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.839+0000 7f95e2800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc06ce30 con 0x7f95dc06c9b0 2024-03-12T17:11:24.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.840+0000 7f95e1400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95c4003970 0x7f95dc15d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:24.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.840+0000 7f95e1400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95c4003970 0x7f95dc15d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49528/0 (socket says 172.21.15.52:49528) 2024-03-12T17:11:24.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.840+0000 7f95e1400640 1 -- 172.21.15.52:0/602426450 learned_addr learned my addr 172.21.15.52:0/602426450 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:24.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.840+0000 7f95e1e00640 1 --2- 172.21.15.52:0/602426450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95dc0ad230 0x7f95dc0ad6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:24.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.840+0000 7f95e1400640 1 -- 172.21.15.52:0/602426450 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95dc06c9b0 msgr2=0x7f95dc0c6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:24.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.840+0000 7f95e0a00640 1 --2- 172.21.15.52:0/602426450 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95dc06c9b0 0x7f95dc0c6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:24.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.840+0000 7f95e1400640 1 --2- 172.21.15.52:0/602426450 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95dc06c9b0 0x7f95dc0c6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:24.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.840+0000 7f95e1400640 1 -- 172.21.15.52:0/602426450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95dc0ad230 msgr2=0x7f95dc0ad6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:24.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.840+0000 7f95e1400640 1 --2- 172.21.15.52:0/602426450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95dc0ad230 0x7f95dc0ad6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:24.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.840+0000 7f95e1400640 1 -- 172.21.15.52:0/602426450 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95dc0c71f0 con 0x7f95c4003970 2024-03-12T17:11:24.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.840+0000 7f95e1e00640 1 --2- 172.21.15.52:0/602426450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95dc0ad230 0x7f95dc0ad6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:11:24.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.840+0000 7f95e1400640 1 --2- 172.21.15.52:0/602426450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95c4003970 0x7f95dc15d7b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f95d000c2d0 tx=0x7f95d002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:24.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.841+0000 7f95d9600640 1 -- 172.21.15.52:0/602426450 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d003c040 con 0x7f95c4003970 2024-03-12T17:11:24.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.841+0000 7f95d9600640 1 -- 172.21.15.52:0/602426450 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d002cc80 con 0x7f95c4003970 2024-03-12T17:11:24.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.841+0000 7f95e2800640 1 -- 172.21.15.52:0/602426450 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95dc0c4b90 con 0x7f95c4003970 2024-03-12T17:11:24.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.841+0000 7f95d9600640 1 -- 172.21.15.52:0/602426450 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d0007070 con 0x7f95c4003970 2024-03-12T17:11:24.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.841+0000 7f95e2800640 1 -- 172.21.15.52:0/602426450 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95dc0c4260 con 0x7f95c4003970 2024-03-12T17:11:24.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.842+0000 7f95e2800640 1 -- 172.21.15.52:0/602426450 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95dc0670d0 con 0x7f95c4003970 2024-03-12T17:11:24.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.843+0000 7f95d9600640 1 -- 172.21.15.52:0/602426450 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f95d0032070 con 0x7f95c4003970 2024-03-12T17:11:24.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.844+0000 7f95d9600640 1 --2- 172.21.15.52:0/602426450 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f95bc06ac00 0x7f95bc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:24.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.844+0000 7f95d9600640 1 -- 172.21.15.52:0/602426450 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f95d00aa980 con 0x7f95c4003970 2024-03-12T17:11:24.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.845+0000 7f95e0a00640 1 --2- 172.21.15.52:0/602426450 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f95bc06ac00 0x7f95bc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:24.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.846+0000 7f95e0a00640 1 --2- 172.21.15.52:0/602426450 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f95bc06ac00 0x7f95bc06d0c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f95dc1f55e0 tx=0x7f95c8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:24.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:24.849+0000 7f95d9600640 1 -- 172.21.15.52:0/602426450 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f95d0075a10 con 0x7f95c4003970 2024-03-12T17:11:25.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.051+0000 7f95e2800640 1 -- 172.21.15.52:0/602426450 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f95dc04e030 con 0x7f95c4003970 2024-03-12T17:11:25.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.052+0000 7f95d9600640 1 -- 172.21.15.52:0/602426450 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f95d0046070 con 0x7f95c4003970 2024-03-12T17:11:25.046 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-03-12T17:11:25.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.054+0000 7f95e2800640 1 -- 172.21.15.52:0/602426450 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f95bc06ac00 msgr2=0x7f95bc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:25.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.054+0000 7f95e2800640 1 --2- 172.21.15.52:0/602426450 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f95bc06ac00 0x7f95bc06d0c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f95dc1f55e0 tx=0x7f95c8009040 comp rx=0 tx=0).stop 2024-03-12T17:11:25.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.054+0000 7f95e2800640 1 -- 172.21.15.52:0/602426450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95c4003970 msgr2=0x7f95dc15d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:25.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.054+0000 7f95e2800640 1 --2- 172.21.15.52:0/602426450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95c4003970 0x7f95dc15d7b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f95d000c2d0 tx=0x7f95d002e070 comp rx=0 tx=0).stop 2024-03-12T17:11:25.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.054+0000 7f95e2800640 1 -- 172.21.15.52:0/602426450 shutdown_connections 2024-03-12T17:11:25.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.054+0000 7f95e2800640 1 --2- 172.21.15.52:0/602426450 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f95bc06ac00 0x7f95bc06d0c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:25.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.054+0000 7f95e2800640 1 --2- 172.21.15.52:0/602426450 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f95dc06c9b0 0x7f95dc0c6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:25.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.054+0000 7f95e2800640 1 --2- 172.21.15.52:0/602426450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95dc0ad230 0x7f95dc0ad6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:25.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.054+0000 7f95e2800640 1 --2- 172.21.15.52:0/602426450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95c4003970 0x7f95dc15d7b0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:25.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.054+0000 7f95e2800640 1 -- 172.21.15.52:0/602426450 >> 172.21.15.52:0/602426450 conn(0x7f95dc0d0540 msgr2=0x7f95dc098c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:25.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.055+0000 7f95e2800640 1 -- 172.21.15.52:0/602426450 shutdown_connections 2024-03-12T17:11:25.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.055+0000 7f95e2800640 1 -- 172.21.15.52:0/602426450 wait complete. 2024-03-12T17:11:25.067 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-12T17:11:25.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.294+0000 7f4e5fe00640 1 Processor -- start 2024-03-12T17:11:25.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.294+0000 7f4e5fe00640 1 -- start start 2024-03-12T17:11:25.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.295+0000 7f4e5fe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4e600cf5c0 0x7f4e600b8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:25.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.295+0000 7f4e5fe00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e600d0c20 0x7f4e600d10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:25.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.295+0000 7f4e5fe00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e6015ce90 con 0x7f4e6015cab0 2024-03-12T17:11:25.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.295+0000 7f4e5fe00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e600cddf0 con 0x7f4e600cf5c0 2024-03-12T17:11:25.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.295+0000 7f4e5fe00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e600cd570 con 0x7f4e600d0c20 2024-03-12T17:11:25.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.296+0000 7f4e5ea00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f4e6015cab0 0x7f4e600ceeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:40070/0 (socket says 172.21.15.52:40070) 2024-03-12T17:11:25.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.296+0000 7f4e5ea00640 1 -- 172.21.15.52:0/3002875114 learned_addr learned my addr 172.21.15.52:0/3002875114 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:11:25.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.296+0000 7f4e5e000640 1 --2- 172.21.15.52:0/3002875114 >> v2:172.21.15.52:3300/0 conn(0x7f4e600cf5c0 0x7f4e600b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:25.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.296+0000 7f4e5f400640 1 --2- 172.21.15.52:0/3002875114 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e600d0c20 0x7f4e600d10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:25.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.296+0000 7f4e5e000640 1 -- 172.21.15.52:0/3002875114 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e600d0c20 msgr2=0x7f4e600d10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:25.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.296+0000 7f4e5d600640 1 -- 172.21.15.52:0/3002875114 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4084614785 0 0) 0x7f4e6015ce90 con 0x7f4e6015cab0 2024-03-12T17:11:25.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.296+0000 7f4e5e000640 1 --2- 172.21.15.52:0/3002875114 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e600d0c20 0x7f4e600d10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:25.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.296+0000 7f4e5e000640 1 -- 172.21.15.52:0/3002875114 >> v1:172.21.15.52:6789/0 conn(0x7f4e6015cab0 legacy=0x7f4e600ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:25.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.296+0000 7f4e5e000640 1 -- 172.21.15.52:0/3002875114 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e60161370 con 0x7f4e600cf5c0 2024-03-12T17:11:25.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.296+0000 7f4e5f400640 1 --2- 172.21.15.52:0/3002875114 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e600d0c20 0x7f4e600d10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:11:25.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.297+0000 7f4e5e000640 1 --2- 172.21.15.52:0/3002875114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e600cf5c0 0x7f4e600b8050 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009d90 tx=0x7f4e4c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7446f285ecf329 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:25.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.298+0000 7f4e5d600640 1 -- 172.21.15.52:0/3002875114 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c02cc60 con 0x7f4e600cf5c0 2024-03-12T17:11:25.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.298+0000 7f4e5d600640 1 -- 172.21.15.52:0/3002875114 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e4c02cdc0 con 0x7f4e600cf5c0 2024-03-12T17:11:25.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.298+0000 7f4e5d600640 1 -- 172.21.15.52:0/3002875114 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c034660 con 0x7f4e600cf5c0 2024-03-12T17:11:25.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.298+0000 7f4e5fe00640 1 -- 172.21.15.52:0/3002875114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e600cf5c0 msgr2=0x7f4e600b8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:25.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.298+0000 7f4e5fe00640 1 --2- 172.21.15.52:0/3002875114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e600cf5c0 0x7f4e600b8050 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009d90 tx=0x7f4e4c02beb0 comp rx=0 tx=0).stop 2024-03-12T17:11:25.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.298+0000 7f4e5fe00640 1 -- 172.21.15.52:0/3002875114 shutdown_connections 2024-03-12T17:11:25.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.298+0000 7f4e5fe00640 1 --2- 172.21.15.52:0/3002875114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e600cf5c0 0x7f4e600b8050 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:25.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.298+0000 7f4e5fe00640 1 --2- 172.21.15.52:0/3002875114 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e600d0c20 0x7f4e600d10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:25.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.298+0000 7f4e5fe00640 1 -- 172.21.15.52:0/3002875114 >> 172.21.15.52:0/3002875114 conn(0x7f4e6006ae90 msgr2=0x7f4e6006b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:25.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.299+0000 7f4e5fe00640 1 -- 172.21.15.52:0/3002875114 shutdown_connections 2024-03-12T17:11:25.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.299+0000 7f4e5fe00640 1 -- 172.21.15.52:0/3002875114 wait complete. 2024-03-12T17:11:25.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.300+0000 7f4e5fe00640 1 Processor -- start 2024-03-12T17:11:25.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.301+0000 7f4e5fe00640 1 -- start start 2024-03-12T17:11:25.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.301+0000 7f4e5fe00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e600cf5c0 0x7f4e600c6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:25.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.301+0000 7f4e5fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e600d0c20 0x7f4e6015e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:25.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.301+0000 7f4e5fe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e6015cab0 0x7f4e6015e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:25.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.301+0000 7f4e5fe00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e600c9190 con 0x7f4e6015cab0 2024-03-12T17:11:25.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.301+0000 7f4e5fe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e601f3240 con 0x7f4e600d0c20 2024-03-12T17:11:25.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.301+0000 7f4e5fe00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e600c8bd0 con 0x7f4e600cf5c0 2024-03-12T17:11:25.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.302+0000 7f4e5f400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e6015cab0 0x7f4e6015e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:25.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.302+0000 7f4e5f400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e6015cab0 0x7f4e6015e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36110/0 (socket says 172.21.15.52:36110) 2024-03-12T17:11:25.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.302+0000 7f4e5f400640 1 -- 172.21.15.52:0/3632119198 learned_addr learned my addr 172.21.15.52:0/3632119198 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:25.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.302+0000 7f4e5ea00640 1 --2- 172.21.15.52:0/3632119198 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e600cf5c0 0x7f4e600c6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:25.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.302+0000 7f4e5e000640 1 --2- 172.21.15.52:0/3632119198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e600d0c20 0x7f4e6015e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:25.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.302+0000 7f4e5f400640 1 -- 172.21.15.52:0/3632119198 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e600cf5c0 msgr2=0x7f4e600c6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:25.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.302+0000 7f4e5f400640 1 --2- 172.21.15.52:0/3632119198 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e600cf5c0 0x7f4e600c6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:25.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.302+0000 7f4e5f400640 1 -- 172.21.15.52:0/3632119198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e600d0c20 msgr2=0x7f4e6015e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:25.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.302+0000 7f4e5f400640 1 --2- 172.21.15.52:0/3632119198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e600d0c20 0x7f4e6015e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:25.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.302+0000 7f4e5f400640 1 -- 172.21.15.52:0/3632119198 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e4c009a70 con 0x7f4e6015cab0 2024-03-12T17:11:25.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.302+0000 7f4e5e000640 1 --2- 172.21.15.52:0/3632119198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e600d0c20 0x7f4e6015e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:11:25.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.303+0000 7f4e5f400640 1 --2- 172.21.15.52:0/3632119198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e6015cab0 0x7f4e6015e680 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4e5800c820 tx=0x7f4e5800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:25.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.303+0000 7f4e56a00640 1 -- 172.21.15.52:0/3632119198 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e58007a30 con 0x7f4e6015cab0 2024-03-12T17:11:25.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.303+0000 7f4e56a00640 1 -- 172.21.15.52:0/3632119198 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e58007b90 con 0x7f4e6015cab0 2024-03-12T17:11:25.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.303+0000 7f4e5fe00640 1 -- 172.21.15.52:0/3632119198 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e600c8eb0 con 0x7f4e6015cab0 2024-03-12T17:11:25.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.303+0000 7f4e56a00640 1 -- 172.21.15.52:0/3632119198 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5800ce10 con 0x7f4e6015cab0 2024-03-12T17:11:25.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.303+0000 7f4e5fe00640 1 -- 172.21.15.52:0/3632119198 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e600c7120 con 0x7f4e6015cab0 2024-03-12T17:11:25.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.304+0000 7f4e5fe00640 1 -- 172.21.15.52:0/3632119198 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e6004e030 con 0x7f4e6015cab0 2024-03-12T17:11:25.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.305+0000 7f4e56a00640 1 -- 172.21.15.52:0/3632119198 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4e580113f0 con 0x7f4e6015cab0 2024-03-12T17:11:25.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.306+0000 7f4e56a00640 1 --2- 172.21.15.52:0/3632119198 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4e4006ac00 0x7f4e4006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:25.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.306+0000 7f4e56a00640 1 -- 172.21.15.52:0/3632119198 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4e58089640 con 0x7f4e6015cab0 2024-03-12T17:11:25.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.307+0000 7f4e5ea00640 1 --2- 172.21.15.52:0/3632119198 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4e4006ac00 0x7f4e4006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:25.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.308+0000 7f4e5ea00640 1 --2- 172.21.15.52:0/3632119198 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4e4006ac00 0x7f4e4006d0c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4e50000f60 tx=0x7f4e50007c00 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:25.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.311+0000 7f4e56a00640 1 -- 172.21.15.52:0/3632119198 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f4e580546b0 con 0x7f4e6015cab0 2024-03-12T17:11:25.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.509+0000 7f4e5fe00640 1 -- 172.21.15.52:0/3632119198 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4e601f3380 con 0x7f4e6015cab0 2024-03-12T17:11:25.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.510+0000 7f4e56a00640 1 -- 172.21.15.52:0/3632119198 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4e58057cd0 con 0x7f4e6015cab0 2024-03-12T17:11:25.505 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-12T17:11:25.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.513+0000 7f4e5fe00640 1 -- 172.21.15.52:0/3632119198 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4e4006ac00 msgr2=0x7f4e4006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:25.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.513+0000 7f4e5fe00640 1 --2- 172.21.15.52:0/3632119198 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4e4006ac00 0x7f4e4006d0c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4e50000f60 tx=0x7f4e50007c00 comp rx=0 tx=0).stop 2024-03-12T17:11:25.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.513+0000 7f4e5fe00640 1 -- 172.21.15.52:0/3632119198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e6015cab0 msgr2=0x7f4e6015e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:25.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.513+0000 7f4e5fe00640 1 --2- 172.21.15.52:0/3632119198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e6015cab0 0x7f4e6015e680 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4e5800c820 tx=0x7f4e5800cce0 comp rx=0 tx=0).stop 2024-03-12T17:11:25.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.513+0000 7f4e5fe00640 1 -- 172.21.15.52:0/3632119198 shutdown_connections 2024-03-12T17:11:25.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.513+0000 7f4e5fe00640 1 --2- 172.21.15.52:0/3632119198 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f4e4006ac00 0x7f4e4006d0c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:25.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.513+0000 7f4e5fe00640 1 --2- 172.21.15.52:0/3632119198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e6015cab0 0x7f4e6015e680 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:25.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.513+0000 7f4e5fe00640 1 --2- 172.21.15.52:0/3632119198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e600d0c20 0x7f4e6015e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:25.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.513+0000 7f4e5fe00640 1 --2- 172.21.15.52:0/3632119198 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e600cf5c0 0x7f4e600c6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:25.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.513+0000 7f4e5fe00640 1 -- 172.21.15.52:0/3632119198 >> 172.21.15.52:0/3632119198 conn(0x7f4e6006ae90 msgr2=0x7f4e600b5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:25.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.513+0000 7f4e5fe00640 1 -- 172.21.15.52:0/3632119198 shutdown_connections 2024-03-12T17:11:25.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.513+0000 7f4e5fe00640 1 -- 172.21.15.52:0/3632119198 wait complete. 2024-03-12T17:11:25.528 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-12T17:11:25.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.757+0000 7ff57e200640 1 Processor -- start 2024-03-12T17:11:25.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.758+0000 7ff57e200640 1 -- start start 2024-03-12T17:11:25.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.759+0000 7ff57e200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff5780b84b0 0x7ff5780d0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:25.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.759+0000 7ff57e200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff5780b5aa0 0x7ff5780b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:25.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.759+0000 7ff57e200640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5780d1280 con 0x7ff5780cd740 2024-03-12T17:11:25.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.759+0000 7ff57e200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5780d20c0 con 0x7ff5780b84b0 2024-03-12T17:11:25.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.759+0000 7ff57e200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5780b8940 con 0x7ff5780b5aa0 2024-03-12T17:11:25.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.759+0000 7ff577400640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7ff5780cd740 0x7ff5780cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:40078/0 (socket says 172.21.15.52:40078) 2024-03-12T17:11:25.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.759+0000 7ff577400640 1 -- 172.21.15.52:0/447268122 learned_addr learned my addr 172.21.15.52:0/447268122 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:11:25.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.759+0000 7ff576a00640 1 --2- 172.21.15.52:0/447268122 >> v2:172.21.15.52:3300/0 conn(0x7ff5780b84b0 0x7ff5780d0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:25.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.760+0000 7ff577e00640 1 --2- 172.21.15.52:0/447268122 >> v2:172.21.15.102:3300/0 conn(0x7ff5780b5aa0 0x7ff5780b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:25.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.760+0000 7ff576a00640 1 -- 172.21.15.52:0/447268122 >> v2:172.21.15.102:3300/0 conn(0x7ff5780b5aa0 msgr2=0x7ff5780b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:25.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.760+0000 7ff576a00640 1 --2- 172.21.15.52:0/447268122 >> v2:172.21.15.102:3300/0 conn(0x7ff5780b5aa0 0x7ff5780b5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:25.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.760+0000 7ff576a00640 1 -- 172.21.15.52:0/447268122 >> v1:172.21.15.52:6789/0 conn(0x7ff5780cd740 legacy=0x7ff5780cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:25.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.760+0000 7ff576a00640 1 -- 172.21.15.52:0/447268122 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5780d14b0 con 0x7ff5780b84b0 2024-03-12T17:11:25.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.761+0000 7ff576a00640 1 --2- 172.21.15.52:0/447268122 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5780b84b0 0x7ff5780d0d40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff5780c99c0 tx=0x7ff56002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb9989206bca592 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:25.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.761+0000 7ff576000640 1 -- 172.21.15.52:0/447268122 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff560031830 con 0x7ff5780b84b0 2024-03-12T17:11:25.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.762+0000 7ff576000640 1 -- 172.21.15.52:0/447268122 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff560031990 con 0x7ff5780b84b0 2024-03-12T17:11:25.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.762+0000 7ff576000640 1 -- 172.21.15.52:0/447268122 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff560031d20 con 0x7ff5780b84b0 2024-03-12T17:11:25.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.762+0000 7ff57e200640 1 -- 172.21.15.52:0/447268122 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5780b84b0 msgr2=0x7ff5780d0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:25.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.762+0000 7ff57e200640 1 --2- 172.21.15.52:0/447268122 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5780b84b0 0x7ff5780d0d40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff5780c99c0 tx=0x7ff56002b8d0 comp rx=0 tx=0).stop 2024-03-12T17:11:25.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.762+0000 7ff57e200640 1 -- 172.21.15.52:0/447268122 shutdown_connections 2024-03-12T17:11:25.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.762+0000 7ff57e200640 1 --2- 172.21.15.52:0/447268122 >> v2:172.21.15.102:3300/0 conn(0x7ff5780b5aa0 0x7ff5780b5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:25.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.762+0000 7ff57e200640 1 --2- 172.21.15.52:0/447268122 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5780b84b0 0x7ff5780d0d40 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:25.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.762+0000 7ff57e200640 1 -- 172.21.15.52:0/447268122 >> 172.21.15.52:0/447268122 conn(0x7ff5780cf5d0 msgr2=0x7ff5780cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:25.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.763+0000 7ff57e200640 1 -- 172.21.15.52:0/447268122 shutdown_connections 2024-03-12T17:11:25.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.763+0000 7ff57e200640 1 -- 172.21.15.52:0/447268122 wait complete. 2024-03-12T17:11:25.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.765+0000 7ff57e200640 1 Processor -- start 2024-03-12T17:11:25.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.765+0000 7ff57e200640 1 -- start start 2024-03-12T17:11:25.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.766+0000 7ff57e200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff5780b5aa0 0x7ff5781f3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:25.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.766+0000 7ff57e200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5780b84b0 0x7ff578152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:25.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.766+0000 7ff57e200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5780cd740 0x7ff578154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:25.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.766+0000 7ff57e200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5781545d0 con 0x7ff5780cd740 2024-03-12T17:11:25.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.766+0000 7ff57e200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff578154870 con 0x7ff5780b84b0 2024-03-12T17:11:25.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.766+0000 7ff57e200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5780b33c0 con 0x7ff5780b5aa0 2024-03-12T17:11:25.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.767+0000 7ff577e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5780cd740 0x7ff578154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:25.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.767+0000 7ff577e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5780cd740 0x7ff578154090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36128/0 (socket says 172.21.15.52:36128) 2024-03-12T17:11:25.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.767+0000 7ff577e00640 1 -- 172.21.15.52:0/3979078729 learned_addr learned my addr 172.21.15.52:0/3979078729 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:25.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.767+0000 7ff576a00640 1 --2- 172.21.15.52:0/3979078729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5780b84b0 0x7ff578152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:25.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.767+0000 7ff577400640 1 --2- 172.21.15.52:0/3979078729 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff5780b5aa0 0x7ff5781f3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:25.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.767+0000 7ff577e00640 1 -- 172.21.15.52:0/3979078729 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff5780b5aa0 msgr2=0x7ff5781f3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:25.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.767+0000 7ff577e00640 1 --2- 172.21.15.52:0/3979078729 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff5780b5aa0 0x7ff5781f3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:25.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.767+0000 7ff577e00640 1 -- 172.21.15.52:0/3979078729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5780b84b0 msgr2=0x7ff578152ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:25.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.767+0000 7ff577e00640 1 --2- 172.21.15.52:0/3979078729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5780b84b0 0x7ff578152ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:25.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.767+0000 7ff577e00640 1 -- 172.21.15.52:0/3979078729 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff560009400 con 0x7ff5780cd740 2024-03-12T17:11:25.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.768+0000 7ff577e00640 1 --2- 172.21.15.52:0/3979078729 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5780cd740 0x7ff578154090 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff57000d800 tx=0x7ff57000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:25.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.768+0000 7ff56f400640 1 -- 172.21.15.52:0/3979078729 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57000b6c0 con 0x7ff5780cd740 2024-03-12T17:11:25.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.768+0000 7ff56f400640 1 -- 172.21.15.52:0/3979078729 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff57000b840 con 0x7ff5780cd740 2024-03-12T17:11:25.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.768+0000 7ff56f400640 1 -- 172.21.15.52:0/3979078729 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57000bbf0 con 0x7ff5780cd740 2024-03-12T17:11:25.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.768+0000 7ff57e200640 1 -- 172.21.15.52:0/3979078729 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5780b36a0 con 0x7ff5780cd740 2024-03-12T17:11:25.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.768+0000 7ff57e200640 1 -- 172.21.15.52:0/3979078729 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff578154b50 con 0x7ff5780cd740 2024-03-12T17:11:25.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.769+0000 7ff57e200640 1 -- 172.21.15.52:0/3979078729 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff578066500 con 0x7ff5780cd740 2024-03-12T17:11:25.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.770+0000 7ff56f400640 1 -- 172.21.15.52:0/3979078729 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff57000bd50 con 0x7ff5780cd740 2024-03-12T17:11:25.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.771+0000 7ff56f400640 1 --2- 172.21.15.52:0/3979078729 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff55406ac30 0x7ff55406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:25.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.771+0000 7ff56f400640 1 -- 172.21.15.52:0/3979078729 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff5700899d0 con 0x7ff5780cd740 2024-03-12T17:11:25.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.772+0000 7ff577400640 1 --2- 172.21.15.52:0/3979078729 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff55406ac30 0x7ff55406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:25.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.773+0000 7ff577400640 1 --2- 172.21.15.52:0/3979078729 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff55406ac30 0x7ff55406d0f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff568005c00 tx=0x7ff568009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:25.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.776+0000 7ff56f400640 1 -- 172.21.15.52:0/3979078729 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7ff570054a40 con 0x7ff5780cd740 2024-03-12T17:11:25.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.972+0000 7ff57e200640 1 -- 172.21.15.52:0/3979078729 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff578073390 con 0x7ff5780cd740 2024-03-12T17:11:25.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.973+0000 7ff56f400640 1 -- 172.21.15.52:0/3979078729 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff570058060 con 0x7ff5780cd740 2024-03-12T17:11:25.968 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-03-12T17:11:25.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.976+0000 7ff57e200640 1 -- 172.21.15.52:0/3979078729 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff55406ac30 msgr2=0x7ff55406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:25.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.976+0000 7ff57e200640 1 --2- 172.21.15.52:0/3979078729 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff55406ac30 0x7ff55406d0f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff568005c00 tx=0x7ff568009040 comp rx=0 tx=0).stop 2024-03-12T17:11:25.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.976+0000 7ff57e200640 1 -- 172.21.15.52:0/3979078729 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5780cd740 msgr2=0x7ff578154090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:25.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.976+0000 7ff57e200640 1 --2- 172.21.15.52:0/3979078729 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5780cd740 0x7ff578154090 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff57000d800 tx=0x7ff57000dcc0 comp rx=0 tx=0).stop 2024-03-12T17:11:25.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.976+0000 7ff57e200640 1 -- 172.21.15.52:0/3979078729 shutdown_connections 2024-03-12T17:11:25.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.976+0000 7ff57e200640 1 --2- 172.21.15.52:0/3979078729 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff55406ac30 0x7ff55406d0f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:25.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.976+0000 7ff57e200640 1 --2- 172.21.15.52:0/3979078729 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5780cd740 0x7ff578154090 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:25.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.976+0000 7ff57e200640 1 --2- 172.21.15.52:0/3979078729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5780b84b0 0x7ff578152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:25.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.976+0000 7ff57e200640 1 --2- 172.21.15.52:0/3979078729 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff5780b5aa0 0x7ff5781f3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:25.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.976+0000 7ff57e200640 1 -- 172.21.15.52:0/3979078729 >> 172.21.15.52:0/3979078729 conn(0x7ff5780cf5d0 msgr2=0x7ff57815c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:25.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.976+0000 7ff57e200640 1 -- 172.21.15.52:0/3979078729 shutdown_connections 2024-03-12T17:11:25.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:25.976+0000 7ff57e200640 1 -- 172.21.15.52:0/3979078729 wait complete. 2024-03-12T17:11:25.993 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-12T17:11:26.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.221+0000 7f14a8200640 1 Processor -- start 2024-03-12T17:11:26.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.222+0000 7f14a8200640 1 -- start start 2024-03-12T17:11:26.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.223+0000 7f14a8200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f14a00b5a30 0x7f14a00b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:26.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.223+0000 7f14a8200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f14a00b4350 0x7f14a00b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:26.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.223+0000 7f14a8200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14a00d1830 con 0x7f14a015cda0 2024-03-12T17:11:26.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.223+0000 7f14a8200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14a00b4d20 con 0x7f14a00b5a30 2024-03-12T17:11:26.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.223+0000 7f14a8200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14a00b1f20 con 0x7f14a00b4350 2024-03-12T17:11:26.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.224+0000 7f14a6200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f14a00b4350 0x7f14a00b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:26.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.224+0000 7f14a5800640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f14a015cda0 0x7f14a00cdb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.52:47586/0 (socket says 172.21.15.52:47586) 2024-03-12T17:11:26.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.224+0000 7f14a5800640 1 -- 172.21.15.52:0/1180078054 learned_addr learned my addr 172.21.15.52:0/1180078054 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:11:26.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.224+0000 7f14a4e00640 1 --2- 172.21.15.52:0/1180078054 >> v2:172.21.15.102:3300/0 conn(0x7f14a00b5a30 0x7f14a00b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:26.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.225+0000 7f149fe00640 1 -- 172.21.15.52:0/1180078054 <== mon.1 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1224818645 0 0) 0x7f14a00d1830 con 0x7f14a015cda0 2024-03-12T17:11:26.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.225+0000 7f149fe00640 1 -- 172.21.15.52:0/1180078054 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1488003520 con 0x7f14a015cda0 2024-03-12T17:11:26.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.225+0000 7f14a6200640 1 -- 172.21.15.52:0/1180078054 >> v2:172.21.15.102:3300/0 conn(0x7f14a00b5a30 msgr2=0x7f14a00b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:26.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.225+0000 7f14a6200640 1 --2- 172.21.15.52:0/1180078054 >> v2:172.21.15.102:3300/0 conn(0x7f14a00b5a30 0x7f14a00b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:26.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.225+0000 7f14a6200640 1 -- 172.21.15.52:0/1180078054 >> v1:172.21.15.102:6789/0 conn(0x7f14a015cda0 legacy=0x7f14a00cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:26.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.225+0000 7f14a6200640 1 -- 172.21.15.52:0/1180078054 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a00d0c20 con 0x7f14a00b4350 2024-03-12T17:11:26.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.226+0000 7f14a6200640 1 --2- 172.21.15.52:0/1180078054 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f14a00b4350 0x7f14a00b47e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f14980097a0 tx=0x7f149802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c344258a5108060 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:26.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.227+0000 7f149fe00640 1 -- 172.21.15.52:0/1180078054 <== mon.2 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1498031830 con 0x7f14a00b4350 2024-03-12T17:11:26.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.227+0000 7f149fe00640 1 -- 172.21.15.52:0/1180078054 <== mon.2 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1498031990 con 0x7f14a00b4350 2024-03-12T17:11:26.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.227+0000 7f149fe00640 1 -- 172.21.15.52:0/1180078054 <== mon.2 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1498031d20 con 0x7f14a00b4350 2024-03-12T17:11:26.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.227+0000 7f14a8200640 1 -- 172.21.15.52:0/1180078054 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f14a00b4350 msgr2=0x7f14a00b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:26.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.227+0000 7f14a8200640 1 --2- 172.21.15.52:0/1180078054 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f14a00b4350 0x7f14a00b47e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f14980097a0 tx=0x7f149802b8d0 comp rx=0 tx=0).stop 2024-03-12T17:11:26.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.228+0000 7f14a8200640 1 -- 172.21.15.52:0/1180078054 shutdown_connections 2024-03-12T17:11:26.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.228+0000 7f14a8200640 1 --2- 172.21.15.52:0/1180078054 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f14a00b4350 0x7f14a00b47e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:26.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.228+0000 7f14a8200640 1 --2- 172.21.15.52:0/1180078054 >> v2:172.21.15.102:3300/0 conn(0x7f14a00b5a30 0x7f14a00b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:26.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.228+0000 7f14a8200640 1 -- 172.21.15.52:0/1180078054 >> 172.21.15.52:0/1180078054 conn(0x7f14a00cf350 msgr2=0x7f14a00cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:26.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.228+0000 7f14a8200640 1 -- 172.21.15.52:0/1180078054 shutdown_connections 2024-03-12T17:11:26.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.228+0000 7f14a8200640 1 -- 172.21.15.52:0/1180078054 wait complete. 2024-03-12T17:11:26.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.230+0000 7f14a8200640 1 Processor -- start 2024-03-12T17:11:26.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.230+0000 7f14a8200640 1 -- start start 2024-03-12T17:11:26.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.231+0000 7f14a8200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14a00b4350 0x7f14a00c6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:26.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.231+0000 7f14a8200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f14a00b5a30 0x7f14a00c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:26.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.231+0000 7f14a8200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f14a015cda0 0x7f14a01590d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:26.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.231+0000 7f14a8200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a00c82a0 con 0x7f14a00b4350 2024-03-12T17:11:26.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.231+0000 7f14a8200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a00c7350 con 0x7f14a00b5a30 2024-03-12T17:11:26.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.231+0000 7f14a8200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14a00c7580 con 0x7f14a015cda0 2024-03-12T17:11:26.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.231+0000 7f14a5800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14a00b4350 0x7f14a00c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:26.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.231+0000 7f14a5800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14a00b4350 0x7f14a00c6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36132/0 (socket says 172.21.15.52:36132) 2024-03-12T17:11:26.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.231+0000 7f14a5800640 1 -- 172.21.15.52:0/1399954475 learned_addr learned my addr 172.21.15.52:0/1399954475 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:26.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.232+0000 7f14a6200640 1 --2- 172.21.15.52:0/1399954475 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f14a015cda0 0x7f14a01590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:26.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.232+0000 7f14a4e00640 1 --2- 172.21.15.52:0/1399954475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f14a00b5a30 0x7f14a00c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:26.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.232+0000 7f14a5800640 1 -- 172.21.15.52:0/1399954475 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f14a015cda0 msgr2=0x7f14a01590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:26.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.232+0000 7f14a5800640 1 --2- 172.21.15.52:0/1399954475 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f14a015cda0 0x7f14a01590d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:26.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.232+0000 7f14a5800640 1 -- 172.21.15.52:0/1399954475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f14a00b5a30 msgr2=0x7f14a00c6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:26.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.232+0000 7f14a5800640 1 --2- 172.21.15.52:0/1399954475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f14a00b5a30 0x7f14a00c6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:26.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.232+0000 7f14a5800640 1 -- 172.21.15.52:0/1399954475 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1498009400 con 0x7f14a00b4350 2024-03-12T17:11:26.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.232+0000 7f14a5800640 1 --2- 172.21.15.52:0/1399954475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14a00b4350 0x7f14a00c6690 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f149400c390 tx=0x7f149400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:26.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.232+0000 7f149d600640 1 -- 172.21.15.52:0/1399954475 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1494005ce0 con 0x7f14a00b4350 2024-03-12T17:11:26.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.232+0000 7f149d600640 1 -- 172.21.15.52:0/1399954475 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1494005e40 con 0x7f14a00b4350 2024-03-12T17:11:26.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.232+0000 7f149d600640 1 -- 172.21.15.52:0/1399954475 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14940072f0 con 0x7f14a00b4350 2024-03-12T17:11:26.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.232+0000 7f14a8200640 1 -- 172.21.15.52:0/1399954475 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14a015d340 con 0x7f14a00b4350 2024-03-12T17:11:26.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.233+0000 7f14a8200640 1 -- 172.21.15.52:0/1399954475 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a015d900 con 0x7f14a00b4350 2024-03-12T17:11:26.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.234+0000 7f14a8200640 1 -- 172.21.15.52:0/1399954475 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14a0066e30 con 0x7f14a00b4350 2024-03-12T17:11:26.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.235+0000 7f149d600640 1 -- 172.21.15.52:0/1399954475 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1494007450 con 0x7f14a00b4350 2024-03-12T17:11:26.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.236+0000 7f149d600640 1 --2- 172.21.15.52:0/1399954475 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f148406ac00 0x7f148406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:26.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.236+0000 7f149d600640 1 -- 172.21.15.52:0/1399954475 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1494088a60 con 0x7f14a00b4350 2024-03-12T17:11:26.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.237+0000 7f14a4e00640 1 --2- 172.21.15.52:0/1399954475 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f148406ac00 0x7f148406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:26.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.238+0000 7f14a4e00640 1 --2- 172.21.15.52:0/1399954475 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f148406ac00 0x7f148406d0c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f14a00b5650 tx=0x7f149000b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:26.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.242+0000 7f149d600640 1 -- 172.21.15.52:0/1399954475 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f1494053ad0 con 0x7f14a00b4350 2024-03-12T17:11:26.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.459+0000 7f14a8200640 1 -- 172.21.15.52:0/1399954475 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f14a0073390 con 0x7f14a00b4350 2024-03-12T17:11:26.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.460+0000 7f149d600640 1 -- 172.21.15.52:0/1399954475 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1494017020 con 0x7f14a00b4350 2024-03-12T17:11:26.455 INFO:teuthology.orchestra.run.smithi052.stderr:listed 0 entries 2024-03-12T17:11:26.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.463+0000 7f14a8200640 1 -- 172.21.15.52:0/1399954475 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f148406ac00 msgr2=0x7f148406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:26.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.463+0000 7f14a8200640 1 --2- 172.21.15.52:0/1399954475 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f148406ac00 0x7f148406d0c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f14a00b5650 tx=0x7f149000b040 comp rx=0 tx=0).stop 2024-03-12T17:11:26.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.463+0000 7f14a8200640 1 -- 172.21.15.52:0/1399954475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14a00b4350 msgr2=0x7f14a00c6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:26.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.463+0000 7f14a8200640 1 --2- 172.21.15.52:0/1399954475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14a00b4350 0x7f14a00c6690 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f149400c390 tx=0x7f149400c850 comp rx=0 tx=0).stop 2024-03-12T17:11:26.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.463+0000 7f14a8200640 1 -- 172.21.15.52:0/1399954475 shutdown_connections 2024-03-12T17:11:26.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.463+0000 7f14a8200640 1 --2- 172.21.15.52:0/1399954475 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f148406ac00 0x7f148406d0c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:26.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.463+0000 7f14a8200640 1 --2- 172.21.15.52:0/1399954475 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f14a015cda0 0x7f14a01590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:26.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.463+0000 7f14a8200640 1 --2- 172.21.15.52:0/1399954475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f14a00b5a30 0x7f14a00c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:26.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.463+0000 7f14a8200640 1 --2- 172.21.15.52:0/1399954475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14a00b4350 0x7f14a00c6690 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:26.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.463+0000 7f14a8200640 1 -- 172.21.15.52:0/1399954475 >> 172.21.15.52:0/1399954475 conn(0x7f14a00cf350 msgr2=0x7f14a00b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:26.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.463+0000 7f14a8200640 1 -- 172.21.15.52:0/1399954475 shutdown_connections 2024-03-12T17:11:26.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:26.463+0000 7f14a8200640 1 -- 172.21.15.52:0/1399954475 wait complete. 2024-03-12T17:11:26.477 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-12T17:11:26.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:11:26.477 DEBUG:teuthology.orchestra.run.smithi052:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-12T17:11:26.583 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi052.front.sepia.ceph.com 2024-03-12T17:11:26.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:11:26.583 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-12T17:11:26.637 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T17:11:26.637 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-12T17:11:26.695 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053427,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-12T17:11:26.695 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:11:26.695 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-12T17:11:26.755 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054202,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-12T17:11:26.755 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T17:11:26.755 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-12T17:11:26.811 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053427,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-12T17:11:26.811 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T17:11:27.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.048+0000 7f6199200640 1 Processor -- start 2024-03-12T17:11:27.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.048+0000 7f6199200640 1 -- start start 2024-03-12T17:11:27.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.049+0000 7f6199200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f619415c690 0x7f61940d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:27.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.049+0000 7f6199200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f619415ca70 0x7f61940cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:27.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.049+0000 7f6199200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61940d1760 con 0x7f61940d2e10 2024-03-12T17:11:27.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.049+0000 7f6199200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61940d0b50 con 0x7f619415ca70 2024-03-12T17:11:27.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.049+0000 7f6199200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61940d0e30 con 0x7f619415c690 2024-03-12T17:11:27.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.050+0000 7f618aa00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f619415ca70 0x7f61940cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:27.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.050+0000 7f618aa00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f619415ca70 0x7f61940cd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36134/0 (socket says 172.21.15.52:36134) 2024-03-12T17:11:27.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.050+0000 7f618aa00640 1 -- 172.21.15.52:0/2018145624 learned_addr learned my addr 172.21.15.52:0/2018145624 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:27.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.050+0000 7f6193400640 1 --2- 172.21.15.52:0/2018145624 >> v2:172.21.15.102:3300/0 conn(0x7f619415c690 0x7f61940d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:27.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.050+0000 7f618aa00640 1 -- 172.21.15.52:0/2018145624 >> v2:172.21.15.102:3300/0 conn(0x7f619415c690 msgr2=0x7f61940d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:27.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.050+0000 7f618aa00640 1 --2- 172.21.15.52:0/2018145624 >> v2:172.21.15.102:3300/0 conn(0x7f619415c690 0x7f61940d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:27.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.050+0000 7f618aa00640 1 -- 172.21.15.52:0/2018145624 >> v1:172.21.15.102:6789/0 conn(0x7f61940d2e10 legacy=0x7f61940b41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:27.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.050+0000 7f618aa00640 1 -- 172.21.15.52:0/2018145624 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61940d1110 con 0x7f619415ca70 2024-03-12T17:11:27.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.052+0000 7f618aa00640 1 --2- 172.21.15.52:0/2018145624 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f619415ca70 0x7f61940cd0b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f61940cf110 tx=0x7f617c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c6b6a7a2e920a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:27.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.052+0000 7f6192a00640 1 -- 172.21.15.52:0/2018145624 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c031870 con 0x7f619415ca70 2024-03-12T17:11:27.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.052+0000 7f6192a00640 1 -- 172.21.15.52:0/2018145624 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f617c0319d0 con 0x7f619415ca70 2024-03-12T17:11:27.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.052+0000 7f6192a00640 1 -- 172.21.15.52:0/2018145624 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c031d60 con 0x7f619415ca70 2024-03-12T17:11:27.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.053+0000 7f6199200640 1 -- 172.21.15.52:0/2018145624 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f619415ca70 msgr2=0x7f61940cd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:27.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.053+0000 7f6199200640 1 --2- 172.21.15.52:0/2018145624 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f619415ca70 0x7f61940cd0b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f61940cf110 tx=0x7f617c02b910 comp rx=0 tx=0).stop 2024-03-12T17:11:27.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.053+0000 7f6199200640 1 -- 172.21.15.52:0/2018145624 shutdown_connections 2024-03-12T17:11:27.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.053+0000 7f6199200640 1 --2- 172.21.15.52:0/2018145624 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f619415ca70 0x7f61940cd0b0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:27.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.053+0000 7f6199200640 1 --2- 172.21.15.52:0/2018145624 >> v2:172.21.15.102:3300/0 conn(0x7f619415c690 0x7f61940d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:27.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.053+0000 7f6199200640 1 -- 172.21.15.52:0/2018145624 >> 172.21.15.52:0/2018145624 conn(0x7f61940717b0 msgr2=0x7f6194071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:27.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.053+0000 7f6199200640 1 -- 172.21.15.52:0/2018145624 shutdown_connections 2024-03-12T17:11:27.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.053+0000 7f6199200640 1 -- 172.21.15.52:0/2018145624 wait complete. 2024-03-12T17:11:27.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.055+0000 7f6199200640 1 Processor -- start 2024-03-12T17:11:27.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.056+0000 7f6199200640 1 -- start start 2024-03-12T17:11:27.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.057+0000 7f6199200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61940d2e10 0x7f61941f1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:27.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.057+0000 7f6199200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f619415c690 0x7f61940b49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:27.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.057+0000 7f6199200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f619415ca70 0x7f6194150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:27.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.057+0000 7f6199200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61940b2190 con 0x7f619415ca70 2024-03-12T17:11:27.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.057+0000 7f6199200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61940b23e0 con 0x7f61940d2e10 2024-03-12T17:11:27.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.057+0000 7f6199200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61940b26c0 con 0x7f619415c690 2024-03-12T17:11:27.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.057+0000 7f6193e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f619415ca70 0x7f6194150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:27.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.057+0000 7f6193e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f619415ca70 0x7f6194150e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36150/0 (socket says 172.21.15.52:36150) 2024-03-12T17:11:27.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.057+0000 7f6193e00640 1 -- 172.21.15.52:0/3711719173 learned_addr learned my addr 172.21.15.52:0/3711719173 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:27.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.057+0000 7f6193400640 1 --2- 172.21.15.52:0/3711719173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61940d2e10 0x7f61941f1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:27.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.058+0000 7f618aa00640 1 --2- 172.21.15.52:0/3711719173 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f619415c690 0x7f61940b49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:27.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.058+0000 7f6193e00640 1 -- 172.21.15.52:0/3711719173 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f619415c690 msgr2=0x7f61940b49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:27.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.058+0000 7f6193e00640 1 --2- 172.21.15.52:0/3711719173 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f619415c690 0x7f61940b49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:27.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.058+0000 7f6193e00640 1 -- 172.21.15.52:0/3711719173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61940d2e10 msgr2=0x7f61941f1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:27.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.058+0000 7f6193e00640 1 --2- 172.21.15.52:0/3711719173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61940d2e10 0x7f61941f1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:27.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.058+0000 7f6193e00640 1 -- 172.21.15.52:0/3711719173 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f617c0094d0 con 0x7f619415ca70 2024-03-12T17:11:27.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.058+0000 7f618aa00640 1 --2- 172.21.15.52:0/3711719173 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f619415c690 0x7f61940b49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:11:27.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.058+0000 7f6193e00640 1 --2- 172.21.15.52:0/3711719173 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f619415ca70 0x7f6194150e80 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f618c00b970 tx=0x7f618c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:27.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.058+0000 7f618be00640 1 -- 172.21.15.52:0/3711719173 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618c00cb40 con 0x7f619415ca70 2024-03-12T17:11:27.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.058+0000 7f618be00640 1 -- 172.21.15.52:0/3711719173 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618c00cca0 con 0x7f619415ca70 2024-03-12T17:11:27.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.058+0000 7f618be00640 1 -- 172.21.15.52:0/3711719173 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618c014680 con 0x7f619415ca70 2024-03-12T17:11:27.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.058+0000 7f6199200640 1 -- 172.21.15.52:0/3711719173 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61940b29a0 con 0x7f619415ca70 2024-03-12T17:11:27.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.059+0000 7f6199200640 1 -- 172.21.15.52:0/3711719173 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61940b4ef0 con 0x7f619415ca70 2024-03-12T17:11:27.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.060+0000 7f6199200640 1 -- 172.21.15.52:0/3711719173 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61940b2ec0 con 0x7f619415ca70 2024-03-12T17:11:27.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.061+0000 7f618be00640 1 -- 172.21.15.52:0/3711719173 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f618c01d3b0 con 0x7f619415ca70 2024-03-12T17:11:27.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.062+0000 7f618be00640 1 --2- 172.21.15.52:0/3711719173 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f617006ac30 0x7f617006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:27.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.062+0000 7f618be00640 1 -- 172.21.15.52:0/3711719173 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f618c089370 con 0x7f619415ca70 2024-03-12T17:11:27.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.062+0000 7f6193400640 1 --2- 172.21.15.52:0/3711719173 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f617006ac30 0x7f617006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:27.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.063+0000 7f6193400640 1 --2- 172.21.15.52:0/3711719173 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f617006ac30 0x7f617006d0f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6184006fd0 tx=0x7f6184008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:27.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.067+0000 7f618be00640 1 -- 172.21.15.52:0/3711719173 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f61940b2ec0 con 0x7f619415ca70 2024-03-12T17:11:27.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.309+0000 7f6199200640 1 -- 172.21.15.52:0/3711719173 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61940b2ec0 con 0x7f619415ca70 2024-03-12T17:11:27.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.310+0000 7f618be00640 1 -- 172.21.15.52:0/3711719173 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4734 (secure 0 0 0) 0x7f61940b2ec0 con 0x7f619415ca70 2024-03-12T17:11:27.305 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:11:27.305 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6836/285561331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":285561331},{"type":"v1","addr":"172.21.15.102:6836","nonce":285561331}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4909,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1312095655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6835","nonce":1312095655},{"type":"v1","addr":"172.21.15.102:6837","nonce":1312095655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5337,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2451989690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2451989690},{"type":"v1","addr":"172.21.15.52:6833","nonce":2451989690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-12T17:08:10.679436+0000","modified":"2024-03-12T17:08:14.140830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5343},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5343":{"gid":5343,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.52:6835/543567396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":543567396},{"type":"v1","addr":"172.21.15.52:6835","nonce":543567396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-12T17:11:27.305 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 17 2024-03-12T17:11:27.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.313+0000 7f6199200640 1 -- 172.21.15.52:0/3711719173 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f617006ac30 msgr2=0x7f617006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:27.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.313+0000 7f6199200640 1 --2- 172.21.15.52:0/3711719173 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f617006ac30 0x7f617006d0f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6184006fd0 tx=0x7f6184008040 comp rx=0 tx=0).stop 2024-03-12T17:11:27.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.313+0000 7f6199200640 1 -- 172.21.15.52:0/3711719173 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f619415ca70 msgr2=0x7f6194150e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:27.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.313+0000 7f6199200640 1 --2- 172.21.15.52:0/3711719173 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f619415ca70 0x7f6194150e80 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f618c00b970 tx=0x7f618c00be30 comp rx=0 tx=0).stop 2024-03-12T17:11:27.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.313+0000 7f6199200640 1 -- 172.21.15.52:0/3711719173 shutdown_connections 2024-03-12T17:11:27.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.313+0000 7f6199200640 1 --2- 172.21.15.52:0/3711719173 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f617006ac30 0x7f617006d0f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:27.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.313+0000 7f6199200640 1 --2- 172.21.15.52:0/3711719173 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f619415c690 0x7f61940b49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:27.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.313+0000 7f6199200640 1 --2- 172.21.15.52:0/3711719173 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f619415ca70 0x7f6194150e80 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:27.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.313+0000 7f6199200640 1 --2- 172.21.15.52:0/3711719173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61940d2e10 0x7f61941f1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:27.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.313+0000 7f6199200640 1 -- 172.21.15.52:0/3711719173 >> 172.21.15.52:0/3711719173 conn(0x7f61940717b0 msgr2=0x7f61940c6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:27.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.314+0000 7f6199200640 1 -- 172.21.15.52:0/3711719173 shutdown_connections 2024-03-12T17:11:27.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.314+0000 7f6199200640 1 -- 172.21.15.52:0/3711719173 wait complete. 2024-03-12T17:11:27.326 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:11:27.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.544+0000 7fbe77200640 1 Processor -- start 2024-03-12T17:11:27.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.544+0000 7fbe77200640 1 -- start start 2024-03-12T17:11:27.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.546+0000 7fbe77200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fbe780cf5c0 0x7fbe780b8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:27.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.546+0000 7fbe77200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbe780d0c20 0x7fbe780d10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:27.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.546+0000 7fbe77200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe7815ce90 con 0x7fbe7815cab0 2024-03-12T17:11:27.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.546+0000 7fbe77200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe780cddf0 con 0x7fbe780cf5c0 2024-03-12T17:11:27.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.546+0000 7fbe77200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe780cd570 con 0x7fbe780d0c20 2024-03-12T17:11:27.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.546+0000 7fbe6fe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fbe780cf5c0 0x7fbe780b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:27.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.546+0000 7fbe6fe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fbe780cf5c0 0x7fbe780b8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36164/0 (socket says 172.21.15.52:36164) 2024-03-12T17:11:27.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.546+0000 7fbe6fe00640 1 -- 172.21.15.52:0/3795008178 learned_addr learned my addr 172.21.15.52:0/3795008178 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:27.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.546+0000 7fbe76800640 1 --2- 172.21.15.52:0/3795008178 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbe780d0c20 0x7fbe780d10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:27.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.546+0000 7fbe6fe00640 1 -- 172.21.15.52:0/3795008178 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbe780d0c20 msgr2=0x7fbe780d10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:27.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.547+0000 7fbe6fe00640 1 --2- 172.21.15.52:0/3795008178 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbe780d0c20 0x7fbe780d10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:27.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.547+0000 7fbe6fe00640 1 -- 172.21.15.52:0/3795008178 >> v1:172.21.15.102:6789/0 conn(0x7fbe7815cab0 legacy=0x7fbe780ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:27.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.547+0000 7fbe6fe00640 1 -- 172.21.15.52:0/3795008178 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe78161370 con 0x7fbe780cf5c0 2024-03-12T17:11:27.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.548+0000 7fbe6fe00640 1 --2- 172.21.15.52:0/3795008178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe780cf5c0 0x7fbe780b8050 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009870 tx=0x7fbe6002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b419dcdf4ba7258 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:27.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.548+0000 7fbe75400640 1 -- 172.21.15.52:0/3795008178 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe60031640 con 0x7fbe780cf5c0 2024-03-12T17:11:27.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.548+0000 7fbe75400640 1 -- 172.21.15.52:0/3795008178 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe600317a0 con 0x7fbe780cf5c0 2024-03-12T17:11:27.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.548+0000 7fbe75400640 1 -- 172.21.15.52:0/3795008178 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe60031b30 con 0x7fbe780cf5c0 2024-03-12T17:11:27.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.549+0000 7fbe77200640 1 -- 172.21.15.52:0/3795008178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe780cf5c0 msgr2=0x7fbe780b8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:27.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.549+0000 7fbe77200640 1 --2- 172.21.15.52:0/3795008178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe780cf5c0 0x7fbe780b8050 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009870 tx=0x7fbe6002b8f0 comp rx=0 tx=0).stop 2024-03-12T17:11:27.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.549+0000 7fbe77200640 1 -- 172.21.15.52:0/3795008178 shutdown_connections 2024-03-12T17:11:27.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.549+0000 7fbe77200640 1 --2- 172.21.15.52:0/3795008178 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbe780d0c20 0x7fbe780d10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:27.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.549+0000 7fbe77200640 1 --2- 172.21.15.52:0/3795008178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe780cf5c0 0x7fbe780b8050 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:27.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.549+0000 7fbe77200640 1 -- 172.21.15.52:0/3795008178 >> 172.21.15.52:0/3795008178 conn(0x7fbe7806ae90 msgr2=0x7fbe7806b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:27.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.549+0000 7fbe77200640 1 -- 172.21.15.52:0/3795008178 shutdown_connections 2024-03-12T17:11:27.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.549+0000 7fbe77200640 1 -- 172.21.15.52:0/3795008178 wait complete. 2024-03-12T17:11:27.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.551+0000 7fbe77200640 1 Processor -- start 2024-03-12T17:11:27.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.551+0000 7fbe77200640 1 -- start start 2024-03-12T17:11:27.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.552+0000 7fbe77200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe780cf5c0 0x7fbe780c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:27.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.552+0000 7fbe77200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbe780d0c20 0x7fbe781577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:27.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.552+0000 7fbe77200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe7815cab0 0x7fbe780b7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:27.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.552+0000 7fbe77200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe780c6710 con 0x7fbe7815cab0 2024-03-12T17:11:27.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.552+0000 7fbe77200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe780b2440 con 0x7fbe780cf5c0 2024-03-12T17:11:27.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.552+0000 7fbe77200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe780b2720 con 0x7fbe780d0c20 2024-03-12T17:11:27.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.552+0000 7fbe76800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe7815cab0 0x7fbe780b7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:27.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.552+0000 7fbe76800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe7815cab0 0x7fbe780b7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36172/0 (socket says 172.21.15.52:36172) 2024-03-12T17:11:27.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.552+0000 7fbe76800640 1 -- 172.21.15.52:0/3908416065 learned_addr learned my addr 172.21.15.52:0/3908416065 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:27.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.552+0000 7fbe6fe00640 1 --2- 172.21.15.52:0/3908416065 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbe780d0c20 0x7fbe781577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:27.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.553+0000 7fbe75e00640 1 --2- 172.21.15.52:0/3908416065 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe780cf5c0 0x7fbe780c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:27.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.553+0000 7fbe76800640 1 -- 172.21.15.52:0/3908416065 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbe780d0c20 msgr2=0x7fbe781577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:27.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.553+0000 7fbe76800640 1 --2- 172.21.15.52:0/3908416065 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbe780d0c20 0x7fbe781577a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:27.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.553+0000 7fbe76800640 1 -- 172.21.15.52:0/3908416065 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe780cf5c0 msgr2=0x7fbe780c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:27.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.553+0000 7fbe76800640 1 --2- 172.21.15.52:0/3908416065 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe780cf5c0 0x7fbe780c61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:27.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.553+0000 7fbe76800640 1 -- 172.21.15.52:0/3908416065 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe600094d0 con 0x7fbe7815cab0 2024-03-12T17:11:27.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.553+0000 7fbe6fe00640 1 --2- 172.21.15.52:0/3908416065 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbe780d0c20 0x7fbe781577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:11:27.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.553+0000 7fbe76800640 1 --2- 172.21.15.52:0/3908416065 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe7815cab0 0x7fbe780b7a60 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fbe7000cd70 tx=0x7fbe7000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:27.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.553+0000 7fbe6e000640 1 -- 172.21.15.52:0/3908416065 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe70003b90 con 0x7fbe7815cab0 2024-03-12T17:11:27.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.553+0000 7fbe6e000640 1 -- 172.21.15.52:0/3908416065 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe70003cf0 con 0x7fbe7815cab0 2024-03-12T17:11:27.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.553+0000 7fbe6e000640 1 -- 172.21.15.52:0/3908416065 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe70004240 con 0x7fbe7815cab0 2024-03-12T17:11:27.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.553+0000 7fbe77200640 1 -- 172.21.15.52:0/3908416065 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe780b2a00 con 0x7fbe7815cab0 2024-03-12T17:11:27.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.553+0000 7fbe77200640 1 -- 172.21.15.52:0/3908416065 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe780b2fc0 con 0x7fbe7815cab0 2024-03-12T17:11:27.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.555+0000 7fbe77200640 1 -- 172.21.15.52:0/3908416065 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe7804e030 con 0x7fbe7815cab0 2024-03-12T17:11:27.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.556+0000 7fbe6e000640 1 -- 172.21.15.52:0/3908416065 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fbe700043a0 con 0x7fbe7815cab0 2024-03-12T17:11:27.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.556+0000 7fbe6e000640 1 --2- 172.21.15.52:0/3908416065 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fbe5406ac00 0x7fbe5406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:27.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.556+0000 7fbe6e000640 1 -- 172.21.15.52:0/3908416065 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbe70012070 con 0x7fbe7815cab0 2024-03-12T17:11:27.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.557+0000 7fbe75e00640 1 --2- 172.21.15.52:0/3908416065 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fbe5406ac00 0x7fbe5406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:27.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.558+0000 7fbe75e00640 1 --2- 172.21.15.52:0/3908416065 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fbe5406ac00 0x7fbe5406d0c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbe68006fd0 tx=0x7fbe68008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:27.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.561+0000 7fbe6e000640 1 -- 172.21.15.52:0/3908416065 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fbe70054fa0 con 0x7fbe7815cab0 2024-03-12T17:11:27.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.756+0000 7fbe77200640 1 -- 172.21.15.52:0/3908416065 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe780b8680 con 0x7fbe7815cab0 2024-03-12T17:11:27.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.759+0000 7fbe6e000640 1 -- 172.21.15.52:0/3908416065 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+16298 (secure 0 0 0) 0x7fbe700585c0 con 0x7fbe7815cab0 2024-03-12T17:11:27.753 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:11:27.753 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":42,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:09:13.132892+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T17:06:19.218543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-12T17:08:09.114678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-12T17:08:09.691941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.733031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.775178+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.591522+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.820838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.829252+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.582481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.d","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:11:27.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.761+0000 7fbe77200640 1 -- 172.21.15.52:0/3908416065 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fbe5406ac00 msgr2=0x7fbe5406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:27.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.761+0000 7fbe77200640 1 --2- 172.21.15.52:0/3908416065 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fbe5406ac00 0x7fbe5406d0c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbe68006fd0 tx=0x7fbe68008040 comp rx=0 tx=0).stop 2024-03-12T17:11:27.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.761+0000 7fbe77200640 1 -- 172.21.15.52:0/3908416065 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe7815cab0 msgr2=0x7fbe780b7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:27.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.761+0000 7fbe77200640 1 --2- 172.21.15.52:0/3908416065 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe7815cab0 0x7fbe780b7a60 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fbe7000cd70 tx=0x7fbe7000a660 comp rx=0 tx=0).stop 2024-03-12T17:11:27.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.761+0000 7fbe77200640 1 -- 172.21.15.52:0/3908416065 shutdown_connections 2024-03-12T17:11:27.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.761+0000 7fbe77200640 1 --2- 172.21.15.52:0/3908416065 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fbe5406ac00 0x7fbe5406d0c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:27.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.761+0000 7fbe77200640 1 --2- 172.21.15.52:0/3908416065 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbe780d0c20 0x7fbe781577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:27.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.761+0000 7fbe77200640 1 --2- 172.21.15.52:0/3908416065 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe7815cab0 0x7fbe780b7a60 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:27.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.761+0000 7fbe77200640 1 --2- 172.21.15.52:0/3908416065 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe780cf5c0 0x7fbe780c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:27.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.761+0000 7fbe77200640 1 -- 172.21.15.52:0/3908416065 >> 172.21.15.52:0/3908416065 conn(0x7fbe7806ae90 msgr2=0x7fbe7806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:27.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.762+0000 7fbe77200640 1 -- 172.21.15.52:0/3908416065 shutdown_connections 2024-03-12T17:11:27.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:27.762+0000 7fbe77200640 1 -- 172.21.15.52:0/3908416065 wait complete. 2024-03-12T17:11:27.776 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T17:06:19.218543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-12T17:08:09.114678+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-12T17:08:09.691941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-12T17:11:27.776 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-12T17:11:27.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.004+0000 7f07d8a00640 1 Processor -- start 2024-03-12T17:11:27.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.004+0000 7f07d8a00640 1 -- start start 2024-03-12T17:11:28.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.006+0000 7f07d8a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f07d415c820 0x7f07d40cd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:28.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.006+0000 7f07d8a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f07d40cf7c0 0x7f07d40d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:28.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.006+0000 7f07d8a00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07d40ccd90 con 0x7f07d4160e90 2024-03-12T17:11:28.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.006+0000 7f07d8a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07d40cdd70 con 0x7f07d415c820 2024-03-12T17:11:28.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.006+0000 7f07d8a00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d40b7fc0 con 0x7f07d40cf7c0 2024-03-12T17:11:28.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.007+0000 7f07d3400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f07d415c820 0x7f07d40cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:28.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.007+0000 7f07d3400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f07d415c820 0x7f07d40cd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:55256/0 (socket says 172.21.15.52:55256) 2024-03-12T17:11:28.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.007+0000 7f07d3e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f07d40cf7c0 0x7f07d40d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:28.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.007+0000 7f07d3400640 1 -- 172.21.15.52:0/117240454 learned_addr learned my addr 172.21.15.52:0/117240454 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:28.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.007+0000 7f07d3400640 1 -- 172.21.15.52:0/117240454 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f07d40cf7c0 msgr2=0x7f07d40d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:28.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.007+0000 7f07d3400640 1 --2- 172.21.15.52:0/117240454 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f07d40cf7c0 0x7f07d40d0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:28.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.007+0000 7f07d3400640 1 -- 172.21.15.52:0/117240454 >> v1:172.21.15.102:6789/0 conn(0x7f07d4160e90 legacy=0x7f07d40cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:28.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.007+0000 7f07d3400640 1 -- 172.21.15.52:0/117240454 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d40b82a0 con 0x7f07d415c820 2024-03-12T17:11:28.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.009+0000 7f07d3400640 1 --2- 172.21.15.52:0/117240454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07d415c820 0x7f07d40cd830 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f07c4009870 tx=0x7f07c402ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e010df9b6750abc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:28.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.009+0000 7f07d2000640 1 -- 172.21.15.52:0/117240454 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c402f830 con 0x7f07d415c820 2024-03-12T17:11:28.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.009+0000 7f07d2000640 1 -- 172.21.15.52:0/117240454 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c402f990 con 0x7f07d415c820 2024-03-12T17:11:28.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.009+0000 7f07d2000640 1 -- 172.21.15.52:0/117240454 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c402fd20 con 0x7f07d415c820 2024-03-12T17:11:28.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.010+0000 7f07d8a00640 1 -- 172.21.15.52:0/117240454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07d415c820 msgr2=0x7f07d40cd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:28.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.010+0000 7f07d8a00640 1 --2- 172.21.15.52:0/117240454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07d415c820 0x7f07d40cd830 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f07c4009870 tx=0x7f07c402ba10 comp rx=0 tx=0).stop 2024-03-12T17:11:28.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.010+0000 7f07d8a00640 1 -- 172.21.15.52:0/117240454 shutdown_connections 2024-03-12T17:11:28.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.010+0000 7f07d8a00640 1 --2- 172.21.15.52:0/117240454 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f07d40cf7c0 0x7f07d40d0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:28.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.010+0000 7f07d8a00640 1 --2- 172.21.15.52:0/117240454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07d415c820 0x7f07d40cd830 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:28.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.010+0000 7f07d8a00640 1 -- 172.21.15.52:0/117240454 >> 172.21.15.52:0/117240454 conn(0x7f07d406b050 msgr2=0x7f07d406b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:28.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.010+0000 7f07d8a00640 1 -- 172.21.15.52:0/117240454 shutdown_connections 2024-03-12T17:11:28.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.010+0000 7f07d8a00640 1 -- 172.21.15.52:0/117240454 wait complete. 2024-03-12T17:11:28.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.012+0000 7f07d8a00640 1 Processor -- start 2024-03-12T17:11:28.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.012+0000 7f07d8a00640 1 -- start start 2024-03-12T17:11:28.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.013+0000 7f07d8a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f07d40cf7c0 0x7f07d414db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:28.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.013+0000 7f07d8a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07d415c820 0x7f07d415ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:28.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.013+0000 7f07d8a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07d4160e90 0x7f07d415ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:28.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.013+0000 7f07d8a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d4160390 con 0x7f07d415c820 2024-03-12T17:11:28.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.013+0000 7f07d8a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d4157fe0 con 0x7f07d4160e90 2024-03-12T17:11:28.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.013+0000 7f07d8a00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d415f480 con 0x7f07d40cf7c0 2024-03-12T17:11:28.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.013+0000 7f07d2a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07d415c820 0x7f07d415ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:28.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.013+0000 7f07d2a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07d415c820 0x7f07d415ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36176/0 (socket says 172.21.15.52:36176) 2024-03-12T17:11:28.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.013+0000 7f07d2a00640 1 -- 172.21.15.52:0/2713037750 learned_addr learned my addr 172.21.15.52:0/2713037750 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:28.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.014+0000 7f07d3400640 1 --2- 172.21.15.52:0/2713037750 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f07d40cf7c0 0x7f07d414db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:28.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.014+0000 7f07d3e00640 1 --2- 172.21.15.52:0/2713037750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07d4160e90 0x7f07d415ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:28.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.014+0000 7f07d2a00640 1 -- 172.21.15.52:0/2713037750 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f07d40cf7c0 msgr2=0x7f07d414db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:28.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.014+0000 7f07d2a00640 1 --2- 172.21.15.52:0/2713037750 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f07d40cf7c0 0x7f07d414db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:28.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.014+0000 7f07d2a00640 1 -- 172.21.15.52:0/2713037750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07d4160e90 msgr2=0x7f07d415ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:28.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.014+0000 7f07d2a00640 1 --2- 172.21.15.52:0/2713037750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07d4160e90 0x7f07d415ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:28.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.014+0000 7f07d2a00640 1 -- 172.21.15.52:0/2713037750 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c40094d0 con 0x7f07d415c820 2024-03-12T17:11:28.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.014+0000 7f07d2a00640 1 --2- 172.21.15.52:0/2713037750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07d415c820 0x7f07d415ea00 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f07bc00c3c0 tx=0x7f07bc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:28.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.015+0000 7f07cb400640 1 -- 172.21.15.52:0/2713037750 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07bc0052d0 con 0x7f07d415c820 2024-03-12T17:11:28.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.015+0000 7f07cb400640 1 -- 172.21.15.52:0/2713037750 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07bc005450 con 0x7f07d415c820 2024-03-12T17:11:28.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.015+0000 7f07cb400640 1 -- 172.21.15.52:0/2713037750 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07bc007260 con 0x7f07d415c820 2024-03-12T17:11:28.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.015+0000 7f07d8a00640 1 -- 172.21.15.52:0/2713037750 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07d415f710 con 0x7f07d415c820 2024-03-12T17:11:28.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.015+0000 7f07d8a00640 1 -- 172.21.15.52:0/2713037750 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07d415fcd0 con 0x7f07d415c820 2024-03-12T17:11:28.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.016+0000 7f07d8a00640 1 -- 172.21.15.52:0/2713037750 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07d41604d0 con 0x7f07d415c820 2024-03-12T17:11:28.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.017+0000 7f07cb400640 1 -- 172.21.15.52:0/2713037750 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f07bc0073c0 con 0x7f07d415c820 2024-03-12T17:11:28.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.018+0000 7f07cb400640 1 --2- 172.21.15.52:0/2713037750 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f07b006ac00 0x7f07b006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:28.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.018+0000 7f07cb400640 1 -- 172.21.15.52:0/2713037750 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f07bc088910 con 0x7f07d415c820 2024-03-12T17:11:28.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.018+0000 7f07d3400640 1 --2- 172.21.15.52:0/2713037750 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f07b006ac00 0x7f07b006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:28.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.019+0000 7f07d3400640 1 --2- 172.21.15.52:0/2713037750 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f07b006ac00 0x7f07b006d0c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f07c4036000 tx=0x7f07c4008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:28.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.022+0000 7f07cb400640 1 -- 172.21.15.52:0/2713037750 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f07d41604d0 con 0x7f07d415c820 2024-03-12T17:11:28.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.221+0000 7f07d8a00640 1 -- 172.21.15.52:0/2713037750 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f07d41604d0 con 0x7f07d415c820 2024-03-12T17:11:28.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.222+0000 7f07cb400640 1 -- 172.21.15.52:0/2713037750 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f07d41604d0 con 0x7f07d415c820 2024-03-12T17:11:28.217 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-03-12T17:11:28.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.225+0000 7f07d8a00640 1 -- 172.21.15.52:0/2713037750 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f07b006ac00 msgr2=0x7f07b006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:28.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.225+0000 7f07d8a00640 1 --2- 172.21.15.52:0/2713037750 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f07b006ac00 0x7f07b006d0c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f07c4036000 tx=0x7f07c4008030 comp rx=0 tx=0).stop 2024-03-12T17:11:28.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.225+0000 7f07d8a00640 1 -- 172.21.15.52:0/2713037750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07d415c820 msgr2=0x7f07d415ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:28.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.225+0000 7f07d8a00640 1 --2- 172.21.15.52:0/2713037750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07d415c820 0x7f07d415ea00 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f07bc00c3c0 tx=0x7f07bc00c880 comp rx=0 tx=0).stop 2024-03-12T17:11:28.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.226+0000 7f07d8a00640 1 -- 172.21.15.52:0/2713037750 shutdown_connections 2024-03-12T17:11:28.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.226+0000 7f07d8a00640 1 --2- 172.21.15.52:0/2713037750 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f07b006ac00 0x7f07b006d0c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:28.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.226+0000 7f07d8a00640 1 --2- 172.21.15.52:0/2713037750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07d4160e90 0x7f07d415ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:28.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.226+0000 7f07d8a00640 1 --2- 172.21.15.52:0/2713037750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07d415c820 0x7f07d415ea00 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:28.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.226+0000 7f07d8a00640 1 --2- 172.21.15.52:0/2713037750 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f07d40cf7c0 0x7f07d414db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:28.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.226+0000 7f07d8a00640 1 -- 172.21.15.52:0/2713037750 >> 172.21.15.52:0/2713037750 conn(0x7f07d406b050 msgr2=0x7f07d40c9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:28.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.226+0000 7f07d8a00640 1 -- 172.21.15.52:0/2713037750 shutdown_connections 2024-03-12T17:11:28.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.226+0000 7f07d8a00640 1 -- 172.21.15.52:0/2713037750 wait complete. 2024-03-12T17:11:28.238 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-12T17:11:28.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.463+0000 7f85b5200640 1 Processor -- start 2024-03-12T17:11:28.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.463+0000 7f85b5200640 1 -- start start 2024-03-12T17:11:28.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.465+0000 7f85b5200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f85b0070290 0x7f85b00b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:28.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.465+0000 7f85b5200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f85b015d1f0 0x7f85b013eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:28.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.465+0000 7f85b5200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85b00b2f50 con 0x7f85b0070670 2024-03-12T17:11:28.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.465+0000 7f85b5200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85b015d660 con 0x7f85b0070290 2024-03-12T17:11:28.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.465+0000 7f85b5200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85b00b3920 con 0x7f85b015d1f0 2024-03-12T17:11:28.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.466+0000 7f85af400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f85b0070290 0x7f85b00b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:28.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.466+0000 7f85af400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f85b0070290 0x7f85b00b33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36190/0 (socket says 172.21.15.52:36190) 2024-03-12T17:11:28.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.466+0000 7f85af400640 1 -- 172.21.15.52:0/1820281886 learned_addr learned my addr 172.21.15.52:0/1820281886 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:28.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.466+0000 7f85afe00640 1 --2- 172.21.15.52:0/1820281886 >> v2:172.21.15.102:3300/0 conn(0x7f85b015d1f0 0x7f85b013eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:28.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.466+0000 7f85af400640 1 -- 172.21.15.52:0/1820281886 >> v2:172.21.15.102:3300/0 conn(0x7f85b015d1f0 msgr2=0x7f85b013eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:28.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.466+0000 7f85af400640 1 --2- 172.21.15.52:0/1820281886 >> v2:172.21.15.102:3300/0 conn(0x7f85b015d1f0 0x7f85b013eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:28.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.466+0000 7f85af400640 1 -- 172.21.15.52:0/1820281886 >> v1:172.21.15.102:6789/0 conn(0x7f85b0070670 legacy=0x7f85b015cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:28.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.467+0000 7f85af400640 1 -- 172.21.15.52:0/1820281886 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b00b2620 con 0x7f85b0070290 2024-03-12T17:11:28.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.468+0000 7f85af400640 1 --2- 172.21.15.52:0/1820281886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85b0070290 0x7f85b00b33e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f85b00b1500 tx=0x7f85a002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee2be5134f1890d server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:28.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.469+0000 7f85ae000640 1 -- 172.21.15.52:0/1820281886 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a0031870 con 0x7f85b0070290 2024-03-12T17:11:28.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.469+0000 7f85ae000640 1 -- 172.21.15.52:0/1820281886 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a00319d0 con 0x7f85b0070290 2024-03-12T17:11:28.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.469+0000 7f85ae000640 1 -- 172.21.15.52:0/1820281886 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a0031d60 con 0x7f85b0070290 2024-03-12T17:11:28.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.469+0000 7f85b5200640 1 -- 172.21.15.52:0/1820281886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85b0070290 msgr2=0x7f85b00b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:28.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.469+0000 7f85b5200640 1 --2- 172.21.15.52:0/1820281886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85b0070290 0x7f85b00b33e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f85b00b1500 tx=0x7f85a002b910 comp rx=0 tx=0).stop 2024-03-12T17:11:28.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.470+0000 7f85b5200640 1 -- 172.21.15.52:0/1820281886 shutdown_connections 2024-03-12T17:11:28.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.470+0000 7f85b5200640 1 --2- 172.21.15.52:0/1820281886 >> v2:172.21.15.102:3300/0 conn(0x7f85b015d1f0 0x7f85b013eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:28.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.470+0000 7f85b5200640 1 --2- 172.21.15.52:0/1820281886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85b0070290 0x7f85b00b33e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:28.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.470+0000 7f85b5200640 1 -- 172.21.15.52:0/1820281886 >> 172.21.15.52:0/1820281886 conn(0x7f85b0065df0 msgr2=0x7f85b0069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:28.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.470+0000 7f85b5200640 1 -- 172.21.15.52:0/1820281886 shutdown_connections 2024-03-12T17:11:28.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.470+0000 7f85b5200640 1 -- 172.21.15.52:0/1820281886 wait complete. 2024-03-12T17:11:28.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.471+0000 7f85b5200640 1 Processor -- start 2024-03-12T17:11:28.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.472+0000 7f85b5200640 1 -- start start 2024-03-12T17:11:28.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.472+0000 7f85b5200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f85b0070290 0x7f85b01576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:28.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.473+0000 7f85b5200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85b0070670 0x7f85b012d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:28.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.473+0000 7f85b5200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85b015d1f0 0x7f85b0153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:28.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.473+0000 7f85b5200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b013b240 con 0x7f85b015d1f0 2024-03-12T17:11:28.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.473+0000 7f85b5200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b01538f0 con 0x7f85b0070670 2024-03-12T17:11:28.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.473+0000 7f85b5200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b0157d80 con 0x7f85b0070290 2024-03-12T17:11:28.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.473+0000 7f85afe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85b015d1f0 0x7f85b0153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:28.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.473+0000 7f85afe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85b015d1f0 0x7f85b0153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36204/0 (socket says 172.21.15.52:36204) 2024-03-12T17:11:28.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.473+0000 7f85afe00640 1 -- 172.21.15.52:0/860624523 learned_addr learned my addr 172.21.15.52:0/860624523 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:28.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.473+0000 7f85afe00640 1 -- 172.21.15.52:0/860624523 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f85b0070290 msgr2=0x7f85b01576a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:28.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.473+0000 7f85afe00640 1 --2- 172.21.15.52:0/860624523 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f85b0070290 0x7f85b01576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:28.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.473+0000 7f85afe00640 1 -- 172.21.15.52:0/860624523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85b0070670 msgr2=0x7f85b012d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:28.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.473+0000 7f85aea00640 1 --2- 172.21.15.52:0/860624523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85b0070670 0x7f85b012d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:28.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.473+0000 7f85afe00640 1 --2- 172.21.15.52:0/860624523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85b0070670 0x7f85b012d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:28.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.473+0000 7f85afe00640 1 -- 172.21.15.52:0/860624523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a00094d0 con 0x7f85b015d1f0 2024-03-12T17:11:28.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.474+0000 7f85afe00640 1 --2- 172.21.15.52:0/860624523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85b015d1f0 0x7f85b0153dd0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f85a8007e60 tx=0x7f85a800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:28.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.474+0000 7f85a7400640 1 -- 172.21.15.52:0/860624523 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a8014420 con 0x7f85b015d1f0 2024-03-12T17:11:28.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.474+0000 7f85a7400640 1 -- 172.21.15.52:0/860624523 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a80145a0 con 0x7f85b015d1f0 2024-03-12T17:11:28.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.474+0000 7f85b5200640 1 -- 172.21.15.52:0/860624523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85b0158060 con 0x7f85b015d1f0 2024-03-12T17:11:28.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.474+0000 7f85a7400640 1 -- 172.21.15.52:0/860624523 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a8014950 con 0x7f85b015d1f0 2024-03-12T17:11:28.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.474+0000 7f85b5200640 1 -- 172.21.15.52:0/860624523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b0158620 con 0x7f85b015d1f0 2024-03-12T17:11:28.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.475+0000 7f85b5200640 1 -- 172.21.15.52:0/860624523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85b0153af0 con 0x7f85b015d1f0 2024-03-12T17:11:28.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.476+0000 7f85a7400640 1 -- 172.21.15.52:0/860624523 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f85a8014ab0 con 0x7f85b015d1f0 2024-03-12T17:11:28.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.477+0000 7f85a7400640 1 --2- 172.21.15.52:0/860624523 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f858c06ac00 0x7f858c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:28.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.477+0000 7f85a7400640 1 -- 172.21.15.52:0/860624523 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f85a8089e50 con 0x7f85b015d1f0 2024-03-12T17:11:28.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.478+0000 7f85af400640 1 --2- 172.21.15.52:0/860624523 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f858c06ac00 0x7f858c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:28.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.479+0000 7f85af400640 1 --2- 172.21.15.52:0/860624523 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f858c06ac00 0x7f858c06d0c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f85a0036000 tx=0x7f85a0030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:28.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.481+0000 7f85a7400640 1 -- 172.21.15.52:0/860624523 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f85b0153af0 con 0x7f85b015d1f0 2024-03-12T17:11:28.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.677+0000 7f85b5200640 1 -- 172.21.15.52:0/860624523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f85b0153ab0 con 0x7f85b015d1f0 2024-03-12T17:11:28.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.678+0000 7f85a7400640 1 -- 172.21.15.52:0/860624523 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7f85a8054e30 con 0x7f85b015d1f0 2024-03-12T17:11:28.673 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-12T17:11:28.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.680+0000 7f85b5200640 1 -- 172.21.15.52:0/860624523 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f858c06ac00 msgr2=0x7f858c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:28.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.680+0000 7f85b5200640 1 --2- 172.21.15.52:0/860624523 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f858c06ac00 0x7f858c06d0c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f85a0036000 tx=0x7f85a0030000 comp rx=0 tx=0).stop 2024-03-12T17:11:28.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.680+0000 7f85b5200640 1 -- 172.21.15.52:0/860624523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85b015d1f0 msgr2=0x7f85b0153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:28.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.680+0000 7f85b5200640 1 --2- 172.21.15.52:0/860624523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85b015d1f0 0x7f85b0153dd0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f85a8007e60 tx=0x7f85a800e4b0 comp rx=0 tx=0).stop 2024-03-12T17:11:28.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.680+0000 7f85b5200640 1 -- 172.21.15.52:0/860624523 shutdown_connections 2024-03-12T17:11:28.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.680+0000 7f85b5200640 1 --2- 172.21.15.52:0/860624523 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f858c06ac00 0x7f858c06d0c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:28.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.680+0000 7f85b5200640 1 --2- 172.21.15.52:0/860624523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85b015d1f0 0x7f85b0153dd0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:28.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.680+0000 7f85b5200640 1 --2- 172.21.15.52:0/860624523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85b0070670 0x7f85b012d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:28.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.680+0000 7f85b5200640 1 --2- 172.21.15.52:0/860624523 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f85b0070290 0x7f85b01576a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:28.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.681+0000 7f85b5200640 1 -- 172.21.15.52:0/860624523 >> 172.21.15.52:0/860624523 conn(0x7f85b0065df0 msgr2=0x7f85b013d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:28.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.681+0000 7f85b5200640 1 -- 172.21.15.52:0/860624523 shutdown_connections 2024-03-12T17:11:28.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.681+0000 7f85b5200640 1 -- 172.21.15.52:0/860624523 wait complete. 2024-03-12T17:11:28.694 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-12T17:11:28.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.916+0000 7f614ea00640 1 Processor -- start 2024-03-12T17:11:28.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.917+0000 7f614ea00640 1 -- start start 2024-03-12T17:11:28.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.918+0000 7f614ea00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f61481456a0 0x7f61480ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:28.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.918+0000 7f614ea00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f6148070cd0 0x7f61480cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:28.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.918+0000 7f614ea00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f614806d390 con 0x7f6148145c20 2024-03-12T17:11:28.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.918+0000 7f614ea00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61480ad890 con 0x7f6148070cd0 2024-03-12T17:11:28.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.918+0000 7f614ea00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6148145a80 con 0x7f61481456a0 2024-03-12T17:11:28.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.919+0000 7f613fe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f6148070cd0 0x7f61480cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:28.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.919+0000 7f613fe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f6148070cd0 0x7f61480cee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36210/0 (socket says 172.21.15.52:36210) 2024-03-12T17:11:28.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.919+0000 7f613fe00640 1 -- 172.21.15.52:0/2857903412 learned_addr learned my addr 172.21.15.52:0/2857903412 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:28.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.919+0000 7f6147e00640 1 --2- 172.21.15.52:0/2857903412 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f61481456a0 0x7f61480ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:28.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.919+0000 7f613fe00640 1 -- 172.21.15.52:0/2857903412 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f61481456a0 msgr2=0x7f61480ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:28.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.919+0000 7f613fe00640 1 --2- 172.21.15.52:0/2857903412 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f61481456a0 0x7f61480ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:28.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.919+0000 7f613fe00640 1 -- 172.21.15.52:0/2857903412 >> v1:172.21.15.102:6789/0 conn(0x7f6148145c20 legacy=0x7f61480ab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:28.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.919+0000 7f613fe00640 1 -- 172.21.15.52:0/2857903412 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f614806d670 con 0x7f6148070cd0 2024-03-12T17:11:28.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.921+0000 7f613fe00640 1 --2- 172.21.15.52:0/2857903412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6148070cd0 0x7f61480cee70 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6130009870 tx=0x7f613002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3cf2f6b9f5c0bd server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:28.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.921+0000 7f6147400640 1 -- 172.21.15.52:0/2857903412 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6130030840 con 0x7f6148070cd0 2024-03-12T17:11:28.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.921+0000 7f6147400640 1 -- 172.21.15.52:0/2857903412 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61300309a0 con 0x7f6148070cd0 2024-03-12T17:11:28.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.921+0000 7f6147400640 1 -- 172.21.15.52:0/2857903412 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6130030d30 con 0x7f6148070cd0 2024-03-12T17:11:28.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.922+0000 7f614ea00640 1 -- 172.21.15.52:0/2857903412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6148070cd0 msgr2=0x7f61480cee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:28.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.922+0000 7f614ea00640 1 --2- 172.21.15.52:0/2857903412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6148070cd0 0x7f61480cee70 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6130009870 tx=0x7f613002b980 comp rx=0 tx=0).stop 2024-03-12T17:11:28.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.922+0000 7f614ea00640 1 -- 172.21.15.52:0/2857903412 shutdown_connections 2024-03-12T17:11:28.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.922+0000 7f614ea00640 1 --2- 172.21.15.52:0/2857903412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6148070cd0 0x7f61480cee70 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:28.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.922+0000 7f614ea00640 1 --2- 172.21.15.52:0/2857903412 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f61481456a0 0x7f61480ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:28.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.922+0000 7f614ea00640 1 -- 172.21.15.52:0/2857903412 >> 172.21.15.52:0/2857903412 conn(0x7f614806a0b0 msgr2=0x7f614806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:28.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.922+0000 7f614ea00640 1 -- 172.21.15.52:0/2857903412 shutdown_connections 2024-03-12T17:11:28.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.922+0000 7f614ea00640 1 -- 172.21.15.52:0/2857903412 wait complete. 2024-03-12T17:11:28.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.924+0000 7f614ea00640 1 Processor -- start 2024-03-12T17:11:28.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.924+0000 7f614ea00640 1 -- start start 2024-03-12T17:11:28.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.925+0000 7f614ea00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6148070cd0 0x7f6148159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:28.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.925+0000 7f614ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61481456a0 0x7f61481f6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:28.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.925+0000 7f614ea00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6148145c20 0x7f614813c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:28.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.925+0000 7f614ea00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6148159cb0 con 0x7f6148145c20 2024-03-12T17:11:28.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.925+0000 7f614ea00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61481f6960 con 0x7f61481456a0 2024-03-12T17:11:28.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.925+0000 7f614ea00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f614813a8e0 con 0x7f6148070cd0 2024-03-12T17:11:28.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.925+0000 7f614ca00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6148145c20 0x7f614813c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:28.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.925+0000 7f614ca00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6148145c20 0x7f614813c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36226/0 (socket says 172.21.15.52:36226) 2024-03-12T17:11:28.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.925+0000 7f614ca00640 1 -- 172.21.15.52:0/3458616713 learned_addr learned my addr 172.21.15.52:0/3458616713 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:28.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.926+0000 7f6147e00640 1 --2- 172.21.15.52:0/3458616713 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6148070cd0 0x7f6148159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:28.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.926+0000 7f613fe00640 1 --2- 172.21.15.52:0/3458616713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61481456a0 0x7f61481f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:28.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.926+0000 7f614ca00640 1 -- 172.21.15.52:0/3458616713 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6148070cd0 msgr2=0x7f6148159770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:28.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.926+0000 7f614ca00640 1 --2- 172.21.15.52:0/3458616713 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6148070cd0 0x7f6148159770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:28.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.926+0000 7f614ca00640 1 -- 172.21.15.52:0/3458616713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61481456a0 msgr2=0x7f61481f6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:28.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.926+0000 7f614ca00640 1 --2- 172.21.15.52:0/3458616713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61481456a0 0x7f61481f6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:28.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.926+0000 7f614ca00640 1 -- 172.21.15.52:0/3458616713 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61300094d0 con 0x7f6148145c20 2024-03-12T17:11:28.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.926+0000 7f6147e00640 1 --2- 172.21.15.52:0/3458616713 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6148070cd0 0x7f6148159770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:11:28.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.926+0000 7f614ca00640 1 --2- 172.21.15.52:0/3458616713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6148145c20 0x7f614813c5a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f614000c6e0 tx=0x7f614000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:28.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.926+0000 7f6144c00640 1 -- 172.21.15.52:0/3458616713 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6140003b70 con 0x7f6148145c20 2024-03-12T17:11:28.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.926+0000 7f6144c00640 1 -- 172.21.15.52:0/3458616713 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6140003cf0 con 0x7f6148145c20 2024-03-12T17:11:28.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.926+0000 7f6144c00640 1 -- 172.21.15.52:0/3458616713 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6140005310 con 0x7f6148145c20 2024-03-12T17:11:28.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.926+0000 7f614ea00640 1 -- 172.21.15.52:0/3458616713 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f614813ab80 con 0x7f6148145c20 2024-03-12T17:11:28.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.927+0000 7f614ea00640 1 -- 172.21.15.52:0/3458616713 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f614813b140 con 0x7f6148145c20 2024-03-12T17:11:28.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.928+0000 7f614ea00640 1 -- 172.21.15.52:0/3458616713 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6148073390 con 0x7f6148145c20 2024-03-12T17:11:28.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.929+0000 7f6144c00640 1 -- 172.21.15.52:0/3458616713 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6140005470 con 0x7f6148145c20 2024-03-12T17:11:28.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.929+0000 7f6144c00640 1 --2- 172.21.15.52:0/3458616713 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f612406ac00 0x7f612406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:28.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.930+0000 7f6144c00640 1 -- 172.21.15.52:0/3458616713 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f61400892e0 con 0x7f6148145c20 2024-03-12T17:11:28.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.930+0000 7f6147e00640 1 --2- 172.21.15.52:0/3458616713 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f612406ac00 0x7f612406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:28.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.931+0000 7f6147e00640 1 --2- 172.21.15.52:0/3458616713 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f612406ac00 0x7f612406d0c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6138009ac0 tx=0x7f6138008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:28.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:28.934+0000 7f6144c00640 1 -- 172.21.15.52:0/3458616713 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f61400542c0 con 0x7f6148145c20 2024-03-12T17:11:29.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.133+0000 7f614ea00640 1 -- 172.21.15.52:0/3458616713 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f614806cbc0 con 0x7f6148145c20 2024-03-12T17:11:29.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.134+0000 7f6144c00640 1 -- 172.21.15.52:0/3458616713 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6140003e60 con 0x7f6148145c20 2024-03-12T17:11:29.128 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-03-12T17:11:29.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.137+0000 7f614ea00640 1 -- 172.21.15.52:0/3458616713 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f612406ac00 msgr2=0x7f612406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:29.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.137+0000 7f614ea00640 1 --2- 172.21.15.52:0/3458616713 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f612406ac00 0x7f612406d0c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6138009ac0 tx=0x7f6138008040 comp rx=0 tx=0).stop 2024-03-12T17:11:29.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.137+0000 7f614ea00640 1 -- 172.21.15.52:0/3458616713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6148145c20 msgr2=0x7f614813c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:29.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.137+0000 7f614ea00640 1 --2- 172.21.15.52:0/3458616713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6148145c20 0x7f614813c5a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f614000c6e0 tx=0x7f614000cba0 comp rx=0 tx=0).stop 2024-03-12T17:11:29.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.137+0000 7f614ea00640 1 -- 172.21.15.52:0/3458616713 shutdown_connections 2024-03-12T17:11:29.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.137+0000 7f614ea00640 1 --2- 172.21.15.52:0/3458616713 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f612406ac00 0x7f612406d0c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:29.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.137+0000 7f614ea00640 1 --2- 172.21.15.52:0/3458616713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6148145c20 0x7f614813c5a0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:29.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.137+0000 7f614ea00640 1 --2- 172.21.15.52:0/3458616713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61481456a0 0x7f61481f6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:29.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.137+0000 7f614ea00640 1 --2- 172.21.15.52:0/3458616713 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6148070cd0 0x7f6148159770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:29.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.137+0000 7f614ea00640 1 -- 172.21.15.52:0/3458616713 >> 172.21.15.52:0/3458616713 conn(0x7f614806a0b0 msgr2=0x7f61481402b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:29.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.137+0000 7f614ea00640 1 -- 172.21.15.52:0/3458616713 shutdown_connections 2024-03-12T17:11:29.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.137+0000 7f614ea00640 1 -- 172.21.15.52:0/3458616713 wait complete. 2024-03-12T17:11:29.151 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T17:11:29.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.355+0000 7fa21b200640 1 Processor -- start 2024-03-12T17:11:29.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.356+0000 7fa21b200640 1 -- start start 2024-03-12T17:11:29.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.357+0000 7fa21b200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa21c15cab0 0x7fa21c0b8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:29.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.357+0000 7fa21b200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa21c0ceeb0 0x7fa21c0cf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:29.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.357+0000 7fa21b200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa21c0b4120 0x7fa21c0b4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:29.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.357+0000 7fa21b200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa21c0cf850 con 0x7fa21c15cab0 2024-03-12T17:11:29.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.357+0000 7fa21b200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa21c15ce90 con 0x7fa21c0ceeb0 2024-03-12T17:11:29.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.357+0000 7fa21b200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c0cddf0 con 0x7fa21c0b4120 2024-03-12T17:11:29.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.357+0000 7fa219e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa21c15cab0 0x7fa21c0b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:29.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.357+0000 7fa219e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa21c15cab0 0x7fa21c0b8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36236/0 (socket says 172.21.15.52:36236) 2024-03-12T17:11:29.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.357+0000 7fa219e00640 1 -- 172.21.15.52:0/59696106 learned_addr learned my addr 172.21.15.52:0/59696106 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:29.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.358+0000 7fa21a800640 1 --2- 172.21.15.52:0/59696106 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa21c0b4120 0x7fa21c0b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:29.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.358+0000 7fa219400640 1 --2- 172.21.15.52:0/59696106 >> v2:172.21.15.102:3300/0 conn(0x7fa21c0ceeb0 0x7fa21c0cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:29.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.358+0000 7fa219e00640 1 -- 172.21.15.52:0/59696106 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa21c0b4120 msgr2=0x7fa21c0b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:29.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.358+0000 7fa219e00640 1 --2- 172.21.15.52:0/59696106 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa21c0b4120 0x7fa21c0b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:29.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.358+0000 7fa219e00640 1 -- 172.21.15.52:0/59696106 >> v2:172.21.15.102:3300/0 conn(0x7fa21c0ceeb0 msgr2=0x7fa21c0cf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:29.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.358+0000 7fa219e00640 1 --2- 172.21.15.52:0/59696106 >> v2:172.21.15.102:3300/0 conn(0x7fa21c0ceeb0 0x7fa21c0cf310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:29.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.358+0000 7fa219e00640 1 -- 172.21.15.52:0/59696106 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21c0cd570 con 0x7fa21c15cab0 2024-03-12T17:11:29.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.359+0000 7fa219e00640 1 --2- 172.21.15.52:0/59696106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa21c15cab0 0x7fa21c0b8050 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa21c0d3750 tx=0x7fa20c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0927cf2804bfe80 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:29.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.359+0000 7fa218a00640 1 -- 172.21.15.52:0/59696106 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c031860 con 0x7fa21c15cab0 2024-03-12T17:11:29.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.359+0000 7fa218a00640 1 -- 172.21.15.52:0/59696106 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa20c0319c0 con 0x7fa21c15cab0 2024-03-12T17:11:29.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.359+0000 7fa218a00640 1 -- 172.21.15.52:0/59696106 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c031d50 con 0x7fa21c15cab0 2024-03-12T17:11:29.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.360+0000 7fa21b200640 1 -- 172.21.15.52:0/59696106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa21c15cab0 msgr2=0x7fa21c0b8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:29.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.360+0000 7fa21b200640 1 --2- 172.21.15.52:0/59696106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa21c15cab0 0x7fa21c0b8050 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa21c0d3750 tx=0x7fa20c02b900 comp rx=0 tx=0).stop 2024-03-12T17:11:29.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.360+0000 7fa21b200640 1 -- 172.21.15.52:0/59696106 shutdown_connections 2024-03-12T17:11:29.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.360+0000 7fa21b200640 1 --2- 172.21.15.52:0/59696106 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa21c0b4120 0x7fa21c0b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:29.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.360+0000 7fa21b200640 1 --2- 172.21.15.52:0/59696106 >> v2:172.21.15.102:3300/0 conn(0x7fa21c0ceeb0 0x7fa21c0cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:29.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.360+0000 7fa21b200640 1 --2- 172.21.15.52:0/59696106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa21c15cab0 0x7fa21c0b8050 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:29.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.360+0000 7fa21b200640 1 -- 172.21.15.52:0/59696106 >> 172.21.15.52:0/59696106 conn(0x7fa21c06ae90 msgr2=0x7fa21c06b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:29.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.360+0000 7fa21b200640 1 -- 172.21.15.52:0/59696106 shutdown_connections 2024-03-12T17:11:29.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.360+0000 7fa21b200640 1 -- 172.21.15.52:0/59696106 wait complete. 2024-03-12T17:11:29.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.362+0000 7fa21b200640 1 Processor -- start 2024-03-12T17:11:29.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.362+0000 7fa21b200640 1 -- start start 2024-03-12T17:11:29.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.363+0000 7fa21b200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa21c0b4120 0x7fa21c0b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:29.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.363+0000 7fa21b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa21c0ceeb0 0x7fa21c0b6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:29.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.363+0000 7fa21b200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa21c15cab0 0x7fa21c15d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:29.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.363+0000 7fa21b200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c1576f0 con 0x7fa21c15cab0 2024-03-12T17:11:29.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.363+0000 7fa21b200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c157410 con 0x7fa21c0ceeb0 2024-03-12T17:11:29.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.363+0000 7fa21b200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c0b63f0 con 0x7fa21c0b4120 2024-03-12T17:11:29.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.363+0000 7fa21a800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa21c15cab0 0x7fa21c15d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:29.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.363+0000 7fa21a800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa21c15cab0 0x7fa21c15d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36252/0 (socket says 172.21.15.52:36252) 2024-03-12T17:11:29.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.363+0000 7fa21a800640 1 -- 172.21.15.52:0/4025000520 learned_addr learned my addr 172.21.15.52:0/4025000520 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:29.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.363+0000 7fa219e00640 1 --2- 172.21.15.52:0/4025000520 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa21c0b4120 0x7fa21c0b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:29.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.363+0000 7fa219400640 1 --2- 172.21.15.52:0/4025000520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa21c0ceeb0 0x7fa21c0b6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:29.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.364+0000 7fa21a800640 1 -- 172.21.15.52:0/4025000520 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa21c0b4120 msgr2=0x7fa21c0b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:29.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.364+0000 7fa21a800640 1 --2- 172.21.15.52:0/4025000520 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa21c0b4120 0x7fa21c0b2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:29.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.364+0000 7fa21a800640 1 -- 172.21.15.52:0/4025000520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa21c0ceeb0 msgr2=0x7fa21c0b6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:29.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.364+0000 7fa21a800640 1 --2- 172.21.15.52:0/4025000520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa21c0ceeb0 0x7fa21c0b6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:29.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.364+0000 7fa21a800640 1 -- 172.21.15.52:0/4025000520 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20c0094d0 con 0x7fa21c15cab0 2024-03-12T17:11:29.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.364+0000 7fa21a800640 1 --2- 172.21.15.52:0/4025000520 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa21c15cab0 0x7fa21c15d060 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa214007d20 tx=0x7fa21400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:29.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.364+0000 7fa212000640 1 -- 172.21.15.52:0/4025000520 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa214014420 con 0x7fa21c15cab0 2024-03-12T17:11:29.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.364+0000 7fa212000640 1 -- 172.21.15.52:0/4025000520 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2140145a0 con 0x7fa21c15cab0 2024-03-12T17:11:29.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.364+0000 7fa212000640 1 -- 172.21.15.52:0/4025000520 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa214014950 con 0x7fa21c15cab0 2024-03-12T17:11:29.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.364+0000 7fa21b200640 1 -- 172.21.15.52:0/4025000520 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa21c0b66d0 con 0x7fa21c15cab0 2024-03-12T17:11:29.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.364+0000 7fa21b200640 1 -- 172.21.15.52:0/4025000520 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21c157030 con 0x7fa21c15cab0 2024-03-12T17:11:29.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.366+0000 7fa21b200640 1 -- 172.21.15.52:0/4025000520 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa21c0b6250 con 0x7fa21c15cab0 2024-03-12T17:11:29.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.366+0000 7fa212000640 1 -- 172.21.15.52:0/4025000520 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa214014ab0 con 0x7fa21c15cab0 2024-03-12T17:11:29.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.368+0000 7fa212000640 1 --2- 172.21.15.52:0/4025000520 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa1fc06ac30 0x7fa1fc06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:29.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.368+0000 7fa212000640 1 -- 172.21.15.52:0/4025000520 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa214089dc0 con 0x7fa21c15cab0 2024-03-12T17:11:29.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.369+0000 7fa219e00640 1 --2- 172.21.15.52:0/4025000520 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa1fc06ac30 0x7fa1fc06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:29.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.370+0000 7fa219e00640 1 --2- 172.21.15.52:0/4025000520 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa1fc06ac30 0x7fa1fc06d0f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa20c036000 tx=0x7fa20c03c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:29.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.373+0000 7fa212000640 1 -- 172.21.15.52:0/4025000520 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fa21c0b6250 con 0x7fa21c15cab0 2024-03-12T17:11:29.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.608+0000 7fa21b200640 1 -- 172.21.15.52:0/4025000520 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa21c0b6bf0 con 0x7fa21c15cab0 2024-03-12T17:11:29.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.609+0000 7fa212000640 1 -- 172.21.15.52:0/4025000520 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4734 (secure 0 0 0) 0x7fa21c0b6bf0 con 0x7fa21c15cab0 2024-03-12T17:11:29.604 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:11:29.604 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6836/285561331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":285561331},{"type":"v1","addr":"172.21.15.102:6836","nonce":285561331}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4909,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1312095655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6835","nonce":1312095655},{"type":"v1","addr":"172.21.15.102:6837","nonce":1312095655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5337,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2451989690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2451989690},{"type":"v1","addr":"172.21.15.52:6833","nonce":2451989690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-12T17:08:10.679436+0000","modified":"2024-03-12T17:08:14.140830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5343},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5343":{"gid":5343,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.52:6835/543567396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":543567396},{"type":"v1","addr":"172.21.15.52:6835","nonce":543567396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-12T17:11:29.604 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 17 2024-03-12T17:11:29.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.612+0000 7fa21b200640 1 -- 172.21.15.52:0/4025000520 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa1fc06ac30 msgr2=0x7fa1fc06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:29.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.612+0000 7fa21b200640 1 --2- 172.21.15.52:0/4025000520 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa1fc06ac30 0x7fa1fc06d0f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa20c036000 tx=0x7fa20c03c040 comp rx=0 tx=0).stop 2024-03-12T17:11:29.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.612+0000 7fa21b200640 1 -- 172.21.15.52:0/4025000520 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa21c15cab0 msgr2=0x7fa21c15d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:29.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.612+0000 7fa21b200640 1 --2- 172.21.15.52:0/4025000520 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa21c15cab0 0x7fa21c15d060 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa214007d20 tx=0x7fa21400e4b0 comp rx=0 tx=0).stop 2024-03-12T17:11:29.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.612+0000 7fa21b200640 1 -- 172.21.15.52:0/4025000520 shutdown_connections 2024-03-12T17:11:29.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.612+0000 7fa21b200640 1 --2- 172.21.15.52:0/4025000520 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa1fc06ac30 0x7fa1fc06d0f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:29.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.612+0000 7fa21b200640 1 --2- 172.21.15.52:0/4025000520 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa21c15cab0 0x7fa21c15d060 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:29.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.612+0000 7fa21b200640 1 --2- 172.21.15.52:0/4025000520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa21c0ceeb0 0x7fa21c0b6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:29.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.612+0000 7fa21b200640 1 --2- 172.21.15.52:0/4025000520 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa21c0b4120 0x7fa21c0b2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:29.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.612+0000 7fa21b200640 1 -- 172.21.15.52:0/4025000520 >> 172.21.15.52:0/4025000520 conn(0x7fa21c06ae90 msgr2=0x7fa21c0b4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:29.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.613+0000 7fa21b200640 1 -- 172.21.15.52:0/4025000520 shutdown_connections 2024-03-12T17:11:29.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.613+0000 7fa21b200640 1 -- 172.21.15.52:0/4025000520 wait complete. 2024-03-12T17:11:29.626 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:11:29.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.845+0000 7fb555a00640 1 Processor -- start 2024-03-12T17:11:29.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.845+0000 7fb555a00640 1 -- start start 2024-03-12T17:11:29.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.846+0000 7fb555a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb5500afab0 0x7fb5500cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:29.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.846+0000 7fb555a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb5500cd390 0x7fb5500b0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:29.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.846+0000 7fb555a00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5500ce6d0 con 0x7fb5500b0990 2024-03-12T17:11:29.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.846+0000 7fb555a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5500afe90 con 0x7fb5500afab0 2024-03-12T17:11:29.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.846+0000 7fb555a00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5500b14c0 con 0x7fb5500cd390 2024-03-12T17:11:29.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.847+0000 7fb555000640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fb5500b0990 0x7fb55015c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:40088/0 (socket says 172.21.15.52:40088) 2024-03-12T17:11:29.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.847+0000 7fb555000640 1 -- 172.21.15.52:0/3832092131 learned_addr learned my addr 172.21.15.52:0/3832092131 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:11:29.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.847+0000 7fb54f400640 1 --2- 172.21.15.52:0/3832092131 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb5500cd390 0x7fb5500b0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:29.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.847+0000 7fb54fe00640 1 --2- 172.21.15.52:0/3832092131 >> v2:172.21.15.102:3300/0 conn(0x7fb5500afab0 0x7fb5500cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:29.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.847+0000 7fb54ea00640 1 -- 172.21.15.52:0/3832092131 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3488224222 0 0) 0x7fb5500ce6d0 con 0x7fb5500b0990 2024-03-12T17:11:29.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.847+0000 7fb54ea00640 1 -- 172.21.15.52:0/3832092131 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb530003520 con 0x7fb5500b0990 2024-03-12T17:11:29.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.848+0000 7fb54ea00640 1 -- 172.21.15.52:0/3832092131 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1276398209 0 0) 0x7fb530003520 con 0x7fb5500b0990 2024-03-12T17:11:29.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.848+0000 7fb54ea00640 1 -- 172.21.15.52:0/3832092131 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb5500cd390 msgr2=0x7fb5500b0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:29.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.848+0000 7fb54ea00640 1 --2- 172.21.15.52:0/3832092131 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb5500cd390 0x7fb5500b0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:29.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.848+0000 7fb54ea00640 1 -- 172.21.15.52:0/3832092131 >> v2:172.21.15.102:3300/0 conn(0x7fb5500afab0 msgr2=0x7fb5500cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:29.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.848+0000 7fb54ea00640 1 --2- 172.21.15.52:0/3832092131 >> v2:172.21.15.102:3300/0 conn(0x7fb5500afab0 0x7fb5500cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:29.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.848+0000 7fb54ea00640 1 -- 172.21.15.52:0/3832092131 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5500cd7f0 con 0x7fb5500b0990 2024-03-12T17:11:29.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.848+0000 7fb54ea00640 1 -- 172.21.15.52:0/3832092131 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7fb548001670 con 0x7fb5500b0990 2024-03-12T17:11:29.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.848+0000 7fb54ea00640 1 -- 172.21.15.52:0/3832092131 >> v1:172.21.15.52:6789/0 conn(0x7fb5500b0990 legacy=0x7fb55015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:29.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.848+0000 7fb54ea00640 1 --2- 172.21.15.52:0/3832092131 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb5500cd390 0x7fb530003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:29.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.848+0000 7fb54ea00640 1 --2- 172.21.15.52:0/3832092131 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb530003ef0 0x7fb5300047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:29.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.848+0000 7fb54ea00640 1 --2- 172.21.15.52:0/3832092131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb530004d00 0x7fb530005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:29.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.848+0000 7fb54ea00640 1 -- 172.21.15.52:0/3832092131 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5500afe90 con 0x7fb530004d00 2024-03-12T17:11:29.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.848+0000 7fb54ea00640 1 -- 172.21.15.52:0/3832092131 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5500b14c0 con 0x7fb530003ef0 2024-03-12T17:11:29.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.848+0000 7fb54ea00640 1 -- 172.21.15.52:0/3832092131 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb53000b980 con 0x7fb5500cd390 2024-03-12T17:11:29.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.848+0000 7fb555000640 1 --2- 172.21.15.52:0/3832092131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb530004d00 0x7fb530005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:29.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.848+0000 7fb54fe00640 1 --2- 172.21.15.52:0/3832092131 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb5500cd390 0x7fb530003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:29.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.849+0000 7fb54f400640 1 --2- 172.21.15.52:0/3832092131 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb530003ef0 0x7fb5300047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:29.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.849+0000 7fb555000640 1 -- 172.21.15.52:0/3832092131 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb5500cd390 msgr2=0x7fb530003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:29.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.849+0000 7fb555000640 1 --2- 172.21.15.52:0/3832092131 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb5500cd390 0x7fb530003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:29.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.849+0000 7fb555000640 1 -- 172.21.15.52:0/3832092131 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb530003ef0 msgr2=0x7fb5300047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:29.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.849+0000 7fb555000640 1 --2- 172.21.15.52:0/3832092131 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb530003ef0 0x7fb5300047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:29.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.849+0000 7fb555000640 1 -- 172.21.15.52:0/3832092131 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb530003520 con 0x7fb530004d00 2024-03-12T17:11:29.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.850+0000 7fb555000640 1 --2- 172.21.15.52:0/3832092131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb530004d00 0x7fb530005180 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb54800c240 tx=0x7fb5300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66bbafe93d46c5f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:29.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.850+0000 7fb54ea00640 1 -- 172.21.15.52:0/3832092131 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54802cae0 con 0x7fb530004d00 2024-03-12T17:11:29.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.850+0000 7fb54ea00640 1 -- 172.21.15.52:0/3832092131 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54802cc40 con 0x7fb530004d00 2024-03-12T17:11:29.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.850+0000 7fb54ea00640 1 -- 172.21.15.52:0/3832092131 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb548005af0 con 0x7fb530004d00 2024-03-12T17:11:29.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.851+0000 7fb555a00640 1 -- 172.21.15.52:0/3832092131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb530004d00 msgr2=0x7fb530005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:29.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.851+0000 7fb555a00640 1 --2- 172.21.15.52:0/3832092131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb530004d00 0x7fb530005180 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb54800c240 tx=0x7fb5300057f0 comp rx=0 tx=0).stop 2024-03-12T17:11:29.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.851+0000 7fb555000640 1 -- 172.21.15.52:0/3832092131 reap_dead start 2024-03-12T17:11:29.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.851+0000 7fb555a00640 1 -- 172.21.15.52:0/3832092131 shutdown_connections 2024-03-12T17:11:29.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.851+0000 7fb555a00640 1 -- 172.21.15.52:0/3832092131 >> 172.21.15.52:0/3832092131 conn(0x7fb5500d0540 msgr2=0x7fb5500d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:29.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.851+0000 7fb555a00640 1 -- 172.21.15.52:0/3832092131 shutdown_connections 2024-03-12T17:11:29.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.851+0000 7fb555a00640 1 -- 172.21.15.52:0/3832092131 wait complete. 2024-03-12T17:11:29.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.853+0000 7fb555a00640 1 Processor -- start 2024-03-12T17:11:29.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.853+0000 7fb555a00640 1 -- start start 2024-03-12T17:11:29.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.854+0000 7fb555a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb530004d00 0x7fb5501543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:29.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.854+0000 7fb555a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5500cf2e0 0x7fb5500cf760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:29.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.854+0000 7fb555a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5500c5ee0 0x7fb5500c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:29.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.854+0000 7fb555a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55009c3d0 con 0x7fb5500c5ee0 2024-03-12T17:11:29.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.854+0000 7fb555a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5500cfca0 con 0x7fb5500cf2e0 2024-03-12T17:11:29.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.854+0000 7fb555a00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5501f52c0 con 0x7fb530004d00 2024-03-12T17:11:29.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.854+0000 7fb555000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5500c5ee0 0x7fb5500c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:29.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.854+0000 7fb555000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5500c5ee0 0x7fb5500c6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36266/0 (socket says 172.21.15.52:36266) 2024-03-12T17:11:29.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.854+0000 7fb555000640 1 -- 172.21.15.52:0/554142296 learned_addr learned my addr 172.21.15.52:0/554142296 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:29.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.855+0000 7fb54f400640 1 --2- 172.21.15.52:0/554142296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5500cf2e0 0x7fb5500cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:29.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.855+0000 7fb54fe00640 1 --2- 172.21.15.52:0/554142296 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb530004d00 0x7fb5501543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:29.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.855+0000 7fb555000640 1 -- 172.21.15.52:0/554142296 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb530004d00 msgr2=0x7fb5501543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:29.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.855+0000 7fb555000640 1 --2- 172.21.15.52:0/554142296 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb530004d00 0x7fb5501543a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:29.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.855+0000 7fb555000640 1 -- 172.21.15.52:0/554142296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5500cf2e0 msgr2=0x7fb5500cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:29.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.855+0000 7fb555000640 1 --2- 172.21.15.52:0/554142296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5500cf2e0 0x7fb5500cf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:29.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.855+0000 7fb555000640 1 -- 172.21.15.52:0/554142296 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5501f5e40 con 0x7fb5500c5ee0 2024-03-12T17:11:29.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.855+0000 7fb555000640 1 --2- 172.21.15.52:0/554142296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5500c5ee0 0x7fb5500c6360 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb54800c240 tx=0x7fb548001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:29.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.855+0000 7fb547e00640 1 -- 172.21.15.52:0/554142296 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54803c040 con 0x7fb5500c5ee0 2024-03-12T17:11:29.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.856+0000 7fb547e00640 1 -- 172.21.15.52:0/554142296 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb548001630 con 0x7fb5500c5ee0 2024-03-12T17:11:29.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.856+0000 7fb547e00640 1 -- 172.21.15.52:0/554142296 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb548007270 con 0x7fb5500c5ee0 2024-03-12T17:11:29.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.856+0000 7fb555a00640 1 -- 172.21.15.52:0/554142296 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5501f5510 con 0x7fb5500c5ee0 2024-03-12T17:11:29.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.856+0000 7fb555a00640 1 -- 172.21.15.52:0/554142296 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5501f5ad0 con 0x7fb5500c5ee0 2024-03-12T17:11:29.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.857+0000 7fb555a00640 1 -- 172.21.15.52:0/554142296 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5501f5fd0 con 0x7fb5500c5ee0 2024-03-12T17:11:29.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.858+0000 7fb547e00640 1 -- 172.21.15.52:0/554142296 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb548032070 con 0x7fb5500c5ee0 2024-03-12T17:11:29.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.859+0000 7fb547e00640 1 --2- 172.21.15.52:0/554142296 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb52c06ac30 0x7fb52c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:29.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.859+0000 7fb547e00640 1 -- 172.21.15.52:0/554142296 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb5480aa670 con 0x7fb5500c5ee0 2024-03-12T17:11:29.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.859+0000 7fb54fe00640 1 --2- 172.21.15.52:0/554142296 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb52c06ac30 0x7fb52c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:29.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.860+0000 7fb54fe00640 1 --2- 172.21.15.52:0/554142296 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb52c06ac30 0x7fb52c06d0f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb540009a90 tx=0x7fb540008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:29.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:29.863+0000 7fb547e00640 1 -- 172.21.15.52:0/554142296 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fb548075830 con 0x7fb5500c5ee0 2024-03-12T17:11:30.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.067+0000 7fb555a00640 1 -- 172.21.15.52:0/554142296 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb55004e030 con 0x7fb5500c5ee0 2024-03-12T17:11:30.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.070+0000 7fb547e00640 1 -- 172.21.15.52:0/554142296 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+16298 (secure 0 0 0) 0x7fb548046090 con 0x7fb5500c5ee0 2024-03-12T17:11:30.065 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:11:30.065 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":42,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:09:13.132892+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T17:06:19.218543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-12T17:08:09.114678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-12T17:08:09.691941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.733031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.775178+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.591522+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.820838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.829252+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.582481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.d","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:11:30.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.073+0000 7fb555a00640 1 -- 172.21.15.52:0/554142296 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb52c06ac30 msgr2=0x7fb52c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:30.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.073+0000 7fb555a00640 1 --2- 172.21.15.52:0/554142296 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb52c06ac30 0x7fb52c06d0f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb540009a90 tx=0x7fb540008040 comp rx=0 tx=0).stop 2024-03-12T17:11:30.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.073+0000 7fb555a00640 1 -- 172.21.15.52:0/554142296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5500c5ee0 msgr2=0x7fb5500c6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:30.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.073+0000 7fb555a00640 1 --2- 172.21.15.52:0/554142296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5500c5ee0 0x7fb5500c6360 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb54800c240 tx=0x7fb548001a70 comp rx=0 tx=0).stop 2024-03-12T17:11:30.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.073+0000 7fb555a00640 1 -- 172.21.15.52:0/554142296 shutdown_connections 2024-03-12T17:11:30.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.073+0000 7fb555a00640 1 --2- 172.21.15.52:0/554142296 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb52c06ac30 0x7fb52c06d0f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:30.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.073+0000 7fb555a00640 1 --2- 172.21.15.52:0/554142296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5500c5ee0 0x7fb5500c6360 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:30.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.073+0000 7fb555a00640 1 --2- 172.21.15.52:0/554142296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5500cf2e0 0x7fb5500cf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:30.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.073+0000 7fb555a00640 1 --2- 172.21.15.52:0/554142296 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb530004d00 0x7fb5501543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:30.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.073+0000 7fb555a00640 1 -- 172.21.15.52:0/554142296 >> 172.21.15.52:0/554142296 conn(0x7fb5500d0540 msgr2=0x7fb550098a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:30.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.073+0000 7fb555a00640 1 -- 172.21.15.52:0/554142296 shutdown_connections 2024-03-12T17:11:30.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.073+0000 7fb555a00640 1 -- 172.21.15.52:0/554142296 wait complete. 2024-03-12T17:11:30.088 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-12T17:11:30.088 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T17:11:30.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.319+0000 7fa4eec00640 1 Processor -- start 2024-03-12T17:11:30.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.320+0000 7fa4eec00640 1 -- start start 2024-03-12T17:11:30.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.321+0000 7fa4eec00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa4e815cda0 0x7fa4e80cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:30.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.321+0000 7fa4eec00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa4e80b5a30 0x7fa4e80b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:30.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.321+0000 7fa4eec00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4e80d1830 con 0x7fa4e80b4350 2024-03-12T17:11:30.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.321+0000 7fa4eec00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4e80b1f20 con 0x7fa4e80b5a30 2024-03-12T17:11:30.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.321+0000 7fa4eec00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4e80d0c20 con 0x7fa4e815cda0 2024-03-12T17:11:30.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.322+0000 7fa4e7400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa4e80b5a30 0x7fa4e80b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:30.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.322+0000 7fa4e7400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa4e80b5a30 0x7fa4e80b5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36280/0 (socket says 172.21.15.52:36280) 2024-03-12T17:11:30.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.322+0000 7fa4e7400640 1 -- 172.21.15.52:0/2363954618 learned_addr learned my addr 172.21.15.52:0/2363954618 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:30.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.322+0000 7fa4e7e00640 1 --2- 172.21.15.52:0/2363954618 >> v2:172.21.15.102:3300/0 conn(0x7fa4e815cda0 0x7fa4e80cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:30.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.322+0000 7fa4e7400640 1 -- 172.21.15.52:0/2363954618 >> v2:172.21.15.102:3300/0 conn(0x7fa4e815cda0 msgr2=0x7fa4e80cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:30.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.322+0000 7fa4e7400640 1 --2- 172.21.15.52:0/2363954618 >> v2:172.21.15.102:3300/0 conn(0x7fa4e815cda0 0x7fa4e80cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:30.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.322+0000 7fa4e7400640 1 -- 172.21.15.52:0/2363954618 >> v1:172.21.15.102:6789/0 conn(0x7fa4e80b4350 legacy=0x7fa4e80d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:30.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.322+0000 7fa4e7400640 1 -- 172.21.15.52:0/2363954618 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e80d0f00 con 0x7fa4e80b5a30 2024-03-12T17:11:30.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.323+0000 7fa4e7400640 1 --2- 172.21.15.52:0/2363954618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4e80b5a30 0x7fa4e80b5e90 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa4e806ad20 tx=0x7fa4d002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3534afc9d72fe844 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:30.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.324+0000 7fa4e6a00640 1 -- 172.21.15.52:0/2363954618 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d002ca80 con 0x7fa4e80b5a30 2024-03-12T17:11:30.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.324+0000 7fa4e6a00640 1 -- 172.21.15.52:0/2363954618 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d002cbe0 con 0x7fa4e80b5a30 2024-03-12T17:11:30.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.324+0000 7fa4e6a00640 1 -- 172.21.15.52:0/2363954618 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d00343f0 con 0x7fa4e80b5a30 2024-03-12T17:11:30.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.324+0000 7fa4eec00640 1 -- 172.21.15.52:0/2363954618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4e80b5a30 msgr2=0x7fa4e80b5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:30.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.324+0000 7fa4eec00640 1 --2- 172.21.15.52:0/2363954618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4e80b5a30 0x7fa4e80b5e90 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa4e806ad20 tx=0x7fa4d002bee0 comp rx=0 tx=0).stop 2024-03-12T17:11:30.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.324+0000 7fa4eec00640 1 -- 172.21.15.52:0/2363954618 shutdown_connections 2024-03-12T17:11:30.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.324+0000 7fa4eec00640 1 --2- 172.21.15.52:0/2363954618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4e80b5a30 0x7fa4e80b5e90 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:30.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.324+0000 7fa4eec00640 1 --2- 172.21.15.52:0/2363954618 >> v2:172.21.15.102:3300/0 conn(0x7fa4e815cda0 0x7fa4e80cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:30.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.324+0000 7fa4eec00640 1 -- 172.21.15.52:0/2363954618 >> 172.21.15.52:0/2363954618 conn(0x7fa4e80cf350 msgr2=0x7fa4e80cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:30.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.325+0000 7fa4eec00640 1 -- 172.21.15.52:0/2363954618 shutdown_connections 2024-03-12T17:11:30.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.325+0000 7fa4eec00640 1 -- 172.21.15.52:0/2363954618 wait complete. 2024-03-12T17:11:30.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.326+0000 7fa4eec00640 1 Processor -- start 2024-03-12T17:11:30.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.326+0000 7fa4eec00640 1 -- start start 2024-03-12T17:11:30.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.327+0000 7fa4eec00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4e80b4350 0x7fa4e814afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:30.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.327+0000 7fa4eec00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa4e80b5a30 0x7fa4e81f5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:30.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.327+0000 7fa4eec00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4e815cda0 0x7fa4e81f59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:30.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.327+0000 7fa4eec00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e815fbe0 con 0x7fa4e80b4350 2024-03-12T17:11:30.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.327+0000 7fa4eec00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e814b4e0 con 0x7fa4e815cda0 2024-03-12T17:11:30.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.327+0000 7fa4eec00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e81f3810 con 0x7fa4e80b5a30 2024-03-12T17:11:30.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.328+0000 7fa4e7e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4e80b4350 0x7fa4e814afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:30.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.328+0000 7fa4e7e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4e80b4350 0x7fa4e814afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36290/0 (socket says 172.21.15.52:36290) 2024-03-12T17:11:30.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.328+0000 7fa4e7e00640 1 -- 172.21.15.52:0/3219270070 learned_addr learned my addr 172.21.15.52:0/3219270070 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:30.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.328+0000 7fa4ecc00640 1 --2- 172.21.15.52:0/3219270070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4e815cda0 0x7fa4e81f59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:30.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.328+0000 7fa4e7400640 1 --2- 172.21.15.52:0/3219270070 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa4e80b5a30 0x7fa4e81f5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:30.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.328+0000 7fa4e7e00640 1 -- 172.21.15.52:0/3219270070 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa4e80b5a30 msgr2=0x7fa4e81f5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:30.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.328+0000 7fa4e7e00640 1 --2- 172.21.15.52:0/3219270070 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa4e80b5a30 0x7fa4e81f5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:30.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.328+0000 7fa4e7e00640 1 -- 172.21.15.52:0/3219270070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4e815cda0 msgr2=0x7fa4e81f59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:30.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.328+0000 7fa4e7e00640 1 --2- 172.21.15.52:0/3219270070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4e815cda0 0x7fa4e81f59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:30.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.328+0000 7fa4e7e00640 1 -- 172.21.15.52:0/3219270070 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d0009b30 con 0x7fa4e80b4350 2024-03-12T17:11:30.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.329+0000 7fa4e7e00640 1 --2- 172.21.15.52:0/3219270070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4e80b4350 0x7fa4e814afa0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa4d800d5f0 tx=0x7fa4d800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:30.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.329+0000 7fa4dfe00640 1 -- 172.21.15.52:0/3219270070 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d80125f0 con 0x7fa4e80b4350 2024-03-12T17:11:30.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.329+0000 7fa4dfe00640 1 -- 172.21.15.52:0/3219270070 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d8012750 con 0x7fa4e80b4350 2024-03-12T17:11:30.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.329+0000 7fa4dfe00640 1 -- 172.21.15.52:0/3219270070 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8012b00 con 0x7fa4e80b4350 2024-03-12T17:11:30.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.329+0000 7fa4eec00640 1 -- 172.21.15.52:0/3219270070 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e815f900 con 0x7fa4e80b4350 2024-03-12T17:11:30.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.329+0000 7fa4eec00640 1 -- 172.21.15.52:0/3219270070 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e81f34a0 con 0x7fa4e80b4350 2024-03-12T17:11:30.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.330+0000 7fa4eec00640 1 -- 172.21.15.52:0/3219270070 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4e8073390 con 0x7fa4e80b4350 2024-03-12T17:11:30.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.331+0000 7fa4dfe00640 1 -- 172.21.15.52:0/3219270070 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa4d8012c60 con 0x7fa4e80b4350 2024-03-12T17:11:30.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.332+0000 7fa4dfe00640 1 --2- 172.21.15.52:0/3219270070 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa4c406ac00 0x7fa4c406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:30.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.332+0000 7fa4dfe00640 1 -- 172.21.15.52:0/3219270070 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa4d8089a10 con 0x7fa4e80b4350 2024-03-12T17:11:30.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.333+0000 7fa4e7400640 1 --2- 172.21.15.52:0/3219270070 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa4c406ac00 0x7fa4c406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:30.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.334+0000 7fa4e7400640 1 --2- 172.21.15.52:0/3219270070 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa4c406ac00 0x7fa4c406d0c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0035000 tx=0x7fa4d0030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:30.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.337+0000 7fa4dfe00640 1 -- 172.21.15.52:0/3219270070 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fa4d80549f0 con 0x7fa4e80b4350 2024-03-12T17:11:30.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.573+0000 7fa4eec00640 1 -- 172.21.15.52:0/3219270070 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4e80b3c40 con 0x7fa4e80b4350 2024-03-12T17:11:30.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.575+0000 7fa4dfe00640 1 -- 172.21.15.52:0/3219270070 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4734 (secure 0 0 0) 0x7fa4d8058010 con 0x7fa4e80b4350 2024-03-12T17:11:30.569 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:11:30.570 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6836/285561331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":285561331},{"type":"v1","addr":"172.21.15.102:6836","nonce":285561331}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4909,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1312095655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6835","nonce":1312095655},{"type":"v1","addr":"172.21.15.102:6837","nonce":1312095655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5337,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2451989690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2451989690},{"type":"v1","addr":"172.21.15.52:6833","nonce":2451989690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-12T17:08:10.679436+0000","modified":"2024-03-12T17:08:14.140830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5343},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5343":{"gid":5343,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.52:6835/543567396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":543567396},{"type":"v1","addr":"172.21.15.52:6835","nonce":543567396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-12T17:11:30.570 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 17 2024-03-12T17:11:30.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.577+0000 7fa4eec00640 1 -- 172.21.15.52:0/3219270070 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa4c406ac00 msgr2=0x7fa4c406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:30.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.577+0000 7fa4eec00640 1 --2- 172.21.15.52:0/3219270070 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa4c406ac00 0x7fa4c406d0c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0035000 tx=0x7fa4d0030000 comp rx=0 tx=0).stop 2024-03-12T17:11:30.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.578+0000 7fa4eec00640 1 -- 172.21.15.52:0/3219270070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4e80b4350 msgr2=0x7fa4e814afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:30.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.578+0000 7fa4eec00640 1 --2- 172.21.15.52:0/3219270070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4e80b4350 0x7fa4e814afa0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa4d800d5f0 tx=0x7fa4d800dab0 comp rx=0 tx=0).stop 2024-03-12T17:11:30.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.578+0000 7fa4eec00640 1 -- 172.21.15.52:0/3219270070 shutdown_connections 2024-03-12T17:11:30.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.578+0000 7fa4eec00640 1 --2- 172.21.15.52:0/3219270070 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa4c406ac00 0x7fa4c406d0c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:30.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.578+0000 7fa4eec00640 1 --2- 172.21.15.52:0/3219270070 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa4e80b5a30 0x7fa4e81f5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:30.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.578+0000 7fa4eec00640 1 --2- 172.21.15.52:0/3219270070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4e815cda0 0x7fa4e81f59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:30.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.578+0000 7fa4eec00640 1 --2- 172.21.15.52:0/3219270070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4e80b4350 0x7fa4e814afa0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:30.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.578+0000 7fa4eec00640 1 -- 172.21.15.52:0/3219270070 >> 172.21.15.52:0/3219270070 conn(0x7fa4e80cf350 msgr2=0x7fa4e80b3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:30.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.578+0000 7fa4eec00640 1 -- 172.21.15.52:0/3219270070 shutdown_connections 2024-03-12T17:11:30.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.578+0000 7fa4eec00640 1 -- 172.21.15.52:0/3219270070 wait complete. 2024-03-12T17:11:30.592 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T17:11:30.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.821+0000 7fe8e0a00640 1 Processor -- start 2024-03-12T17:11:30.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.822+0000 7fe8e0a00640 1 -- start start 2024-03-12T17:11:30.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.823+0000 7fe8e0a00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe8dc15b560 0x7fe8dc15b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:30.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.823+0000 7fe8e0a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe8dc0b39b0 0x7fe8dc0b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:30.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.823+0000 7fe8e0a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe8dc15e500 0x7fe8dc15e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:30.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.823+0000 7fe8e0a00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8dc0b4370 con 0x7fe8dc15b560 2024-03-12T17:11:30.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.823+0000 7fe8e0a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8dc06c6f0 con 0x7fe8dc15e500 2024-03-12T17:11:30.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.823+0000 7fe8e0a00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc06c9d0 con 0x7fe8dc0b39b0 2024-03-12T17:11:30.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.823+0000 7fe8d9e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe8dc15b560 0x7fe8dc15b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:30.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.823+0000 7fe8d9e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe8dc15b560 0x7fe8dc15b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36294/0 (socket says 172.21.15.52:36294) 2024-03-12T17:11:30.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.823+0000 7fe8d9e00640 1 -- 172.21.15.52:0/1634047707 learned_addr learned my addr 172.21.15.52:0/1634047707 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:30.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.824+0000 7fe8d9e00640 1 -- 172.21.15.52:0/1634047707 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe8dc0b39b0 msgr2=0x7fe8dc0b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:30.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.824+0000 7fe8d9e00640 1 --2- 172.21.15.52:0/1634047707 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe8dc0b39b0 0x7fe8dc0b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:30.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.824+0000 7fe8d9e00640 1 -- 172.21.15.52:0/1634047707 >> v2:172.21.15.102:3300/0 conn(0x7fe8dc15e500 msgr2=0x7fe8dc15e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:30.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.824+0000 7fe8d9e00640 1 --2- 172.21.15.52:0/1634047707 >> v2:172.21.15.102:3300/0 conn(0x7fe8dc15e500 0x7fe8dc15e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:30.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.824+0000 7fe8d9e00640 1 -- 172.21.15.52:0/1634047707 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8dc06ccb0 con 0x7fe8dc15b560 2024-03-12T17:11:30.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.825+0000 7fe8d9e00640 1 --2- 172.21.15.52:0/1634047707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8dc15b560 0x7fe8dc15b940 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc0cab40 tx=0x7fe8cc02b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a059b5a2c1e35a24 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:30.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.825+0000 7fe8d8a00640 1 -- 172.21.15.52:0/1634047707 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc031880 con 0x7fe8dc15b560 2024-03-12T17:11:30.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.825+0000 7fe8d8a00640 1 -- 172.21.15.52:0/1634047707 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8cc0319e0 con 0x7fe8dc15b560 2024-03-12T17:11:30.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.825+0000 7fe8d8a00640 1 -- 172.21.15.52:0/1634047707 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc031d70 con 0x7fe8dc15b560 2024-03-12T17:11:30.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.826+0000 7fe8e0a00640 1 -- 172.21.15.52:0/1634047707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8dc15b560 msgr2=0x7fe8dc15b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:30.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.826+0000 7fe8e0a00640 1 --2- 172.21.15.52:0/1634047707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8dc15b560 0x7fe8dc15b940 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc0cab40 tx=0x7fe8cc02b920 comp rx=0 tx=0).stop 2024-03-12T17:11:30.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.826+0000 7fe8e0a00640 1 -- 172.21.15.52:0/1634047707 shutdown_connections 2024-03-12T17:11:30.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.826+0000 7fe8e0a00640 1 --2- 172.21.15.52:0/1634047707 >> v2:172.21.15.102:3300/0 conn(0x7fe8dc15e500 0x7fe8dc15e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:30.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.826+0000 7fe8e0a00640 1 --2- 172.21.15.52:0/1634047707 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe8dc0b39b0 0x7fe8dc0b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:30.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.826+0000 7fe8e0a00640 1 --2- 172.21.15.52:0/1634047707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8dc15b560 0x7fe8dc15b940 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:30.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.826+0000 7fe8e0a00640 1 -- 172.21.15.52:0/1634047707 >> 172.21.15.52:0/1634047707 conn(0x7fe8dc06a3d0 msgr2=0x7fe8dc0b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:30.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.826+0000 7fe8e0a00640 1 -- 172.21.15.52:0/1634047707 shutdown_connections 2024-03-12T17:11:30.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.826+0000 7fe8e0a00640 1 -- 172.21.15.52:0/1634047707 wait complete. 2024-03-12T17:11:30.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.828+0000 7fe8e0a00640 1 Processor -- start 2024-03-12T17:11:30.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.828+0000 7fe8e0a00640 1 -- start start 2024-03-12T17:11:30.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.829+0000 7fe8e0a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8dc0b39b0 0x7fe8dc152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:30.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.829+0000 7fe8e0a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8dc15b560 0x7fe8dc13fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:30.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.829+0000 7fe8e0a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe8dc15e500 0x7fe8dc0ac6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:30.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.829+0000 7fe8e0a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc1527a0 con 0x7fe8dc0b39b0 2024-03-12T17:11:30.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.829+0000 7fe8e0a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc0b1aa0 con 0x7fe8dc15b560 2024-03-12T17:11:30.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.829+0000 7fe8e0a00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc0b1d80 con 0x7fe8dc15e500 2024-03-12T17:11:30.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.829+0000 7fe8d9e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8dc0b39b0 0x7fe8dc152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:30.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.829+0000 7fe8d9e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8dc0b39b0 0x7fe8dc152260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36298/0 (socket says 172.21.15.52:36298) 2024-03-12T17:11:30.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.829+0000 7fe8d9e00640 1 -- 172.21.15.52:0/1232090904 learned_addr learned my addr 172.21.15.52:0/1232090904 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:30.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.830+0000 7fe8da800640 1 --2- 172.21.15.52:0/1232090904 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe8dc15e500 0x7fe8dc0ac6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:30.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.830+0000 7fe8d9400640 1 --2- 172.21.15.52:0/1232090904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8dc15b560 0x7fe8dc13fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:30.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.830+0000 7fe8d9e00640 1 -- 172.21.15.52:0/1232090904 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe8dc15e500 msgr2=0x7fe8dc0ac6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:30.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.830+0000 7fe8d9e00640 1 --2- 172.21.15.52:0/1232090904 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe8dc15e500 0x7fe8dc0ac6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:30.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.830+0000 7fe8d9e00640 1 -- 172.21.15.52:0/1232090904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8dc15b560 msgr2=0x7fe8dc13fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:30.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.830+0000 7fe8d9e00640 1 --2- 172.21.15.52:0/1232090904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8dc15b560 0x7fe8dc13fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:30.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.830+0000 7fe8d9e00640 1 -- 172.21.15.52:0/1232090904 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8cc0094d0 con 0x7fe8dc0b39b0 2024-03-12T17:11:30.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.830+0000 7fe8d9400640 1 --2- 172.21.15.52:0/1232090904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8dc15b560 0x7fe8dc13fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:11:30.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.830+0000 7fe8d9e00640 1 --2- 172.21.15.52:0/1232090904 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8dc0b39b0 0x7fe8dc152260 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc000c00 tx=0x7fe8cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:30.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.830+0000 7fe8d2000640 1 -- 172.21.15.52:0/1232090904 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc03adf0 con 0x7fe8dc0b39b0 2024-03-12T17:11:30.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.830+0000 7fe8d2000640 1 -- 172.21.15.52:0/1232090904 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8cc0443f0 con 0x7fe8dc0b39b0 2024-03-12T17:11:30.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.831+0000 7fe8d2000640 1 -- 172.21.15.52:0/1232090904 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc0447a0 con 0x7fe8dc0b39b0 2024-03-12T17:11:30.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.831+0000 7fe8e0a00640 1 -- 172.21.15.52:0/1232090904 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8dc0b2060 con 0x7fe8dc0b39b0 2024-03-12T17:11:30.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.831+0000 7fe8e0a00640 1 -- 172.21.15.52:0/1232090904 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8dc0b2620 con 0x7fe8dc0b39b0 2024-03-12T17:11:30.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.832+0000 7fe8e0a00640 1 -- 172.21.15.52:0/1232090904 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8dc06ac20 con 0x7fe8dc0b39b0 2024-03-12T17:11:30.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.833+0000 7fe8d2000640 1 -- 172.21.15.52:0/1232090904 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe8cc04d450 con 0x7fe8dc0b39b0 2024-03-12T17:11:30.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.834+0000 7fe8d2000640 1 --2- 172.21.15.52:0/1232090904 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe8bc06ac30 0x7fe8bc06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:30.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.834+0000 7fe8d2000640 1 -- 172.21.15.52:0/1232090904 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe8cc0abd10 con 0x7fe8dc0b39b0 2024-03-12T17:11:30.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.834+0000 7fe8d9400640 1 --2- 172.21.15.52:0/1232090904 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe8bc06ac30 0x7fe8bc06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:30.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.836+0000 7fe8d9400640 1 --2- 172.21.15.52:0/1232090904 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe8bc06ac30 0x7fe8bc06d0f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8006fd0 tx=0x7fe8c8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:30.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:30.838+0000 7fe8d2000640 1 -- 172.21.15.52:0/1232090904 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fe8cc0b1020 con 0x7fe8dc0b39b0 2024-03-12T17:11:31.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.073+0000 7fe8e0a00640 1 -- 172.21.15.52:0/1232090904 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8dc06dd50 con 0x7fe8dc0b39b0 2024-03-12T17:11:31.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.074+0000 7fe8d2000640 1 -- 172.21.15.52:0/1232090904 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4734 (secure 0 0 0) 0x7fe8cc076cf0 con 0x7fe8dc0b39b0 2024-03-12T17:11:31.069 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:11:31.069 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6836/285561331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":285561331},{"type":"v1","addr":"172.21.15.102:6836","nonce":285561331}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4909,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1312095655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6835","nonce":1312095655},{"type":"v1","addr":"172.21.15.102:6837","nonce":1312095655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5337,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2451989690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2451989690},{"type":"v1","addr":"172.21.15.52:6833","nonce":2451989690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-12T17:08:10.679436+0000","modified":"2024-03-12T17:08:14.140830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5343},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5343":{"gid":5343,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.52:6835/543567396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":543567396},{"type":"v1","addr":"172.21.15.52:6835","nonce":543567396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-12T17:11:31.069 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 17 2024-03-12T17:11:31.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.077+0000 7fe8e0a00640 1 -- 172.21.15.52:0/1232090904 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe8bc06ac30 msgr2=0x7fe8bc06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:31.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.077+0000 7fe8e0a00640 1 --2- 172.21.15.52:0/1232090904 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe8bc06ac30 0x7fe8bc06d0f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8006fd0 tx=0x7fe8c8008040 comp rx=0 tx=0).stop 2024-03-12T17:11:31.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.077+0000 7fe8e0a00640 1 -- 172.21.15.52:0/1232090904 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8dc0b39b0 msgr2=0x7fe8dc152260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:31.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.077+0000 7fe8e0a00640 1 --2- 172.21.15.52:0/1232090904 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8dc0b39b0 0x7fe8dc152260 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc000c00 tx=0x7fe8cc0046c0 comp rx=0 tx=0).stop 2024-03-12T17:11:31.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.077+0000 7fe8e0a00640 1 -- 172.21.15.52:0/1232090904 shutdown_connections 2024-03-12T17:11:31.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.077+0000 7fe8e0a00640 1 --2- 172.21.15.52:0/1232090904 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe8bc06ac30 0x7fe8bc06d0f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:31.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.077+0000 7fe8e0a00640 1 --2- 172.21.15.52:0/1232090904 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe8dc15e500 0x7fe8dc0ac6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:31.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.077+0000 7fe8e0a00640 1 --2- 172.21.15.52:0/1232090904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8dc15b560 0x7fe8dc13fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:31.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.077+0000 7fe8e0a00640 1 --2- 172.21.15.52:0/1232090904 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8dc0b39b0 0x7fe8dc152260 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:31.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.077+0000 7fe8e0a00640 1 -- 172.21.15.52:0/1232090904 >> 172.21.15.52:0/1232090904 conn(0x7fe8dc06a3d0 msgr2=0x7fe8dc151510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:31.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.077+0000 7fe8e0a00640 1 -- 172.21.15.52:0/1232090904 shutdown_connections 2024-03-12T17:11:31.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.077+0000 7fe8e0a00640 1 -- 172.21.15.52:0/1232090904 wait complete. 2024-03-12T17:11:31.092 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:11:31.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.319+0000 7f41f2e00640 1 Processor -- start 2024-03-12T17:11:31.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.319+0000 7f41f2e00640 1 -- start start 2024-03-12T17:11:31.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.321+0000 7f41f2e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f41ec1571d0 0x7f41ec0b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:31.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.321+0000 7f41f2e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f41ec0cdc00 0x7f41ec15aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:31.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.321+0000 7f41f2e00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41ec1575b0 con 0x7f41ec15afe0 2024-03-12T17:11:31.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.321+0000 7f41f2e00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41ec159f20 con 0x7f41ec1571d0 2024-03-12T17:11:31.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.321+0000 7f41f2e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41ec15a200 con 0x7f41ec0cdc00 2024-03-12T17:11:31.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.321+0000 7f41f1a00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f41ec1571d0 0x7f41ec0b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:31.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.321+0000 7f41f1a00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f41ec1571d0 0x7f41ec0b35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36310/0 (socket says 172.21.15.52:36310) 2024-03-12T17:11:31.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.321+0000 7f41f2400640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f41ec15afe0 0x7f41ec0cae50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:40098/0 (socket says 172.21.15.52:40098) 2024-03-12T17:11:31.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.321+0000 7f41f1a00640 1 -- 172.21.15.52:0/4060108393 learned_addr learned my addr 172.21.15.52:0/4060108393 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:31.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.322+0000 7f41f1000640 1 --2- 172.21.15.52:0/4060108393 >> v2:172.21.15.102:3300/0 conn(0x7f41ec0cdc00 0x7f41ec15aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:31.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.322+0000 7f41f1a00640 1 -- 172.21.15.52:0/4060108393 >> v2:172.21.15.102:3300/0 conn(0x7f41ec0cdc00 msgr2=0x7f41ec15aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:31.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.322+0000 7f41f1a00640 1 --2- 172.21.15.52:0/4060108393 >> v2:172.21.15.102:3300/0 conn(0x7f41ec0cdc00 0x7f41ec15aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:31.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.322+0000 7f41f1a00640 1 -- 172.21.15.52:0/4060108393 >> v1:172.21.15.52:6789/0 conn(0x7f41ec15afe0 legacy=0x7f41ec0cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:31.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.322+0000 7f41ebe00640 1 -- 172.21.15.52:0/4060108393 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 169296285 0 0) 0x7f41ec1575b0 con 0x7f41ec15afe0 2024-03-12T17:11:31.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.322+0000 7f41f1a00640 1 -- 172.21.15.52:0/4060108393 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ec15a4e0 con 0x7f41ec1571d0 2024-03-12T17:11:31.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.323+0000 7f41f1a00640 1 --2- 172.21.15.52:0/4060108393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41ec1571d0 0x7f41ec0b35d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0cc150 tx=0x7f41e002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=889c3ff7f1e8bd5c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:31.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.324+0000 7f41ebe00640 1 -- 172.21.15.52:0/4060108393 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e0030860 con 0x7f41ec1571d0 2024-03-12T17:11:31.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.324+0000 7f41ebe00640 1 -- 172.21.15.52:0/4060108393 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e00309c0 con 0x7f41ec1571d0 2024-03-12T17:11:31.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.324+0000 7f41ebe00640 1 -- 172.21.15.52:0/4060108393 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e0030d50 con 0x7f41ec1571d0 2024-03-12T17:11:31.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.325+0000 7f41f2e00640 1 -- 172.21.15.52:0/4060108393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41ec1571d0 msgr2=0x7f41ec0b35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:31.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.325+0000 7f41f2e00640 1 --2- 172.21.15.52:0/4060108393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41ec1571d0 0x7f41ec0b35d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0cc150 tx=0x7f41e002b9a0 comp rx=0 tx=0).stop 2024-03-12T17:11:31.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.325+0000 7f41f2e00640 1 -- 172.21.15.52:0/4060108393 shutdown_connections 2024-03-12T17:11:31.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.325+0000 7f41f2e00640 1 --2- 172.21.15.52:0/4060108393 >> v2:172.21.15.102:3300/0 conn(0x7f41ec0cdc00 0x7f41ec15aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:31.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.325+0000 7f41f2e00640 1 --2- 172.21.15.52:0/4060108393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41ec1571d0 0x7f41ec0b35d0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:31.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.325+0000 7f41f2e00640 1 -- 172.21.15.52:0/4060108393 >> 172.21.15.52:0/4060108393 conn(0x7f41ec065df0 msgr2=0x7f41ec069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:31.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.325+0000 7f41f2e00640 1 -- 172.21.15.52:0/4060108393 shutdown_connections 2024-03-12T17:11:31.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.325+0000 7f41f2e00640 1 -- 172.21.15.52:0/4060108393 wait complete. 2024-03-12T17:11:31.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.327+0000 7f41f2e00640 1 Processor -- start 2024-03-12T17:11:31.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.327+0000 7f41f2e00640 1 -- start start 2024-03-12T17:11:31.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.328+0000 7f41f2e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41ec0cdc00 0x7f41ec13aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:31.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.328+0000 7f41f2e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f41ec1571d0 0x7f41ec137c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:31.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.328+0000 7f41f2e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41ec15afe0 0x7f41ec1f11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:31.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.328+0000 7f41f2e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec1381b0 con 0x7f41ec15afe0 2024-03-12T17:11:31.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.328+0000 7f41f2e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec13b180 con 0x7f41ec0cdc00 2024-03-12T17:11:31.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.328+0000 7f41f2e00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec13d250 con 0x7f41ec1571d0 2024-03-12T17:11:31.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.328+0000 7f41f2400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41ec15afe0 0x7f41ec1f11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:31.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.328+0000 7f41f2400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41ec15afe0 0x7f41ec1f11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36324/0 (socket says 172.21.15.52:36324) 2024-03-12T17:11:31.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.328+0000 7f41f2400640 1 -- 172.21.15.52:0/2898755976 learned_addr learned my addr 172.21.15.52:0/2898755976 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:31.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.328+0000 7f41f1a00640 1 --2- 172.21.15.52:0/2898755976 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41ec0cdc00 0x7f41ec13aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:31.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.328+0000 7f41f1000640 1 --2- 172.21.15.52:0/2898755976 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f41ec1571d0 0x7f41ec137c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:31.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.329+0000 7f41f2400640 1 -- 172.21.15.52:0/2898755976 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f41ec1571d0 msgr2=0x7f41ec137c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:31.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.329+0000 7f41f2400640 1 --2- 172.21.15.52:0/2898755976 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f41ec1571d0 0x7f41ec137c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:31.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.329+0000 7f41f2400640 1 -- 172.21.15.52:0/2898755976 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41ec0cdc00 msgr2=0x7f41ec13aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:31.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.329+0000 7f41f2400640 1 --2- 172.21.15.52:0/2898755976 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41ec0cdc00 0x7f41ec13aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:31.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.329+0000 7f41f2400640 1 -- 172.21.15.52:0/2898755976 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e00094d0 con 0x7f41ec15afe0 2024-03-12T17:11:31.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.329+0000 7f41f1a00640 1 --2- 172.21.15.52:0/2898755976 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41ec0cdc00 0x7f41ec13aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:11:31.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.329+0000 7f41f2400640 1 --2- 172.21.15.52:0/2898755976 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41ec15afe0 0x7f41ec1f11e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f41e400c6d0 tx=0x7f41e400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:31.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.329+0000 7f41e9600640 1 -- 172.21.15.52:0/2898755976 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e4004d60 con 0x7f41ec15afe0 2024-03-12T17:11:31.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.329+0000 7f41e9600640 1 -- 172.21.15.52:0/2898755976 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e400a640 con 0x7f41ec15afe0 2024-03-12T17:11:31.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.329+0000 7f41f2e00640 1 -- 172.21.15.52:0/2898755976 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41ec13d510 con 0x7f41ec15afe0 2024-03-12T17:11:31.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.329+0000 7f41e9600640 1 -- 172.21.15.52:0/2898755976 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e400a9f0 con 0x7f41ec15afe0 2024-03-12T17:11:31.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.329+0000 7f41f2e00640 1 -- 172.21.15.52:0/2898755976 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41ec1f1720 con 0x7f41ec15afe0 2024-03-12T17:11:31.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.331+0000 7f41f2e00640 1 -- 172.21.15.52:0/2898755976 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41ec13dd10 con 0x7f41ec15afe0 2024-03-12T17:11:31.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.331+0000 7f41e9600640 1 -- 172.21.15.52:0/2898755976 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f41e40070d0 con 0x7f41ec15afe0 2024-03-12T17:11:31.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.332+0000 7f41e9600640 1 --2- 172.21.15.52:0/2898755976 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f41cc06ac30 0x7f41cc06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:31.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.332+0000 7f41e9600640 1 -- 172.21.15.52:0/2898755976 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f41e4088aa0 con 0x7f41ec15afe0 2024-03-12T17:11:31.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.333+0000 7f41f1a00640 1 --2- 172.21.15.52:0/2898755976 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f41cc06ac30 0x7f41cc06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:31.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.334+0000 7f41f1a00640 1 --2- 172.21.15.52:0/2898755976 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f41cc06ac30 0x7f41cc06d0f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f41e0036000 tx=0x7f41e003b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:31.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.337+0000 7f41e9600640 1 -- 172.21.15.52:0/2898755976 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f41ec13dd10 con 0x7f41ec15afe0 2024-03-12T17:11:31.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.535+0000 7f41f2e00640 1 -- 172.21.15.52:0/2898755976 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41ec13dd10 con 0x7f41ec15afe0 2024-03-12T17:11:31.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.537+0000 7f41e9600640 1 -- 172.21.15.52:0/2898755976 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f41ec13dd10 con 0x7f41ec15afe0 2024-03-12T17:11:31.531 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:11:31.532 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":42,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:09:13.132892+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T17:06:19.218543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-12T17:08:09.114678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-12T17:08:09.691941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.733031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.775178+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.591522+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.820838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.829252+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.582481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.d","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:11:31.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.540+0000 7f41f2e00640 1 -- 172.21.15.52:0/2898755976 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f41cc06ac30 msgr2=0x7f41cc06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:31.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.540+0000 7f41f2e00640 1 --2- 172.21.15.52:0/2898755976 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f41cc06ac30 0x7f41cc06d0f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f41e0036000 tx=0x7f41e003b040 comp rx=0 tx=0).stop 2024-03-12T17:11:31.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.540+0000 7f41f2e00640 1 -- 172.21.15.52:0/2898755976 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41ec15afe0 msgr2=0x7f41ec1f11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:31.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.540+0000 7f41f2e00640 1 --2- 172.21.15.52:0/2898755976 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41ec15afe0 0x7f41ec1f11e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f41e400c6d0 tx=0x7f41e400cb90 comp rx=0 tx=0).stop 2024-03-12T17:11:31.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.540+0000 7f41f2e00640 1 -- 172.21.15.52:0/2898755976 shutdown_connections 2024-03-12T17:11:31.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.540+0000 7f41f2e00640 1 --2- 172.21.15.52:0/2898755976 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f41cc06ac30 0x7f41cc06d0f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:31.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.540+0000 7f41f2e00640 1 --2- 172.21.15.52:0/2898755976 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f41ec1571d0 0x7f41ec137c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:31.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.540+0000 7f41f2e00640 1 --2- 172.21.15.52:0/2898755976 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41ec15afe0 0x7f41ec1f11e0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:31.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.540+0000 7f41f2e00640 1 --2- 172.21.15.52:0/2898755976 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41ec0cdc00 0x7f41ec13aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:31.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.540+0000 7f41f2e00640 1 -- 172.21.15.52:0/2898755976 >> 172.21.15.52:0/2898755976 conn(0x7f41ec065df0 msgr2=0x7f41ec157760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:31.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.540+0000 7f41f2e00640 1 -- 172.21.15.52:0/2898755976 shutdown_connections 2024-03-12T17:11:31.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.540+0000 7f41f2e00640 1 -- 172.21.15.52:0/2898755976 wait complete. 2024-03-12T17:11:31.552 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-12T17:11:31.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.777+0000 7fc1eae00640 1 Processor -- start 2024-03-12T17:11:31.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.777+0000 7fc1eae00640 1 -- start start 2024-03-12T17:11:31.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.778+0000 7fc1eae00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc1e413faa0 0x7fc1e4160f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:31.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.778+0000 7fc1eae00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc1e4160800 0x7fc1e40b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:31.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.778+0000 7fc1eae00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1e40b2030 con 0x7fc1e40b1be0 2024-03-12T17:11:31.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.778+0000 7fc1eae00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1e40b38a0 con 0x7fc1e413faa0 2024-03-12T17:11:31.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.778+0000 7fc1eae00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1e40b3b80 con 0x7fc1e4160800 2024-03-12T17:11:31.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.778+0000 7fc1e3e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc1e413faa0 0x7fc1e4160f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:31.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.778+0000 7fc1e3e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc1e413faa0 0x7fc1e4160f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36340/0 (socket says 172.21.15.52:36340) 2024-03-12T17:11:31.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.778+0000 7fc1e3e00640 1 -- 172.21.15.52:0/565386904 learned_addr learned my addr 172.21.15.52:0/565386904 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:31.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.779+0000 7fc1e3e00640 1 -- 172.21.15.52:0/565386904 >> v2:172.21.15.102:3300/0 conn(0x7fc1e4160800 msgr2=0x7fc1e40b16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:31.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.779+0000 7fc1e3e00640 1 --2- 172.21.15.52:0/565386904 >> v2:172.21.15.102:3300/0 conn(0x7fc1e4160800 0x7fc1e40b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:31.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.779+0000 7fc1e3e00640 1 -- 172.21.15.52:0/565386904 >> v1:172.21.15.102:6789/0 conn(0x7fc1e40b1be0 legacy=0x7fc1e41422b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:31.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.779+0000 7fc1e3e00640 1 -- 172.21.15.52:0/565386904 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e40b3e60 con 0x7fc1e413faa0 2024-03-12T17:11:31.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.780+0000 7fc1e3e00640 1 --2- 172.21.15.52:0/565386904 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1e413faa0 0x7fc1e4160f30 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4009870 tx=0x7fc1d402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892c775367ec90e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:31.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.780+0000 7fc1e3400640 1 -- 172.21.15.52:0/565386904 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d4031870 con 0x7fc1e413faa0 2024-03-12T17:11:31.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.780+0000 7fc1e3400640 1 -- 172.21.15.52:0/565386904 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d40319d0 con 0x7fc1e413faa0 2024-03-12T17:11:31.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.780+0000 7fc1e3400640 1 -- 172.21.15.52:0/565386904 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d4031d60 con 0x7fc1e413faa0 2024-03-12T17:11:31.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.781+0000 7fc1eae00640 1 -- 172.21.15.52:0/565386904 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1e413faa0 msgr2=0x7fc1e4160f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:31.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.781+0000 7fc1eae00640 1 --2- 172.21.15.52:0/565386904 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1e413faa0 0x7fc1e4160f30 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4009870 tx=0x7fc1d402b910 comp rx=0 tx=0).stop 2024-03-12T17:11:31.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.781+0000 7fc1eae00640 1 -- 172.21.15.52:0/565386904 shutdown_connections 2024-03-12T17:11:31.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.781+0000 7fc1eae00640 1 --2- 172.21.15.52:0/565386904 >> v2:172.21.15.102:3300/0 conn(0x7fc1e4160800 0x7fc1e40b16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:31.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.781+0000 7fc1eae00640 1 --2- 172.21.15.52:0/565386904 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1e413faa0 0x7fc1e4160f30 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:31.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.781+0000 7fc1eae00640 1 -- 172.21.15.52:0/565386904 >> 172.21.15.52:0/565386904 conn(0x7fc1e406a3d0 msgr2=0x7fc1e40b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:31.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.781+0000 7fc1eae00640 1 -- 172.21.15.52:0/565386904 shutdown_connections 2024-03-12T17:11:31.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.781+0000 7fc1eae00640 1 -- 172.21.15.52:0/565386904 wait complete. 2024-03-12T17:11:31.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.783+0000 7fc1eae00640 1 Processor -- start 2024-03-12T17:11:31.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.783+0000 7fc1eae00640 1 -- start start 2024-03-12T17:11:31.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.784+0000 7fc1eae00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1e40b1be0 0x7fc1e41f54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:31.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.784+0000 7fc1eae00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1e413faa0 0x7fc1e4156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:31.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.784+0000 7fc1eae00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1e4160800 0x7fc1e41f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:31.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.784+0000 7fc1eae00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e41f5a20 con 0x7fc1e4160800 2024-03-12T17:11:31.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.784+0000 7fc1eae00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e4155000 con 0x7fc1e40b1be0 2024-03-12T17:11:31.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.784+0000 7fc1eae00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e41552e0 con 0x7fc1e413faa0 2024-03-12T17:11:31.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.785+0000 7fc1e8e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1e4160800 0x7fc1e41f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:31.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.785+0000 7fc1e8e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1e4160800 0x7fc1e41f5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36344/0 (socket says 172.21.15.52:36344) 2024-03-12T17:11:31.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.785+0000 7fc1e8e00640 1 -- 172.21.15.52:0/4163762781 learned_addr learned my addr 172.21.15.52:0/4163762781 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:31.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.785+0000 7fc1e3e00640 1 --2- 172.21.15.52:0/4163762781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1e40b1be0 0x7fc1e41f54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:31.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.785+0000 7fc1dbe00640 1 --2- 172.21.15.52:0/4163762781 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1e413faa0 0x7fc1e4156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:31.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.785+0000 7fc1e8e00640 1 -- 172.21.15.52:0/4163762781 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1e413faa0 msgr2=0x7fc1e4156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:31.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.785+0000 7fc1e8e00640 1 --2- 172.21.15.52:0/4163762781 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1e413faa0 0x7fc1e4156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:31.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.785+0000 7fc1e8e00640 1 -- 172.21.15.52:0/4163762781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1e40b1be0 msgr2=0x7fc1e41f54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:31.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.785+0000 7fc1e8e00640 1 --2- 172.21.15.52:0/4163762781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1e40b1be0 0x7fc1e41f54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:31.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.785+0000 7fc1e8e00640 1 -- 172.21.15.52:0/4163762781 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d40094d0 con 0x7fc1e4160800 2024-03-12T17:11:31.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.786+0000 7fc1e8e00640 1 --2- 172.21.15.52:0/4163762781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1e4160800 0x7fc1e41f5cc0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc00b990 tx=0x7fc1dc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:31.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.786+0000 7fc1e0c00640 1 -- 172.21.15.52:0/4163762781 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc00cba0 con 0x7fc1e4160800 2024-03-12T17:11:31.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.786+0000 7fc1e0c00640 1 -- 172.21.15.52:0/4163762781 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1dc00cd00 con 0x7fc1e4160800 2024-03-12T17:11:31.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.786+0000 7fc1e0c00640 1 -- 172.21.15.52:0/4163762781 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc014680 con 0x7fc1e4160800 2024-03-12T17:11:31.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.786+0000 7fc1eae00640 1 -- 172.21.15.52:0/4163762781 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1e41555c0 con 0x7fc1e4160800 2024-03-12T17:11:31.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.786+0000 7fc1eae00640 1 -- 172.21.15.52:0/4163762781 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1e4155b80 con 0x7fc1e4160800 2024-03-12T17:11:31.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.787+0000 7fc1eae00640 1 -- 172.21.15.52:0/4163762781 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1e41f3890 con 0x7fc1e4160800 2024-03-12T17:11:31.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.789+0000 7fc1e0c00640 1 -- 172.21.15.52:0/4163762781 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc1dc0147e0 con 0x7fc1e4160800 2024-03-12T17:11:31.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.790+0000 7fc1e0c00640 1 --2- 172.21.15.52:0/4163762781 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc1c006ac00 0x7fc1c006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:31.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.790+0000 7fc1e0c00640 1 -- 172.21.15.52:0/4163762781 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc1dc0893c0 con 0x7fc1e4160800 2024-03-12T17:11:31.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.790+0000 7fc1e3e00640 1 --2- 172.21.15.52:0/4163762781 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc1c006ac00 0x7fc1c006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:31.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.791+0000 7fc1e3e00640 1 --2- 172.21.15.52:0/4163762781 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc1c006ac00 0x7fc1c006d0c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4036000 tx=0x7fc1d403c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:31.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:31.795+0000 7fc1e0c00640 1 -- 172.21.15.52:0/4163762781 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fc1e41f3890 con 0x7fc1e4160800 2024-03-12T17:11:32.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.031+0000 7fc1eae00640 1 -- 172.21.15.52:0/4163762781 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc1e41f3890 con 0x7fc1e4160800 2024-03-12T17:11:32.028 INFO:tasks.ceph.mon.a.smithi052.stderr:2024-03-12T17:11:32.033+0000 7f074c400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-12T17:11:32.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.048+0000 7fc1e0c00640 1 -- 172.21.15.52:0/4163762781 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc1e41f3890 con 0x7fc1e4160800 2024-03-12T17:11:32.042 INFO:teuthology.orchestra.run.smithi052.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-12T17:11:32.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.051+0000 7fc1eae00640 1 -- 172.21.15.52:0/4163762781 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc1c006ac00 msgr2=0x7fc1c006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:32.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.051+0000 7fc1eae00640 1 --2- 172.21.15.52:0/4163762781 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc1c006ac00 0x7fc1c006d0c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4036000 tx=0x7fc1d403c040 comp rx=0 tx=0).stop 2024-03-12T17:11:32.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.051+0000 7fc1eae00640 1 -- 172.21.15.52:0/4163762781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1e4160800 msgr2=0x7fc1e41f5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:32.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.051+0000 7fc1eae00640 1 --2- 172.21.15.52:0/4163762781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1e4160800 0x7fc1e41f5cc0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc00b990 tx=0x7fc1dc00be50 comp rx=0 tx=0).stop 2024-03-12T17:11:32.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.051+0000 7fc1eae00640 1 -- 172.21.15.52:0/4163762781 shutdown_connections 2024-03-12T17:11:32.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.051+0000 7fc1eae00640 1 --2- 172.21.15.52:0/4163762781 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc1c006ac00 0x7fc1c006d0c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:32.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.051+0000 7fc1eae00640 1 --2- 172.21.15.52:0/4163762781 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1e413faa0 0x7fc1e4156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:32.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.051+0000 7fc1eae00640 1 --2- 172.21.15.52:0/4163762781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1e4160800 0x7fc1e41f5cc0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:32.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.051+0000 7fc1eae00640 1 --2- 172.21.15.52:0/4163762781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1e40b1be0 0x7fc1e41f54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:32.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.051+0000 7fc1eae00640 1 -- 172.21.15.52:0/4163762781 >> 172.21.15.52:0/4163762781 conn(0x7fc1e406a3d0 msgr2=0x7fc1e40ae950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:32.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.052+0000 7fc1eae00640 1 -- 172.21.15.52:0/4163762781 shutdown_connections 2024-03-12T17:11:32.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.052+0000 7fc1eae00640 1 -- 172.21.15.52:0/4163762781 wait complete. 2024-03-12T17:11:32.067 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-12T17:11:32.146 INFO:tasks.ceph.mds.a.smithi052.stdout:starting mds.a at 2024-03-12T17:11:32.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.277+0000 7ff8eb800640 1 Processor -- start 2024-03-12T17:11:32.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.277+0000 7ff8eb800640 1 -- start start 2024-03-12T17:11:32.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.278+0000 7ff8eb800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff8e4159830 0x7ff8e40cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:32.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.278+0000 7ff8eb800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff8e40ca7b0 0x7ff8e40cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:32.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.279+0000 7ff8eb800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e4159c10 con 0x7ff8e40ce930 2024-03-12T17:11:32.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.279+0000 7ff8eb800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8e406de50 con 0x7ff8e4159830 2024-03-12T17:11:32.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.279+0000 7ff8eb800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e40cd840 con 0x7ff8e40ca7b0 2024-03-12T17:11:32.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.279+0000 7ff8e9800640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7ff8e40ce930 0x7ff8e40b18d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.52:47692/0 (socket says 172.21.15.52:47692) 2024-03-12T17:11:32.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.279+0000 7ff8e8e00640 1 --2- 172.21.15.52:0/1183273304 >> v2:172.21.15.102:3300/0 conn(0x7ff8e4159830 0x7ff8e40cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:32.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.279+0000 7ff8e3e00640 1 --2- 172.21.15.52:0/1183273304 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff8e40ca7b0 0x7ff8e40cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:32.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.280+0000 7ff8e9800640 1 -- 172.21.15.52:0/1183273304 learned_addr learned my addr 172.21.15.52:0/1183273304 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:11:32.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.280+0000 7ff8e3e00640 1 -- 172.21.15.52:0/1183273304 >> v2:172.21.15.102:3300/0 conn(0x7ff8e4159830 msgr2=0x7ff8e40cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:32.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.280+0000 7ff8e3e00640 1 --2- 172.21.15.52:0/1183273304 >> v2:172.21.15.102:3300/0 conn(0x7ff8e4159830 0x7ff8e40cc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:32.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.280+0000 7ff8e3e00640 1 -- 172.21.15.52:0/1183273304 >> v1:172.21.15.102:6789/0 conn(0x7ff8e40ce930 legacy=0x7ff8e40b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:32.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.280+0000 7ff8e3e00640 1 -- 172.21.15.52:0/1183273304 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e40ccc30 con 0x7ff8e40ca7b0 2024-03-12T17:11:32.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.280+0000 7ff8e8e00640 1 --2- 172.21.15.52:0/1183273304 >> v2:172.21.15.102:3300/0 conn(0x7ff8e4159830 0x7ff8e40cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T17:11:32.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.282+0000 7ff8e3e00640 1 --2- 172.21.15.52:0/1183273304 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff8e40ca7b0 0x7ff8e40cdcd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc0097a0 tx=0x7ff8cc02b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fbe9fe938f12566 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:32.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.282+0000 7ff8e3400640 1 -- 172.21.15.52:0/1183273304 <== mon.2 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc030800 con 0x7ff8e40ca7b0 2024-03-12T17:11:32.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.282+0000 7ff8e3400640 1 -- 172.21.15.52:0/1183273304 <== mon.2 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8cc030960 con 0x7ff8e40ca7b0 2024-03-12T17:11:32.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.282+0000 7ff8e3400640 1 -- 172.21.15.52:0/1183273304 <== mon.2 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc030cf0 con 0x7ff8e40ca7b0 2024-03-12T17:11:32.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.283+0000 7ff8eb800640 1 -- 172.21.15.52:0/1183273304 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff8e40ca7b0 msgr2=0x7ff8e40cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:32.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.283+0000 7ff8eb800640 1 --2- 172.21.15.52:0/1183273304 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff8e40ca7b0 0x7ff8e40cdcd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc0097a0 tx=0x7ff8cc02b940 comp rx=0 tx=0).stop 2024-03-12T17:11:32.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.283+0000 7ff8eb800640 1 -- 172.21.15.52:0/1183273304 shutdown_connections 2024-03-12T17:11:32.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.283+0000 7ff8eb800640 1 --2- 172.21.15.52:0/1183273304 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff8e40ca7b0 0x7ff8e40cdcd0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:32.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.283+0000 7ff8eb800640 1 --2- 172.21.15.52:0/1183273304 >> v2:172.21.15.102:3300/0 conn(0x7ff8e4159830 0x7ff8e40cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:32.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.283+0000 7ff8eb800640 1 -- 172.21.15.52:0/1183273304 >> 172.21.15.52:0/1183273304 conn(0x7ff8e406a0b0 msgr2=0x7ff8e406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:32.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.283+0000 7ff8eb800640 1 -- 172.21.15.52:0/1183273304 shutdown_connections 2024-03-12T17:11:32.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.283+0000 7ff8eb800640 1 -- 172.21.15.52:0/1183273304 wait complete. 2024-03-12T17:11:32.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.285+0000 7ff8eb800640 1 Processor -- start 2024-03-12T17:11:32.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.285+0000 7ff8eb800640 1 -- start start 2024-03-12T17:11:32.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.286+0000 7ff8eb800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff8e40ca7b0 0x7ff8e40b4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:32.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.286+0000 7ff8eb800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8e40ce930 0x7ff8e414b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:32.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.286+0000 7ff8eb800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8e4159830 0x7ff8e40ac900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:32.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.286+0000 7ff8eb800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e40add90 con 0x7ff8e40ce930 2024-03-12T17:11:32.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.286+0000 7ff8eb800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e40ad7d0 con 0x7ff8e4159830 2024-03-12T17:11:32.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.286+0000 7ff8eb800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e40adab0 con 0x7ff8e40ca7b0 2024-03-12T17:11:32.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.286+0000 7ff8e3e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8e40ce930 0x7ff8e414b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:32.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.286+0000 7ff8e3e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8e40ce930 0x7ff8e414b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36358/0 (socket says 172.21.15.52:36358) 2024-03-12T17:11:32.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.286+0000 7ff8e3e00640 1 -- 172.21.15.52:0/3821708434 learned_addr learned my addr 172.21.15.52:0/3821708434 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:32.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.286+0000 7ff8e8e00640 1 --2- 172.21.15.52:0/3821708434 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff8e40ca7b0 0x7ff8e40b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:32.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.287+0000 7ff8e9800640 1 --2- 172.21.15.52:0/3821708434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8e4159830 0x7ff8e40ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:32.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.287+0000 7ff8e3e00640 1 -- 172.21.15.52:0/3821708434 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff8e40ca7b0 msgr2=0x7ff8e40b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:32.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.287+0000 7ff8e3e00640 1 --2- 172.21.15.52:0/3821708434 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff8e40ca7b0 0x7ff8e40b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:32.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.287+0000 7ff8e3e00640 1 -- 172.21.15.52:0/3821708434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8e4159830 msgr2=0x7ff8e40ac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:32.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.287+0000 7ff8e3e00640 1 --2- 172.21.15.52:0/3821708434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8e4159830 0x7ff8e40ac900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:32.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.287+0000 7ff8e3e00640 1 -- 172.21.15.52:0/3821708434 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d4009400 con 0x7ff8e40ce930 2024-03-12T17:11:32.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.287+0000 7ff8e9800640 1 --2- 172.21.15.52:0/3821708434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8e4159830 0x7ff8e40ac900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:11:32.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.287+0000 7ff8e3e00640 1 --2- 172.21.15.52:0/3821708434 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8e40ce930 0x7ff8e414b1e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc02bf50 tx=0x7ff8cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:32.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.288+0000 7ff8e0c00640 1 -- 172.21.15.52:0/3821708434 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc03e070 con 0x7ff8e40ce930 2024-03-12T17:11:32.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.288+0000 7ff8e0c00640 1 -- 172.21.15.52:0/3821708434 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8cc039a10 con 0x7ff8e40ce930 2024-03-12T17:11:32.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.288+0000 7ff8e0c00640 1 -- 172.21.15.52:0/3821708434 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc039da0 con 0x7ff8e40ce930 2024-03-12T17:11:32.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.288+0000 7ff8eb800640 1 -- 172.21.15.52:0/3821708434 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8cc009400 con 0x7ff8e40ce930 2024-03-12T17:11:32.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.288+0000 7ff8eb800640 1 -- 172.21.15.52:0/3821708434 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e40ae7f0 con 0x7ff8e40ce930 2024-03-12T17:11:32.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.289+0000 7ff8eb800640 1 -- 172.21.15.52:0/3821708434 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e40aea00 con 0x7ff8e40ce930 2024-03-12T17:11:32.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.290+0000 7ff8e0c00640 1 -- 172.21.15.52:0/3821708434 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff8cc0423f0 con 0x7ff8e40ce930 2024-03-12T17:11:32.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.291+0000 7ff8e0c00640 1 --2- 172.21.15.52:0/3821708434 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff8c006ac00 0x7ff8c006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:32.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.291+0000 7ff8e0c00640 1 -- 172.21.15.52:0/3821708434 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6144+0+0 (secure 0 0 0) 0x7ff8cc0ab0f0 con 0x7ff8e40ce930 2024-03-12T17:11:32.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.291+0000 7ff8e8e00640 1 --2- 172.21.15.52:0/3821708434 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff8c006ac00 0x7ff8c006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:32.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.293+0000 7ff8e8e00640 1 --2- 172.21.15.52:0/3821708434 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff8c006ac00 0x7ff8c006d0c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4009c60 tx=0x7ff8d4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:32.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.295+0000 7ff8e0c00640 1 -- 172.21.15.52:0/3821708434 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7ff8cc076130 con 0x7ff8e40ce930 2024-03-12T17:11:32.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:32.536+0000 7ff8eb800640 1 -- 172.21.15.52:0/3821708434 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff8e4073390 con 0x7ff8e40ce930 2024-03-12T17:11:33.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.053+0000 7ff8e0c00640 1 -- 172.21.15.52:0/3821708434 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff8cc079750 con 0x7ff8e40ce930 2024-03-12T17:11:33.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.058+0000 7ff8eb800640 1 -- 172.21.15.52:0/3821708434 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff8c006ac00 msgr2=0x7ff8c006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:33.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.058+0000 7ff8eb800640 1 --2- 172.21.15.52:0/3821708434 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff8c006ac00 0x7ff8c006d0c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4009c60 tx=0x7ff8d4008040 comp rx=0 tx=0).stop 2024-03-12T17:11:33.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.058+0000 7ff8eb800640 1 -- 172.21.15.52:0/3821708434 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8e40ce930 msgr2=0x7ff8e414b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:33.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.058+0000 7ff8eb800640 1 --2- 172.21.15.52:0/3821708434 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8e40ce930 0x7ff8e414b1e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc02bf50 tx=0x7ff8cc0046c0 comp rx=0 tx=0).stop 2024-03-12T17:11:33.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.058+0000 7ff8eb800640 1 -- 172.21.15.52:0/3821708434 shutdown_connections 2024-03-12T17:11:33.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.058+0000 7ff8eb800640 1 --2- 172.21.15.52:0/3821708434 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff8c006ac00 0x7ff8c006d0c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:33.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.058+0000 7ff8eb800640 1 --2- 172.21.15.52:0/3821708434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8e4159830 0x7ff8e40ac900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:33.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.058+0000 7ff8eb800640 1 --2- 172.21.15.52:0/3821708434 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8e40ce930 0x7ff8e414b1e0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:33.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.058+0000 7ff8eb800640 1 --2- 172.21.15.52:0/3821708434 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff8e40ca7b0 0x7ff8e40b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:33.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.058+0000 7ff8eb800640 1 -- 172.21.15.52:0/3821708434 >> 172.21.15.52:0/3821708434 conn(0x7ff8e406a0b0 msgr2=0x7ff8e41570a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:33.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.058+0000 7ff8eb800640 1 -- 172.21.15.52:0/3821708434 shutdown_connections 2024-03-12T17:11:33.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.058+0000 7ff8eb800640 1 -- 172.21.15.52:0/3821708434 wait complete. 2024-03-12T17:11:33.075 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-12T17:11:33.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.283+0000 7f5f8f200640 1 Processor -- start 2024-03-12T17:11:33.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.283+0000 7f5f8f200640 1 -- start start 2024-03-12T17:11:33.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.284+0000 7f5f8f200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5f9015a850 0x7f5f900cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:33.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.284+0000 7f5f8f200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f5f900b22b0 0x7f5f9015eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:33.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.285+0000 7f5f8f200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f9015e320 con 0x7f5f9015f020 2024-03-12T17:11:33.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.285+0000 7f5f8f200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f9015ac30 con 0x7f5f900b22b0 2024-03-12T17:11:33.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.285+0000 7f5f8f200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f900cda20 con 0x7f5f9015a850 2024-03-12T17:11:33.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.285+0000 7f5f8d400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f5f900b22b0 0x7f5f9015eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:33.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.285+0000 7f5f8d400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f5f900b22b0 0x7f5f9015eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36364/0 (socket says 172.21.15.52:36364) 2024-03-12T17:11:33.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.285+0000 7f5f8d400640 1 -- 172.21.15.52:0/820470082 learned_addr learned my addr 172.21.15.52:0/820470082 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:33.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.285+0000 7f5f8de00640 1 --2- 172.21.15.52:0/820470082 >> v2:172.21.15.102:3300/0 conn(0x7f5f9015a850 0x7f5f900cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:33.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.285+0000 7f5f8d400640 1 -- 172.21.15.52:0/820470082 >> v2:172.21.15.102:3300/0 conn(0x7f5f9015a850 msgr2=0x7f5f900cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:33.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.285+0000 7f5f8d400640 1 --2- 172.21.15.52:0/820470082 >> v2:172.21.15.102:3300/0 conn(0x7f5f9015a850 0x7f5f900cd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:33.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.285+0000 7f5f8d400640 1 -- 172.21.15.52:0/820470082 >> v1:172.21.15.102:6789/0 conn(0x7f5f9015f020 legacy=0x7f5f900cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:33.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.285+0000 7f5f8d400640 1 -- 172.21.15.52:0/820470082 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f900cdd00 con 0x7f5f900b22b0 2024-03-12T17:11:33.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.285+0000 7f5f8de00640 1 --2- 172.21.15.52:0/820470082 >> v2:172.21.15.102:3300/0 conn(0x7f5f9015a850 0x7f5f900cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:11:33.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.287+0000 7f5f8d400640 1 --2- 172.21.15.52:0/820470082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f900b22b0 0x7f5f9015eae0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f5f9006adb0 tx=0x7f5f8002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7744402e66941b86 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:33.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.287+0000 7f5f8ca00640 1 -- 172.21.15.52:0/820470082 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f80031870 con 0x7f5f900b22b0 2024-03-12T17:11:33.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.287+0000 7f5f8ca00640 1 -- 172.21.15.52:0/820470082 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f800319d0 con 0x7f5f900b22b0 2024-03-12T17:11:33.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.287+0000 7f5f8ca00640 1 -- 172.21.15.52:0/820470082 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f80031d60 con 0x7f5f900b22b0 2024-03-12T17:11:33.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.288+0000 7f5f8f200640 1 -- 172.21.15.52:0/820470082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f900b22b0 msgr2=0x7f5f9015eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:33.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.288+0000 7f5f8f200640 1 --2- 172.21.15.52:0/820470082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f900b22b0 0x7f5f9015eae0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f5f9006adb0 tx=0x7f5f8002b910 comp rx=0 tx=0).stop 2024-03-12T17:11:33.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.288+0000 7f5f8f200640 1 -- 172.21.15.52:0/820470082 shutdown_connections 2024-03-12T17:11:33.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.288+0000 7f5f8f200640 1 --2- 172.21.15.52:0/820470082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f900b22b0 0x7f5f9015eae0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:33.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.288+0000 7f5f8f200640 1 --2- 172.21.15.52:0/820470082 >> v2:172.21.15.102:3300/0 conn(0x7f5f9015a850 0x7f5f900cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:33.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.288+0000 7f5f8f200640 1 -- 172.21.15.52:0/820470082 >> 172.21.15.52:0/820470082 conn(0x7f5f900b01b0 msgr2=0x7f5f900b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:33.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.288+0000 7f5f8f200640 1 -- 172.21.15.52:0/820470082 shutdown_connections 2024-03-12T17:11:33.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.288+0000 7f5f8f200640 1 -- 172.21.15.52:0/820470082 wait complete. 2024-03-12T17:11:33.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.291+0000 7f5f8f200640 1 Processor -- start 2024-03-12T17:11:33.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.291+0000 7f5f8f200640 1 -- start start 2024-03-12T17:11:33.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.292+0000 7f5f8f200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f900b22b0 0x7f5f90156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:33.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.292+0000 7f5f8f200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f9015a850 0x7f5f90069c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:33.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.292+0000 7f5f8f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f9015f020 0x7f5f9006a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:33.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.292+0000 7f5f8f200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f90156fc0 con 0x7f5f900b22b0 2024-03-12T17:11:33.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.292+0000 7f5f8f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f901565f0 con 0x7f5f9015f020 2024-03-12T17:11:33.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.292+0000 7f5f8f200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f901559e0 con 0x7f5f9015a850 2024-03-12T17:11:33.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.292+0000 7f5f8de00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f900b22b0 0x7f5f90156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:33.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.292+0000 7f5f8de00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f900b22b0 0x7f5f90156a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36374/0 (socket says 172.21.15.52:36374) 2024-03-12T17:11:33.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.292+0000 7f5f8de00640 1 -- 172.21.15.52:0/225361467 learned_addr learned my addr 172.21.15.52:0/225361467 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:33.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.292+0000 7f5f8e800640 1 --2- 172.21.15.52:0/225361467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f9015f020 0x7f5f9006a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:33.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.292+0000 7f5f8d400640 1 --2- 172.21.15.52:0/225361467 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f9015a850 0x7f5f90069c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:33.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.292+0000 7f5f8de00640 1 -- 172.21.15.52:0/225361467 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f9015a850 msgr2=0x7f5f90069c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:33.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.292+0000 7f5f8de00640 1 --2- 172.21.15.52:0/225361467 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f9015a850 0x7f5f90069c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:33.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.292+0000 7f5f8de00640 1 -- 172.21.15.52:0/225361467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f9015f020 msgr2=0x7f5f9006a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:33.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.292+0000 7f5f8de00640 1 --2- 172.21.15.52:0/225361467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f9015f020 0x7f5f9006a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:33.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.292+0000 7f5f8de00640 1 -- 172.21.15.52:0/225361467 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f800094d0 con 0x7f5f900b22b0 2024-03-12T17:11:33.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.292+0000 7f5f8e800640 1 --2- 172.21.15.52:0/225361467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f9015f020 0x7f5f9006a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:11:33.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.292+0000 7f5f8d400640 1 --2- 172.21.15.52:0/225361467 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f9015a850 0x7f5f90069c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:11:33.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.292+0000 7f5f8de00640 1 --2- 172.21.15.52:0/225361467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f900b22b0 0x7f5f90156a80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5f7800c7f0 tx=0x7f5f7800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:33.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.292+0000 7f5f86000640 1 -- 172.21.15.52:0/225361467 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f78007a30 con 0x7f5f900b22b0 2024-03-12T17:11:33.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.292+0000 7f5f86000640 1 -- 172.21.15.52:0/225361467 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f78007b90 con 0x7f5f900b22b0 2024-03-12T17:11:33.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.292+0000 7f5f86000640 1 -- 172.21.15.52:0/225361467 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7800ce40 con 0x7f5f900b22b0 2024-03-12T17:11:33.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.292+0000 7f5f8f200640 1 -- 172.21.15.52:0/225361467 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f90155cc0 con 0x7f5f900b22b0 2024-03-12T17:11:33.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.293+0000 7f5f8f200640 1 -- 172.21.15.52:0/225361467 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f90156280 con 0x7f5f900b22b0 2024-03-12T17:11:33.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.294+0000 7f5f8f200640 1 -- 172.21.15.52:0/225361467 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f9004e030 con 0x7f5f900b22b0 2024-03-12T17:11:33.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.295+0000 7f5f86000640 1 -- 172.21.15.52:0/225361467 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5f780113f0 con 0x7f5f900b22b0 2024-03-12T17:11:33.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.296+0000 7f5f86000640 1 --2- 172.21.15.52:0/225361467 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f5f6c06ac00 0x7f5f6c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:33.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.296+0000 7f5f86000640 1 -- 172.21.15.52:0/225361467 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f5f78089650 con 0x7f5f900b22b0 2024-03-12T17:11:33.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.298+0000 7f5f8d400640 1 --2- 172.21.15.52:0/225361467 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f5f6c06ac00 0x7f5f6c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:33.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.298+0000 7f5f8d400640 1 --2- 172.21.15.52:0/225361467 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f5f6c06ac00 0x7f5f6c06d0c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5f80036000 tx=0x7f5f8003c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:33.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.301+0000 7f5f86000640 1 -- 172.21.15.52:0/225361467 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f5f78054710 con 0x7f5f900b22b0 2024-03-12T17:11:33.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:33.492+0000 7f5f8f200640 1 -- 172.21.15.52:0/225361467 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5f900ccaa0 con 0x7f5f900b22b0 2024-03-12T17:11:34.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.216+0000 7f5f86000640 1 -- 172.21.15.52:0/225361467 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v44) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5f78057d30 con 0x7f5f900b22b0 2024-03-12T17:11:34.211 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_metadata' removed 2024-03-12T17:11:34.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.222+0000 7f5f8f200640 1 -- 172.21.15.52:0/225361467 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f5f6c06ac00 msgr2=0x7f5f6c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:34.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.222+0000 7f5f8f200640 1 --2- 172.21.15.52:0/225361467 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f5f6c06ac00 0x7f5f6c06d0c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5f80036000 tx=0x7f5f8003c040 comp rx=0 tx=0).stop 2024-03-12T17:11:34.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.222+0000 7f5f8f200640 1 -- 172.21.15.52:0/225361467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f900b22b0 msgr2=0x7f5f90156a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:34.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.222+0000 7f5f8f200640 1 --2- 172.21.15.52:0/225361467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f900b22b0 0x7f5f90156a80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5f7800c7f0 tx=0x7f5f7800cbf0 comp rx=0 tx=0).stop 2024-03-12T17:11:34.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.222+0000 7f5f8f200640 1 -- 172.21.15.52:0/225361467 shutdown_connections 2024-03-12T17:11:34.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.222+0000 7f5f8f200640 1 --2- 172.21.15.52:0/225361467 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f5f6c06ac00 0x7f5f6c06d0c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:34.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.222+0000 7f5f8f200640 1 --2- 172.21.15.52:0/225361467 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f9015a850 0x7f5f90069c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:34.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.222+0000 7f5f8f200640 1 --2- 172.21.15.52:0/225361467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f9015f020 0x7f5f9006a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:34.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.222+0000 7f5f8f200640 1 --2- 172.21.15.52:0/225361467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f900b22b0 0x7f5f90156a80 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:34.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.222+0000 7f5f8f200640 1 -- 172.21.15.52:0/225361467 >> 172.21.15.52:0/225361467 conn(0x7f5f900b01b0 msgr2=0x7f5f900b0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:34.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.223+0000 7f5f8f200640 1 -- 172.21.15.52:0/225361467 shutdown_connections 2024-03-12T17:11:34.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.223+0000 7f5f8f200640 1 -- 172.21.15.52:0/225361467 wait complete. 2024-03-12T17:11:34.238 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-12T17:11:34.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.473+0000 7f2ba5c00640 1 Processor -- start 2024-03-12T17:11:34.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.474+0000 7f2ba5c00640 1 -- start start 2024-03-12T17:11:34.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.475+0000 7f2ba5c00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2ba015fe90 0x7f2ba00d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:34.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.475+0000 7f2ba5c00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ba00b8470 0x7f2ba00b4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:34.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.475+0000 7f2ba5c00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ba0160270 con 0x7f2ba00b4770 2024-03-12T17:11:34.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.475+0000 7f2ba5c00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ba00b4c20 con 0x7f2ba015fe90 2024-03-12T17:11:34.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.475+0000 7f2ba5c00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba00d24b0 con 0x7f2ba00b8470 2024-03-12T17:11:34.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.475+0000 7f2b9fe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2ba015fe90 0x7f2ba00d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:34.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.475+0000 7f2b9fe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2ba015fe90 0x7f2ba00d0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36376/0 (socket says 172.21.15.52:36376) 2024-03-12T17:11:34.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.475+0000 7f2b9fe00640 1 -- 172.21.15.52:0/3804928601 learned_addr learned my addr 172.21.15.52:0/3804928601 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:34.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.475+0000 7f2b9f400640 1 --2- 172.21.15.52:0/3804928601 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ba00b8470 0x7f2ba00b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:34.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.475+0000 7f2b9fe00640 1 -- 172.21.15.52:0/3804928601 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ba00b8470 msgr2=0x7f2ba00b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:34.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.476+0000 7f2b9fe00640 1 --2- 172.21.15.52:0/3804928601 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ba00b8470 0x7f2ba00b4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:34.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.476+0000 7f2b9fe00640 1 -- 172.21.15.52:0/3804928601 >> v1:172.21.15.102:6789/0 conn(0x7f2ba00b4770 legacy=0x7f2ba006ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:34.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.476+0000 7f2b9fe00640 1 -- 172.21.15.52:0/3804928601 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba00d1410 con 0x7f2ba015fe90 2024-03-12T17:11:34.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.477+0000 7f2b9fe00640 1 --2- 172.21.15.52:0/3804928601 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ba015fe90 0x7f2ba00d0b10 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2b90009870 tx=0x7f2b9002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca09ae71db1422af server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:34.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.477+0000 7f2b9ea00640 1 -- 172.21.15.52:0/3804928601 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b90030840 con 0x7f2ba015fe90 2024-03-12T17:11:34.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.477+0000 7f2b9ea00640 1 -- 172.21.15.52:0/3804928601 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b900309a0 con 0x7f2ba015fe90 2024-03-12T17:11:34.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.477+0000 7f2b9ea00640 1 -- 172.21.15.52:0/3804928601 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b90030d30 con 0x7f2ba015fe90 2024-03-12T17:11:34.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.478+0000 7f2ba5c00640 1 -- 172.21.15.52:0/3804928601 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ba015fe90 msgr2=0x7f2ba00d0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:34.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.478+0000 7f2ba5c00640 1 --2- 172.21.15.52:0/3804928601 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ba015fe90 0x7f2ba00d0b10 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2b90009870 tx=0x7f2b9002b980 comp rx=0 tx=0).stop 2024-03-12T17:11:34.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.478+0000 7f2ba5c00640 1 -- 172.21.15.52:0/3804928601 shutdown_connections 2024-03-12T17:11:34.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.478+0000 7f2ba5c00640 1 --2- 172.21.15.52:0/3804928601 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ba00b8470 0x7f2ba00b4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:34.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.478+0000 7f2ba5c00640 1 --2- 172.21.15.52:0/3804928601 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ba015fe90 0x7f2ba00d0b10 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:34.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.478+0000 7f2ba5c00640 1 -- 172.21.15.52:0/3804928601 >> 172.21.15.52:0/3804928601 conn(0x7f2ba00cf230 msgr2=0x7f2ba00cf640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:34.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.478+0000 7f2ba5c00640 1 -- 172.21.15.52:0/3804928601 shutdown_connections 2024-03-12T17:11:34.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.478+0000 7f2ba5c00640 1 -- 172.21.15.52:0/3804928601 wait complete. 2024-03-12T17:11:34.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.480+0000 7f2ba5c00640 1 Processor -- start 2024-03-12T17:11:34.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.480+0000 7f2ba5c00640 1 -- start start 2024-03-12T17:11:34.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.481+0000 7f2ba5c00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ba00b4770 0x7f2ba01515c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:34.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.481+0000 7f2ba5c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ba00b8470 0x7f2ba0158280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:34.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.481+0000 7f2ba5c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ba015fe90 0x7f2ba00c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:34.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.481+0000 7f2ba5c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba00cb8f0 con 0x7f2ba00b8470 2024-03-12T17:11:34.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.481+0000 7f2ba5c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba00cbbd0 con 0x7f2ba015fe90 2024-03-12T17:11:34.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.481+0000 7f2ba5c00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba00cbeb0 con 0x7f2ba00b4770 2024-03-12T17:11:34.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.481+0000 7f2b9f400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ba00b8470 0x7f2ba0158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:34.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.481+0000 7f2b9f400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ba00b8470 0x7f2ba0158280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36384/0 (socket says 172.21.15.52:36384) 2024-03-12T17:11:34.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.481+0000 7f2b9f400640 1 -- 172.21.15.52:0/474349211 learned_addr learned my addr 172.21.15.52:0/474349211 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:34.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.481+0000 7f2b9f400640 1 -- 172.21.15.52:0/474349211 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ba00b4770 msgr2=0x7f2ba01515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:34.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.482+0000 7f2b9fe00640 1 --2- 172.21.15.52:0/474349211 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ba00b4770 0x7f2ba01515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:34.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.482+0000 7f2ba5200640 1 --2- 172.21.15.52:0/474349211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ba015fe90 0x7f2ba00c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:34.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.482+0000 7f2b9f400640 1 --2- 172.21.15.52:0/474349211 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ba00b4770 0x7f2ba01515c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:34.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.482+0000 7f2b9f400640 1 -- 172.21.15.52:0/474349211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ba015fe90 msgr2=0x7f2ba00c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:34.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.482+0000 7f2b9f400640 1 --2- 172.21.15.52:0/474349211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ba015fe90 0x7f2ba00c7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:34.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.482+0000 7f2b9f400640 1 -- 172.21.15.52:0/474349211 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b900094d0 con 0x7f2ba00b8470 2024-03-12T17:11:34.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.482+0000 7f2ba5200640 1 --2- 172.21.15.52:0/474349211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ba015fe90 0x7f2ba00c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:11:34.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.482+0000 7f2b9f400640 1 --2- 172.21.15.52:0/474349211 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ba00b8470 0x7f2ba0158280 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2b8800ce10 tx=0x7f2b8800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:34.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.482+0000 7f2b97e00640 1 -- 172.21.15.52:0/474349211 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b88003b90 con 0x7f2ba00b8470 2024-03-12T17:11:34.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.482+0000 7f2b97e00640 1 -- 172.21.15.52:0/474349211 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b88003cf0 con 0x7f2ba00b8470 2024-03-12T17:11:34.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.482+0000 7f2b97e00640 1 -- 172.21.15.52:0/474349211 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b88003ea0 con 0x7f2ba00b8470 2024-03-12T17:11:34.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.482+0000 7f2ba5c00640 1 -- 172.21.15.52:0/474349211 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ba00cc190 con 0x7f2ba00b8470 2024-03-12T17:11:34.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.482+0000 7f2ba5c00640 1 -- 172.21.15.52:0/474349211 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ba0151b00 con 0x7f2ba00b8470 2024-03-12T17:11:34.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.483+0000 7f2ba5c00640 1 -- 172.21.15.52:0/474349211 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ba00cc6b0 con 0x7f2ba00b8470 2024-03-12T17:11:34.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.485+0000 7f2b97e00640 1 -- 172.21.15.52:0/474349211 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2b88004240 con 0x7f2ba00b8470 2024-03-12T17:11:34.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.485+0000 7f2b97e00640 1 --2- 172.21.15.52:0/474349211 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2b7c06ac00 0x7f2b7c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:34.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.486+0000 7f2b97e00640 1 -- 172.21.15.52:0/474349211 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f2b88012070 con 0x7f2ba00b8470 2024-03-12T17:11:34.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.486+0000 7f2b9fe00640 1 --2- 172.21.15.52:0/474349211 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2b7c06ac00 0x7f2b7c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:34.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.487+0000 7f2b9fe00640 1 --2- 172.21.15.52:0/474349211 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2b7c06ac00 0x7f2b7c06d0c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2b90036000 tx=0x7f2b9003b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:34.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.490+0000 7f2b97e00640 1 -- 172.21.15.52:0/474349211 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f2ba00cc6b0 con 0x7f2ba00b8470 2024-03-12T17:11:34.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:34.686+0000 7f2ba5c00640 1 -- 172.21.15.52:0/474349211 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2ba00cc6b0 con 0x7f2ba00b8470 2024-03-12T17:11:35.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.220+0000 7f2b97e00640 1 -- 172.21.15.52:0/474349211 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v45) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2ba00cc6b0 con 0x7f2ba00b8470 2024-03-12T17:11:35.214 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_data' removed 2024-03-12T17:11:35.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.225+0000 7f2ba5c00640 1 -- 172.21.15.52:0/474349211 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2b7c06ac00 msgr2=0x7f2b7c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:35.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.225+0000 7f2ba5c00640 1 --2- 172.21.15.52:0/474349211 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2b7c06ac00 0x7f2b7c06d0c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2b90036000 tx=0x7f2b9003b040 comp rx=0 tx=0).stop 2024-03-12T17:11:35.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.225+0000 7f2ba5c00640 1 -- 172.21.15.52:0/474349211 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ba00b8470 msgr2=0x7f2ba0158280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:35.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.225+0000 7f2ba5c00640 1 --2- 172.21.15.52:0/474349211 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ba00b8470 0x7f2ba0158280 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2b8800ce10 tx=0x7f2b8800a660 comp rx=0 tx=0).stop 2024-03-12T17:11:35.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.225+0000 7f2ba5c00640 1 -- 172.21.15.52:0/474349211 shutdown_connections 2024-03-12T17:11:35.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.225+0000 7f2ba5c00640 1 --2- 172.21.15.52:0/474349211 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2b7c06ac00 0x7f2b7c06d0c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:35.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.225+0000 7f2ba5c00640 1 --2- 172.21.15.52:0/474349211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ba015fe90 0x7f2ba00c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:35.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.225+0000 7f2ba5c00640 1 --2- 172.21.15.52:0/474349211 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ba00b8470 0x7f2ba0158280 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:35.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.225+0000 7f2ba5c00640 1 --2- 172.21.15.52:0/474349211 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ba00b4770 0x7f2ba01515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:35.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.225+0000 7f2ba5c00640 1 -- 172.21.15.52:0/474349211 >> 172.21.15.52:0/474349211 conn(0x7f2ba00cf230 msgr2=0x7f2ba015f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:35.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.225+0000 7f2ba5c00640 1 -- 172.21.15.52:0/474349211 shutdown_connections 2024-03-12T17:11:35.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.225+0000 7f2ba5c00640 1 -- 172.21.15.52:0/474349211 wait complete. 2024-03-12T17:11:35.242 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-12T17:11:35.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.463+0000 7f9fb5200640 1 Processor -- start 2024-03-12T17:11:35.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.463+0000 7f9fb5200640 1 -- start start 2024-03-12T17:11:35.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.465+0000 7f9fb5200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9fb00cf5c0 0x7f9fb00b8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:35.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.465+0000 7f9fb5200640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb015ce90 con 0x7f9fb015cab0 2024-03-12T17:11:35.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.465+0000 7f9fb5200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb00cddf0 con 0x7f9fb00d0c20 2024-03-12T17:11:35.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.465+0000 7f9fb5200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fb00cd570 con 0x7f9fb00cf5c0 2024-03-12T17:11:35.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.465+0000 7f9faf400640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f9fb015cab0 0x7f9fb00ceeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51910/0 (socket says 172.21.15.52:51910) 2024-03-12T17:11:35.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.465+0000 7f9faf400640 1 -- 172.21.15.52:0/1096187155 learned_addr learned my addr 172.21.15.52:0/1096187155 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:11:35.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.465+0000 7f9faea00640 1 --2- 172.21.15.52:0/1096187155 >> v2:172.21.15.102:3300/0 conn(0x7f9fb00cf5c0 0x7f9fb00b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:35.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.465+0000 7f9fae000640 1 -- 172.21.15.52:0/1096187155 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3393659698 0 0) 0x7f9fb015ce90 con 0x7f9fb015cab0 2024-03-12T17:11:35.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.466+0000 7f9fae000640 1 -- 172.21.15.52:0/1096187155 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f94003520 con 0x7f9fb015cab0 2024-03-12T17:11:35.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.466+0000 7f9fae000640 1 -- 172.21.15.52:0/1096187155 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3337689237 0 0) 0x7f9f94003520 con 0x7f9fb015cab0 2024-03-12T17:11:35.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.466+0000 7f9fae000640 1 -- 172.21.15.52:0/1096187155 >> v2:172.21.15.102:3300/0 conn(0x7f9fb00cf5c0 msgr2=0x7f9fb00b8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:35.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.466+0000 7f9fae000640 1 --2- 172.21.15.52:0/1096187155 >> v2:172.21.15.102:3300/0 conn(0x7f9fb00cf5c0 0x7f9fb00b8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:35.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.466+0000 7f9fae000640 1 -- 172.21.15.52:0/1096187155 >> v1:172.21.15.102:6789/0 conn(0x7f9fb00d0c20 legacy=0x7f9fb00d1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:35.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.466+0000 7f9fae000640 1 -- 172.21.15.52:0/1096187155 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb0161370 con 0x7f9fb015cab0 2024-03-12T17:11:35.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.466+0000 7f9fae000640 1 -- 172.21.15.52:0/1096187155 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f9fa0001670 con 0x7f9fb015cab0 2024-03-12T17:11:35.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.466+0000 7f9fae000640 1 -- 172.21.15.52:0/1096187155 >> v1:172.21.15.52:6789/0 conn(0x7f9fb015cab0 legacy=0x7f9fb00ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:35.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.466+0000 7f9fae000640 1 --2- 172.21.15.52:0/1096187155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f940039c0 0x7f9f94003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:35.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.466+0000 7f9fae000640 1 --2- 172.21.15.52:0/1096187155 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9f940047c0 0x7f9f94004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:35.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.466+0000 7f9fae000640 1 --2- 172.21.15.52:0/1096187155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f94005180 0x7f9f9400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:35.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.466+0000 7f9fae000640 1 -- 172.21.15.52:0/1096187155 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb00cd570 con 0x7f9f940039c0 2024-03-12T17:11:35.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.466+0000 7f9fae000640 1 -- 172.21.15.52:0/1096187155 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9400c050 con 0x7f9f94005180 2024-03-12T17:11:35.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.466+0000 7f9fae000640 1 -- 172.21.15.52:0/1096187155 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9400c210 con 0x7f9f940047c0 2024-03-12T17:11:35.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.466+0000 7f9faf400640 1 --2- 172.21.15.52:0/1096187155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f940039c0 0x7f9f94003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:35.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.467+0000 7f9fafe00640 1 --2- 172.21.15.52:0/1096187155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f94005180 0x7f9f9400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:35.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.467+0000 7f9faea00640 1 --2- 172.21.15.52:0/1096187155 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9f940047c0 0x7f9f94004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:35.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.467+0000 7f9faf400640 1 -- 172.21.15.52:0/1096187155 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9f940047c0 msgr2=0x7f9f94004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:35.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.467+0000 7f9faf400640 1 --2- 172.21.15.52:0/1096187155 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9f940047c0 0x7f9f94004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:35.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.467+0000 7f9faf400640 1 -- 172.21.15.52:0/1096187155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f94005180 msgr2=0x7f9f9400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:35.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.467+0000 7f9faf400640 1 --2- 172.21.15.52:0/1096187155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f94005180 0x7f9f9400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:35.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.467+0000 7f9faf400640 1 -- 172.21.15.52:0/1096187155 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f94003520 con 0x7f9f940039c0 2024-03-12T17:11:35.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.467+0000 7f9faea00640 1 --2- 172.21.15.52:0/1096187155 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9f940047c0 0x7f9f94004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:11:35.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.467+0000 7f9fafe00640 1 --2- 172.21.15.52:0/1096187155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f94005180 0x7f9f9400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:11:35.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.467+0000 7f9fafe00640 1 -- 172.21.15.52:0/1096187155 reap_dead start 2024-03-12T17:11:35.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.468+0000 7f9faf400640 1 --2- 172.21.15.52:0/1096187155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f940039c0 0x7f9f94003dc0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9fa000c190 tx=0x7f9f940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff35c2bc4859855 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:35.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.468+0000 7f9fae000640 1 -- 172.21.15.52:0/1096187155 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0005ad0 con 0x7f9f940039c0 2024-03-12T17:11:35.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.468+0000 7f9fae000640 1 -- 172.21.15.52:0/1096187155 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa0005c30 con 0x7f9f940039c0 2024-03-12T17:11:35.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.468+0000 7f9fae000640 1 -- 172.21.15.52:0/1096187155 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa002c830 con 0x7f9f940039c0 2024-03-12T17:11:35.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.469+0000 7f9fb5200640 1 -- 172.21.15.52:0/1096187155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f940039c0 msgr2=0x7f9f94003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:35.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.469+0000 7f9fb5200640 1 --2- 172.21.15.52:0/1096187155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f940039c0 0x7f9f94003dc0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9fa000c190 tx=0x7f9f940057f0 comp rx=0 tx=0).stop 2024-03-12T17:11:35.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.469+0000 7f9fb5200640 1 -- 172.21.15.52:0/1096187155 shutdown_connections 2024-03-12T17:11:35.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.469+0000 7f9fb5200640 1 --2- 172.21.15.52:0/1096187155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f940039c0 0x7f9f94003dc0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:35.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.469+0000 7f9fb5200640 1 -- 172.21.15.52:0/1096187155 >> 172.21.15.52:0/1096187155 conn(0x7f9fb006ae90 msgr2=0x7f9fb006b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:35.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.469+0000 7f9fb5200640 1 -- 172.21.15.52:0/1096187155 shutdown_connections 2024-03-12T17:11:35.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.469+0000 7f9fb5200640 1 -- 172.21.15.52:0/1096187155 wait complete. 2024-03-12T17:11:35.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.471+0000 7f9fb5200640 1 Processor -- start 2024-03-12T17:11:35.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.471+0000 7f9fb5200640 1 -- start start 2024-03-12T17:11:35.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.472+0000 7f9fb5200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9f940039c0 0x7f9fb00b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:35.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.472+0000 7f9fb5200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fb00b2fa0 0x7f9fb00b6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:35.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.472+0000 7f9fb5200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fb00b4670 0x7f9fb00b6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:35.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.472+0000 7f9fb5200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb00cabc0 con 0x7f9fb00b4670 2024-03-12T17:11:35.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.472+0000 7f9fb5200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb00b4c00 con 0x7f9fb00b2fa0 2024-03-12T17:11:35.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.472+0000 7f9fb5200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb00cae60 con 0x7f9f940039c0 2024-03-12T17:11:35.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.472+0000 7f9fafe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fb00b4670 0x7f9fb00b6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:35.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.472+0000 7f9fafe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fb00b4670 0x7f9fb00b6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45488/0 (socket says 172.21.15.52:45488) 2024-03-12T17:11:35.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.472+0000 7f9fafe00640 1 -- 172.21.15.52:0/323399384 learned_addr learned my addr 172.21.15.52:0/323399384 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:35.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.473+0000 7f9faea00640 1 --2- 172.21.15.52:0/323399384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fb00b2fa0 0x7f9fb00b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:35.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.473+0000 7f9faf400640 1 --2- 172.21.15.52:0/323399384 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9f940039c0 0x7f9fb00b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:35.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.473+0000 7f9fafe00640 1 -- 172.21.15.52:0/323399384 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9f940039c0 msgr2=0x7f9fb00b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:35.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.473+0000 7f9fafe00640 1 --2- 172.21.15.52:0/323399384 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9f940039c0 0x7f9fb00b2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:35.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.473+0000 7f9fafe00640 1 -- 172.21.15.52:0/323399384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fb00b2fa0 msgr2=0x7f9fb00b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:35.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.473+0000 7f9fafe00640 1 --2- 172.21.15.52:0/323399384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fb00b2fa0 0x7f9fb00b6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:35.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.473+0000 7f9fafe00640 1 -- 172.21.15.52:0/323399384 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb00cb140 con 0x7f9fb00b4670 2024-03-12T17:11:35.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.473+0000 7f9fafe00640 1 --2- 172.21.15.52:0/323399384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fb00b4670 0x7f9fb00b6d60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9fb00c61a0 tx=0x7f9fa800d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:35.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.473+0000 7f9fa7400640 1 -- 172.21.15.52:0/323399384 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8006d50 con 0x7f9fb00b4670 2024-03-12T17:11:35.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.473+0000 7f9fa7400640 1 -- 172.21.15.52:0/323399384 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa8004970 con 0x7f9fb00b4670 2024-03-12T17:11:35.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.473+0000 7f9fa7400640 1 -- 172.21.15.52:0/323399384 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8004d20 con 0x7f9fb00b4670 2024-03-12T17:11:35.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.473+0000 7f9fb5200640 1 -- 172.21.15.52:0/323399384 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb01541a0 con 0x7f9fb00b4670 2024-03-12T17:11:35.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.473+0000 7f9fb5200640 1 -- 172.21.15.52:0/323399384 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb00c61d0 con 0x7f9fb00b4670 2024-03-12T17:11:35.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.475+0000 7f9fb5200640 1 -- 172.21.15.52:0/323399384 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fb00cb330 con 0x7f9fb00b4670 2024-03-12T17:11:35.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.476+0000 7f9fa7400640 1 -- 172.21.15.52:0/323399384 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9fa8005030 con 0x7f9fb00b4670 2024-03-12T17:11:35.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.476+0000 7f9fa7400640 1 --2- 172.21.15.52:0/323399384 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f9f9006ac00 0x7f9f9006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:35.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.477+0000 7f9fa7400640 1 -- 172.21.15.52:0/323399384 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f9fa8088520 con 0x7f9fb00b4670 2024-03-12T17:11:35.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.477+0000 7f9faf400640 1 --2- 172.21.15.52:0/323399384 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f9f9006ac00 0x7f9f9006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:35.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.478+0000 7f9faf400640 1 --2- 172.21.15.52:0/323399384 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f9f9006ac00 0x7f9f9006d0c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0034000 tx=0x7f9fa0009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:35.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.481+0000 7f9fa7400640 1 -- 172.21.15.52:0/323399384 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f9fa80538c0 con 0x7f9fb00b4670 2024-03-12T17:11:35.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.694+0000 7f9fb5200640 1 -- 172.21.15.52:0/323399384 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f9fb004e030 con 0x7f9fb00b4670 2024-03-12T17:11:35.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.703+0000 7f9fa7400640 1 -- 172.21.15.52:0/323399384 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9fa8056ee0 con 0x7f9fb00b4670 2024-03-12T17:11:35.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.706+0000 7f9fb5200640 1 -- 172.21.15.52:0/323399384 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f9f9006ac00 msgr2=0x7f9f9006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:35.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.706+0000 7f9fb5200640 1 --2- 172.21.15.52:0/323399384 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f9f9006ac00 0x7f9f9006d0c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0034000 tx=0x7f9fa0009030 comp rx=0 tx=0).stop 2024-03-12T17:11:35.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.706+0000 7f9fb5200640 1 -- 172.21.15.52:0/323399384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fb00b4670 msgr2=0x7f9fb00b6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:35.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.706+0000 7f9fb5200640 1 --2- 172.21.15.52:0/323399384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fb00b4670 0x7f9fb00b6d60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9fb00c61a0 tx=0x7f9fa800d9d0 comp rx=0 tx=0).stop 2024-03-12T17:11:35.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.706+0000 7f9fb5200640 1 -- 172.21.15.52:0/323399384 shutdown_connections 2024-03-12T17:11:35.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.706+0000 7f9fb5200640 1 --2- 172.21.15.52:0/323399384 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f9f9006ac00 0x7f9f9006d0c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:35.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.706+0000 7f9fb5200640 1 --2- 172.21.15.52:0/323399384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fb00b4670 0x7f9fb00b6d60 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:35.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.706+0000 7f9fb5200640 1 --2- 172.21.15.52:0/323399384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fb00b2fa0 0x7f9fb00b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:35.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.706+0000 7f9fb5200640 1 --2- 172.21.15.52:0/323399384 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9f940039c0 0x7f9fb00b2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:35.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.706+0000 7f9fb5200640 1 -- 172.21.15.52:0/323399384 >> 172.21.15.52:0/323399384 conn(0x7f9fb006ae90 msgr2=0x7f9fb015a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:35.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.706+0000 7f9fb5200640 1 -- 172.21.15.52:0/323399384 shutdown_connections 2024-03-12T17:11:35.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.706+0000 7f9fb5200640 1 -- 172.21.15.52:0/323399384 wait complete. 2024-03-12T17:11:35.719 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_acls.TestACLs.test_acls' 2024-03-12T17:11:35.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.937+0000 7f50eb000640 1 Processor -- start 2024-03-12T17:11:35.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.937+0000 7f50eb000640 1 -- start start 2024-03-12T17:11:35.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.939+0000 7f50eb000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f50e4160800 0x7f50e40b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:35.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.939+0000 7f50eb000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f50e40b1be0 0x7f50e406c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:35.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.939+0000 7f50eb000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50e40b2030 con 0x7f50e413faa0 2024-03-12T17:11:35.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.939+0000 7f50eb000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50e4161640 con 0x7f50e40b1be0 2024-03-12T17:11:35.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.939+0000 7f50eb000640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e40b38a0 con 0x7f50e4160800 2024-03-12T17:11:35.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.939+0000 7f50e3400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f50e4160800 0x7f50e40b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:35.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.939+0000 7f50e3400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f50e4160800 0x7f50e40b16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.52:57692/0 (socket says 172.21.15.52:57692) 2024-03-12T17:11:35.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.939+0000 7f50e3400640 1 -- 172.21.15.52:0/1866147510 learned_addr learned my addr 172.21.15.52:0/1866147510 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:35.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.940+0000 7f50e9000640 1 --2- 172.21.15.52:0/1866147510 >> v2:172.21.15.102:3300/0 conn(0x7f50e40b1be0 0x7f50e406c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:35.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.940+0000 7f50e3400640 1 -- 172.21.15.52:0/1866147510 >> v2:172.21.15.102:3300/0 conn(0x7f50e40b1be0 msgr2=0x7f50e406c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:35.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.940+0000 7f50e3400640 1 --2- 172.21.15.52:0/1866147510 >> v2:172.21.15.102:3300/0 conn(0x7f50e40b1be0 0x7f50e406c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:35.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.940+0000 7f50e3400640 1 -- 172.21.15.52:0/1866147510 >> v1:172.21.15.102:6789/0 conn(0x7f50e413faa0 legacy=0x7f50e4160f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:35.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.940+0000 7f50e3400640 1 -- 172.21.15.52:0/1866147510 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e40b3b80 con 0x7f50e4160800 2024-03-12T17:11:35.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.940+0000 7f50e9000640 1 --2- 172.21.15.52:0/1866147510 >> v2:172.21.15.102:3300/0 conn(0x7f50e40b1be0 0x7f50e406c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-12T17:11:35.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.942+0000 7f50e3400640 1 --2- 172.21.15.52:0/1866147510 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f50e4160800 0x7f50e40b16a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f50cc009870 tx=0x7f50cc02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccede8597aae5911 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:35.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.942+0000 7f50e2a00640 1 -- 172.21.15.52:0/1866147510 <== mon.2 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc030840 con 0x7f50e4160800 2024-03-12T17:11:35.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.942+0000 7f50e2a00640 1 -- 172.21.15.52:0/1866147510 <== mon.2 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50cc0309a0 con 0x7f50e4160800 2024-03-12T17:11:35.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.942+0000 7f50e2a00640 1 -- 172.21.15.52:0/1866147510 <== mon.2 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc030d30 con 0x7f50e4160800 2024-03-12T17:11:35.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.943+0000 7f50eb000640 1 -- 172.21.15.52:0/1866147510 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f50e4160800 msgr2=0x7f50e40b16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:35.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.943+0000 7f50eb000640 1 --2- 172.21.15.52:0/1866147510 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f50e4160800 0x7f50e40b16a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f50cc009870 tx=0x7f50cc02b980 comp rx=0 tx=0).stop 2024-03-12T17:11:35.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.943+0000 7f50eb000640 1 -- 172.21.15.52:0/1866147510 shutdown_connections 2024-03-12T17:11:35.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.943+0000 7f50eb000640 1 --2- 172.21.15.52:0/1866147510 >> v2:172.21.15.102:3300/0 conn(0x7f50e40b1be0 0x7f50e406c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:35.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.943+0000 7f50eb000640 1 --2- 172.21.15.52:0/1866147510 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f50e4160800 0x7f50e40b16a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:35.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.943+0000 7f50eb000640 1 -- 172.21.15.52:0/1866147510 >> 172.21.15.52:0/1866147510 conn(0x7f50e406a3d0 msgr2=0x7f50e40b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:35.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.943+0000 7f50eb000640 1 -- 172.21.15.52:0/1866147510 shutdown_connections 2024-03-12T17:11:35.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.943+0000 7f50eb000640 1 -- 172.21.15.52:0/1866147510 wait complete. 2024-03-12T17:11:35.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.946+0000 7f50eb000640 1 Processor -- start 2024-03-12T17:11:35.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.946+0000 7f50eb000640 1 -- start start 2024-03-12T17:11:35.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.947+0000 7f50eb000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50e40b1be0 0x7f50e4152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:35.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.947+0000 7f50eb000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50e413faa0 0x7f50e4140100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:35.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.947+0000 7f50eb000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f50e4160800 0x7f50e4156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:35.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.947+0000 7f50eb000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e40cca10 con 0x7f50e40b1be0 2024-03-12T17:11:35.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.947+0000 7f50eb000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e40cc070 con 0x7f50e413faa0 2024-03-12T17:11:35.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.947+0000 7f50eb000640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e40cc350 con 0x7f50e4160800 2024-03-12T17:11:35.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.947+0000 7f50e3e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50e40b1be0 0x7f50e4152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:35.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.947+0000 7f50e3e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50e40b1be0 0x7f50e4152c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45498/0 (socket says 172.21.15.52:45498) 2024-03-12T17:11:35.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.947+0000 7f50e3e00640 1 -- 172.21.15.52:0/50201864 learned_addr learned my addr 172.21.15.52:0/50201864 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:35.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.948+0000 7f50e3400640 1 --2- 172.21.15.52:0/50201864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50e413faa0 0x7f50e4140100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:35.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.948+0000 7f50e9000640 1 --2- 172.21.15.52:0/50201864 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f50e4160800 0x7f50e4156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:35.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.948+0000 7f50e3e00640 1 -- 172.21.15.52:0/50201864 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f50e4160800 msgr2=0x7f50e4156180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:35.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.948+0000 7f50e3e00640 1 --2- 172.21.15.52:0/50201864 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f50e4160800 0x7f50e4156180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:35.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.948+0000 7f50e3e00640 1 -- 172.21.15.52:0/50201864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50e413faa0 msgr2=0x7f50e4140100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:35.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.948+0000 7f50e3e00640 1 --2- 172.21.15.52:0/50201864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50e413faa0 0x7f50e4140100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:35.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.948+0000 7f50e3e00640 1 -- 172.21.15.52:0/50201864 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50dc009a70 con 0x7f50e40b1be0 2024-03-12T17:11:35.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.948+0000 7f50e9000640 1 --2- 172.21.15.52:0/50201864 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f50e4160800 0x7f50e4156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:11:35.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.948+0000 7f50e3e00640 1 --2- 172.21.15.52:0/50201864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50e40b1be0 0x7f50e4152c50 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f50d400c450 tx=0x7f50d400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:35.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.949+0000 7f50dbe00640 1 -- 172.21.15.52:0/50201864 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d40052d0 con 0x7f50e40b1be0 2024-03-12T17:11:35.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.949+0000 7f50dbe00640 1 -- 172.21.15.52:0/50201864 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d4005430 con 0x7f50e40b1be0 2024-03-12T17:11:35.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.949+0000 7f50eb000640 1 -- 172.21.15.52:0/50201864 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50cc0094d0 con 0x7f50e40b1be0 2024-03-12T17:11:35.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.949+0000 7f50dbe00640 1 -- 172.21.15.52:0/50201864 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d4012870 con 0x7f50e40b1be0 2024-03-12T17:11:35.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.949+0000 7f50eb000640 1 -- 172.21.15.52:0/50201864 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50e41554f0 con 0x7f50e40b1be0 2024-03-12T17:11:35.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.950+0000 7f50eb000640 1 -- 172.21.15.52:0/50201864 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50e40ccb50 con 0x7f50e40b1be0 2024-03-12T17:11:35.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.951+0000 7f50dbe00640 1 -- 172.21.15.52:0/50201864 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f50d40129d0 con 0x7f50e40b1be0 2024-03-12T17:11:35.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.952+0000 7f50dbe00640 1 --2- 172.21.15.52:0/50201864 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f50c006ac00 0x7f50c006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:35.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.952+0000 7f50dbe00640 1 -- 172.21.15.52:0/50201864 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f50d4088b60 con 0x7f50e40b1be0 2024-03-12T17:11:35.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.953+0000 7f50e3400640 1 --2- 172.21.15.52:0/50201864 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f50c006ac00 0x7f50c006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:35.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.954+0000 7f50e3400640 1 --2- 172.21.15.52:0/50201864 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f50c006ac00 0x7f50c006d0c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f50cc036000 tx=0x7f50cc03b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:35.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:35.956+0000 7f50dbe00640 1 -- 172.21.15.52:0/50201864 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f50e40ccb50 con 0x7f50e40b1be0 2024-03-12T17:11:36.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:36.228+0000 7f50eb000640 1 -- 172.21.15.52:0/50201864 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_acls.TestACLs.test_acls"]} v 0) v1 -- 0x7f50e40b2030 con 0x7f50e40b1be0 2024-03-12T17:11:37.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:37.233+0000 7f50dbe00640 1 -- 172.21.15.52:0/50201864 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_acls.TestACLs.test_acls"]}]=0 v223) v1 ==== 120+0+0 (secure 0 0 0) 0x7f50e40b2030 con 0x7f50e40b1be0 2024-03-12T17:11:37.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:37.237+0000 7f50eb000640 1 -- 172.21.15.52:0/50201864 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f50c006ac00 msgr2=0x7f50c006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:37.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:37.237+0000 7f50eb000640 1 --2- 172.21.15.52:0/50201864 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f50c006ac00 0x7f50c006d0c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f50cc036000 tx=0x7f50cc03b040 comp rx=0 tx=0).stop 2024-03-12T17:11:37.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:37.237+0000 7f50eb000640 1 -- 172.21.15.52:0/50201864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50e40b1be0 msgr2=0x7f50e4152c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:37.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:37.237+0000 7f50eb000640 1 --2- 172.21.15.52:0/50201864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50e40b1be0 0x7f50e4152c50 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f50d400c450 tx=0x7f50d400c910 comp rx=0 tx=0).stop 2024-03-12T17:11:37.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:37.237+0000 7f50eb000640 1 -- 172.21.15.52:0/50201864 shutdown_connections 2024-03-12T17:11:37.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:37.237+0000 7f50eb000640 1 --2- 172.21.15.52:0/50201864 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f50c006ac00 0x7f50c006d0c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:37.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:37.237+0000 7f50eb000640 1 --2- 172.21.15.52:0/50201864 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f50e4160800 0x7f50e4156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:37.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:37.237+0000 7f50eb000640 1 --2- 172.21.15.52:0/50201864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50e413faa0 0x7f50e4140100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:37.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:37.237+0000 7f50eb000640 1 --2- 172.21.15.52:0/50201864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50e40b1be0 0x7f50e4152c50 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:37.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:37.237+0000 7f50eb000640 1 -- 172.21.15.52:0/50201864 >> 172.21.15.52:0/50201864 conn(0x7f50e406a3d0 msgr2=0x7f50e415ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:37.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:37.237+0000 7f50eb000640 1 -- 172.21.15.52:0/50201864 shutdown_connections 2024-03-12T17:11:37.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:37.237+0000 7f50eb000640 1 -- 172.21.15.52:0/50201864 wait complete. 2024-03-12T17:11:37.251 INFO:tasks.cephfs_test_runner:test_acls (tasks.cephfs.test_acls.TestACLs) ... ok 2024-03-12T17:11:37.251 INFO:tasks.cephfs_test_runner: 2024-03-12T17:11:37.251 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-12T17:11:37.251 INFO:tasks.cephfs_test_runner:Ran 1 test in 245.455s 2024-03-12T17:11:37.252 INFO:tasks.cephfs_test_runner: 2024-03-12T17:11:37.252 INFO:tasks.cephfs_test_runner:OK 2024-03-12T17:11:37.252 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-03-12T17:11:37.262 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-12T17:11:37.270 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-12T17:11:37.270 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:11:37.270 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-12T17:11:37.328 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054202,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-12T17:11:37.328 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-12T17:11:37.328 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-12T17:11:37.382 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054202,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-12T17:11:37.383 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T17:11:37.383 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-12T17:11:37.439 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053427,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-12T17:11:37.439 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-12T17:11:37.439 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-12T17:11:37.489 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053427,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-12T17:11:37.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:11:37.489 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-03-12T17:11:37.548 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-12T17:11:37.548 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-12T17:11:37.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:11:37.550 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-12T17:11:37.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:11:37.649 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-12T17:11:37.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:11:37.741 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link delete ceph-brx 2024-03-12T17:11:37.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:11:37.865 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-03-12T17:11:37.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:11:37.913 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link delete ceph-brx 2024-03-12T17:11:38.051 INFO:teuthology.orchestra.run.smithi052.stderr:Cannot find device "ceph-brx" 2024-03-12T17:11:38.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T17:11:38.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:11:38.056 DEBUG:teuthology.orchestra.run.smithi102:> ip netns list 2024-03-12T17:11:38.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-12T17:11:38.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-03-12T17:11:38.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:11:38.115 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-12T17:11:38.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:11:38.214 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-12T17:11:38.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:11:38.304 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link delete ceph-brx 2024-03-12T17:11:38.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:11:38.417 DEBUG:teuthology.orchestra.run.smithi102:> ip netns list 2024-03-12T17:11:38.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-12T17:11:38.462 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link delete ceph-brx 2024-03-12T17:11:38.595 INFO:teuthology.orchestra.run.smithi102.stderr:Cannot find device "ceph-brx" 2024-03-12T17:11:38.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T17:11:38.598 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-12T17:11:38.607 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-12T17:11:38.607 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-12T17:11:38.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.846+0000 7fcaf6a00640 1 Processor -- start 2024-03-12T17:11:38.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.846+0000 7fcaf6a00640 1 -- start start 2024-03-12T17:11:38.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.848+0000 7fcaf6a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcaf00ccec0 0x7fcaf00cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:38.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.848+0000 7fcaf6a00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf006da70 con 0x7fcaf00b37f0 2024-03-12T17:11:38.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.848+0000 7fcaf6a00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf00b1d00 con 0x7fcaf014aa40 2024-03-12T17:11:38.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.848+0000 7fcaf6a00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf00b1fe0 con 0x7fcaf00ccec0 2024-03-12T17:11:38.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.848+0000 7fcae7400640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fcaf00b37f0 0x7fcaf006c280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51922/0 (socket says 172.21.15.52:51922) 2024-03-12T17:11:38.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.849+0000 7fcae7400640 1 -- 172.21.15.52:0/3630204366 learned_addr learned my addr 172.21.15.52:0/3630204366 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:11:38.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.849+0000 7fcaf4a00640 1 --2- 172.21.15.52:0/3630204366 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcaf00ccec0 0x7fcaf00cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:38.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.849+0000 7fcaef400640 1 -- 172.21.15.52:0/3630204366 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3890973779 0 0) 0x7fcaf006da70 con 0x7fcaf00b37f0 2024-03-12T17:11:38.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.849+0000 7fcaef400640 1 -- 172.21.15.52:0/3630204366 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcad4003520 con 0x7fcaf00b37f0 2024-03-12T17:11:38.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.850+0000 7fcaef400640 1 -- 172.21.15.52:0/3630204366 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1716813707 0 0) 0x7fcad4003520 con 0x7fcaf00b37f0 2024-03-12T17:11:38.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.850+0000 7fcaef400640 1 -- 172.21.15.52:0/3630204366 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcaf00ccec0 msgr2=0x7fcaf00cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:38.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.850+0000 7fcaef400640 1 --2- 172.21.15.52:0/3630204366 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcaf00ccec0 0x7fcaf00cd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:38.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.850+0000 7fcaef400640 1 -- 172.21.15.52:0/3630204366 >> v1:172.21.15.102:6789/0 conn(0x7fcaf014aa40 legacy=0x7fcaf00cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:38.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.850+0000 7fcaef400640 1 -- 172.21.15.52:0/3630204366 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf00b22c0 con 0x7fcaf00b37f0 2024-03-12T17:11:38.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.850+0000 7fcaef400640 1 -- 172.21.15.52:0/3630204366 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7fcad80019f0 con 0x7fcaf00b37f0 2024-03-12T17:11:38.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.850+0000 7fcaef400640 1 -- 172.21.15.52:0/3630204366 >> v1:172.21.15.52:6789/0 conn(0x7fcaf00b37f0 legacy=0x7fcaf006c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:38.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.850+0000 7fcaef400640 1 --2- 172.21.15.52:0/3630204366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcad40039a0 0x7fcad4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:38.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.850+0000 7fcaef400640 1 --2- 172.21.15.52:0/3630204366 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcad40047c0 0x7fcad4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:38.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.850+0000 7fcaef400640 1 --2- 172.21.15.52:0/3630204366 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcaf00ccec0 0x7fcad4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:38.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.850+0000 7fcaef400640 1 -- 172.21.15.52:0/3630204366 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf00b1fe0 con 0x7fcad40047c0 2024-03-12T17:11:38.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.850+0000 7fcaef400640 1 -- 172.21.15.52:0/3630204366 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad400ba60 con 0x7fcad40039a0 2024-03-12T17:11:38.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.850+0000 7fcaef400640 1 -- 172.21.15.52:0/3630204366 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcad400bc20 con 0x7fcaf00ccec0 2024-03-12T17:11:38.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.850+0000 7fcae7400640 1 --2- 172.21.15.52:0/3630204366 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcad40047c0 0x7fcad4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:38.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.850+0000 7fcaefe00640 1 --2- 172.21.15.52:0/3630204366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcad40039a0 0x7fcad4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:38.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.850+0000 7fcaf4a00640 1 --2- 172.21.15.52:0/3630204366 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcaf00ccec0 0x7fcad4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:38.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.850+0000 7fcae7400640 1 -- 172.21.15.52:0/3630204366 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcaf00ccec0 msgr2=0x7fcad4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:38.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.851+0000 7fcae7400640 1 --2- 172.21.15.52:0/3630204366 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcaf00ccec0 0x7fcad4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:38.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.851+0000 7fcae7400640 1 -- 172.21.15.52:0/3630204366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcad40039a0 msgr2=0x7fcad4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:38.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.851+0000 7fcae7400640 1 --2- 172.21.15.52:0/3630204366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcad40039a0 0x7fcad4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:38.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.851+0000 7fcae7400640 1 -- 172.21.15.52:0/3630204366 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad4003520 con 0x7fcad40047c0 2024-03-12T17:11:38.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.852+0000 7fcae7400640 1 --2- 172.21.15.52:0/3630204366 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcad40047c0 0x7fcad4004c40 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcad8002230 tx=0x7fcad40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e550a3358cd74b52 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:38.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.852+0000 7fcaef400640 1 -- 172.21.15.52:0/3630204366 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad8034630 con 0x7fcad40047c0 2024-03-12T17:11:38.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.852+0000 7fcaef400640 1 -- 172.21.15.52:0/3630204366 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad802f070 con 0x7fcad40047c0 2024-03-12T17:11:38.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.852+0000 7fcaef400640 1 -- 172.21.15.52:0/3630204366 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad8034a10 con 0x7fcad40047c0 2024-03-12T17:11:38.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.852+0000 7fcaf6a00640 1 -- 172.21.15.52:0/3630204366 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcad40047c0 msgr2=0x7fcad4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:38.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.852+0000 7fcaf6a00640 1 --2- 172.21.15.52:0/3630204366 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcad40047c0 0x7fcad4004c40 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcad8002230 tx=0x7fcad40057f0 comp rx=0 tx=0).stop 2024-03-12T17:11:38.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.852+0000 7fcaf4a00640 1 -- 172.21.15.52:0/3630204366 reap_dead start 2024-03-12T17:11:38.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.853+0000 7fcaf6a00640 1 -- 172.21.15.52:0/3630204366 shutdown_connections 2024-03-12T17:11:38.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.853+0000 7fcaf6a00640 1 -- 172.21.15.52:0/3630204366 >> 172.21.15.52:0/3630204366 conn(0x7fcaf006a3d0 msgr2=0x7fcaf00b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:38.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.853+0000 7fcaf6a00640 1 -- 172.21.15.52:0/3630204366 shutdown_connections 2024-03-12T17:11:38.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.853+0000 7fcaf6a00640 1 -- 172.21.15.52:0/3630204366 wait complete. 2024-03-12T17:11:38.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.854+0000 7fcaf6a00640 1 Processor -- start 2024-03-12T17:11:38.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.855+0000 7fcaf6a00640 1 -- start start 2024-03-12T17:11:38.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.856+0000 7fcaf6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcad40047c0 0x7fcaf01600b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:38.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.856+0000 7fcaf6a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaf0161640 0x7fcaf014e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:38.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.856+0000 7fcaf6a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcaf0161aa0 0x7fcaf014e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:38.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.856+0000 7fcaf6a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf01644e0 con 0x7fcaf0161640 2024-03-12T17:11:38.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.856+0000 7fcaf6a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf0163c40 con 0x7fcad40047c0 2024-03-12T17:11:38.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.856+0000 7fcaf6a00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf0163f20 con 0x7fcaf0161aa0 2024-03-12T17:11:38.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.856+0000 7fcae7400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaf0161640 0x7fcaf014e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:38.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.856+0000 7fcae7400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaf0161640 0x7fcaf014e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45526/0 (socket says 172.21.15.52:45526) 2024-03-12T17:11:38.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.856+0000 7fcae7400640 1 -- 172.21.15.52:0/2777958620 learned_addr learned my addr 172.21.15.52:0/2777958620 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:38.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.856+0000 7fcaefe00640 1 --2- 172.21.15.52:0/2777958620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcad40047c0 0x7fcaf01600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:38.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.856+0000 7fcae7400640 1 -- 172.21.15.52:0/2777958620 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcaf0161aa0 msgr2=0x7fcaf014e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:38.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.856+0000 7fcaf4a00640 1 --2- 172.21.15.52:0/2777958620 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcaf0161aa0 0x7fcaf014e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:38.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.856+0000 7fcae7400640 1 --2- 172.21.15.52:0/2777958620 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcaf0161aa0 0x7fcaf014e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:38.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.856+0000 7fcae7400640 1 -- 172.21.15.52:0/2777958620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcad40047c0 msgr2=0x7fcaf01600b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:38.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.856+0000 7fcae7400640 1 --2- 172.21.15.52:0/2777958620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcad40047c0 0x7fcaf01600b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:38.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.856+0000 7fcae7400640 1 -- 172.21.15.52:0/2777958620 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf0164200 con 0x7fcaf0161640 2024-03-12T17:11:38.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.857+0000 7fcaefe00640 1 --2- 172.21.15.52:0/2777958620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcad40047c0 0x7fcaf01600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:11:38.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.857+0000 7fcae7400640 1 --2- 172.21.15.52:0/2777958620 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaf0161640 0x7fcaf014e080 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fcad802ca20 tx=0x7fcad8001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:38.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.857+0000 7fcaecc00640 1 -- 172.21.15.52:0/2777958620 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad8040680 con 0x7fcaf0161640 2024-03-12T17:11:38.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.857+0000 7fcaecc00640 1 -- 172.21.15.52:0/2777958620 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad80407e0 con 0x7fcaf0161640 2024-03-12T17:11:38.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.857+0000 7fcaecc00640 1 -- 172.21.15.52:0/2777958620 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad8040b90 con 0x7fcaf0161640 2024-03-12T17:11:38.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.857+0000 7fcaf6a00640 1 -- 172.21.15.52:0/2777958620 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf01431d0 con 0x7fcaf0161640 2024-03-12T17:11:38.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.857+0000 7fcaf6a00640 1 -- 172.21.15.52:0/2777958620 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf01605f0 con 0x7fcaf0161640 2024-03-12T17:11:38.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.858+0000 7fcaf6a00640 1 -- 172.21.15.52:0/2777958620 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf00670d0 con 0x7fcaf0161640 2024-03-12T17:11:38.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.859+0000 7fcaecc00640 1 -- 172.21.15.52:0/2777958620 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fcad8031030 con 0x7fcaf0161640 2024-03-12T17:11:38.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.861+0000 7fcaecc00640 1 --2- 172.21.15.52:0/2777958620 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fcacc06ac30 0x7fcacc06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:38.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.861+0000 7fcaecc00640 1 -- 172.21.15.52:0/2777958620 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fcad80ab2d0 con 0x7fcaf0161640 2024-03-12T17:11:38.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.861+0000 7fcaefe00640 1 --2- 172.21.15.52:0/2777958620 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fcacc06ac30 0x7fcacc06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:38.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.863+0000 7fcaefe00640 1 --2- 172.21.15.52:0/2777958620 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fcacc06ac30 0x7fcacc06d0f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcae0000c00 tx=0x7fcae0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:38.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:38.866+0000 7fcaecc00640 1 -- 172.21.15.52:0/2777958620 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fcad80765d0 con 0x7fcaf0161640 2024-03-12T17:11:39.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.061+0000 7fcaf6a00640 1 -- 172.21.15.52:0/2777958620 --> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcaf00cda30 con 0x7fcacc06ac30 2024-03-12T17:11:39.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.068+0000 7fcaecc00640 1 -- 172.21.15.52:0/2777958620 <== mgr.4109 v2:172.21.15.102:6832/36622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48341 (secure 0 0 0) 0x7fcae000d050 con 0x7fcacc06ac30 2024-03-12T17:11:39.062 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:11:39.063 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-03-12T17:11:39.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.071+0000 7fcaf6a00640 1 -- 172.21.15.52:0/2777958620 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fcacc06ac30 msgr2=0x7fcacc06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:39.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.071+0000 7fcaf6a00640 1 --2- 172.21.15.52:0/2777958620 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fcacc06ac30 0x7fcacc06d0f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcae0000c00 tx=0x7fcae0009040 comp rx=0 tx=0).stop 2024-03-12T17:11:39.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.071+0000 7fcaf6a00640 1 -- 172.21.15.52:0/2777958620 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaf0161640 msgr2=0x7fcaf014e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:39.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.071+0000 7fcaf6a00640 1 --2- 172.21.15.52:0/2777958620 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaf0161640 0x7fcaf014e080 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fcad802ca20 tx=0x7fcad8001cb0 comp rx=0 tx=0).stop 2024-03-12T17:11:39.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.071+0000 7fcaf6a00640 1 -- 172.21.15.52:0/2777958620 shutdown_connections 2024-03-12T17:11:39.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.071+0000 7fcaf6a00640 1 --2- 172.21.15.52:0/2777958620 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fcacc06ac30 0x7fcacc06d0f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:39.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.071+0000 7fcaf6a00640 1 --2- 172.21.15.52:0/2777958620 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcaf0161aa0 0x7fcaf014e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:39.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.071+0000 7fcaf6a00640 1 --2- 172.21.15.52:0/2777958620 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaf0161640 0x7fcaf014e080 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:39.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.071+0000 7fcaf6a00640 1 --2- 172.21.15.52:0/2777958620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcad40047c0 0x7fcaf01600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:39.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.071+0000 7fcaf6a00640 1 -- 172.21.15.52:0/2777958620 >> 172.21.15.52:0/2777958620 conn(0x7fcaf006a3d0 msgr2=0x7fcaf01496b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:39.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.071+0000 7fcaf6a00640 1 -- 172.21.15.52:0/2777958620 shutdown_connections 2024-03-12T17:11:39.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.071+0000 7fcaf6a00640 1 -- 172.21.15.52:0/2777958620 wait complete. 2024-03-12T17:11:39.083 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2024-03-12T17:11:37.266054+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238832,"kb_used_data":6300,"kb_used_omap":13,"kb_used_meta":232434,"kb_avail":754735888,"statfs":{"total":773094113280,"available":772849549312,"internally_reserved":0,"allocated":6451200,"data_stored":4138120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":238012504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-617293,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-2,"num_read_kb":-6,"num_write":-129,"num_write_kb":-664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002167"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":184,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-12T17:10:23.233999+0000","last_change":"2024-03-12T17:06:21.714813+0000","last_active":"2024-03-12T17:10:23.233999+0000","last_peered":"2024-03-12T17:10:23.233999+0000","last_clean":"2024-03-12T17:10:23.233999+0000","last_became_active":"2024-03-12T17:06:21.712488+0000","last_became_peered":"2024-03-12T17:06:21.712488+0000","last_unstale":"2024-03-12T17:10:23.233999+0000","last_undegraded":"2024-03-12T17:10:23.233999+0000","last_fullsized":"2024-03-12T17:10:23.233999+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:19.815688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:19.815688+0000","last_clean_scrub_stamp":"2024-03-12T17:06:19.815688+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:38:54.300793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28700,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343140,"statfs":{"total":96636764160,"available":96607375360,"internally_reserved":0,"allocated":1720320,"data_stored":1419745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.1200000000000001,"15min":1.0389999999999999},"min":{"1min":0.68500000000000005,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.8080000000000001,"5min":13.563000000000001,"15min":13.563000000000001},"last":1.234},{"interface":"front","average":{"1min":1.004,"5min":1.1040000000000001,"15min":1.0740000000000001},"min":{"1min":0.68100000000000005,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.776,"5min":6.6779999999999999,"15min":6.6779999999999999},"last":0.85399999999999998}]},{"osd":1,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.107,"15min":1.1519999999999999},"min":{"1min":0.79800000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.488,"5min":2.9990000000000001,"15min":2.9990000000000001},"last":1.131},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.0429999999999999},"min":{"1min":0.748,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.5860000000000001,"5min":5.181,"15min":5.181},"last":0.92200000000000004}]},{"osd":2,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.167,"15min":1.1359999999999999},"min":{"1min":0.754,"5min":0.748,"15min":0.748},"max":{"1min":1.5189999999999999,"5min":5.0819999999999999,"15min":5.0819999999999999},"last":1.036},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.2250000000000001,"15min":1.21},"min":{"1min":0.76300000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.425,"5min":6.5289999999999999,"15min":6.5289999999999999},"last":1.357}]},{"osd":3,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.1339999999999999,"15min":1.1879999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.5149999999999999,"5min":3.9849999999999999,"15min":3.9849999999999999},"last":1.504},{"interface":"front","average":{"1min":1.264,"5min":1.258,"15min":1.26},"min":{"1min":0.98099999999999998,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.657,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":1.276}]},{"osd":4,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.234,"15min":1.268},"min":{"1min":0.98299999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.8740000000000001,"5min":3.8580000000000001,"15min":3.8580000000000001},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.4099999999999999,"15min":1.407},"min":{"1min":1.093,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.5900000000000001,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":1.458}]},{"osd":5,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.246,"15min":1.264},"min":{"1min":0.97599999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.5469999999999999,"5min":3.3799999999999999,"15min":3.3799999999999999},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.337,"5min":1.341,"15min":1.3600000000000001},"min":{"1min":0.98399999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.677,"5min":4.0750000000000002,"15min":4.0750000000000002},"last":1.677}]},{"osd":6,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.375,"15min":1.3919999999999999},"min":{"1min":1.01,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.599,"5min":3.7599999999999998,"15min":3.7599999999999998},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4690000000000001,"15min":1.53},"min":{"1min":0.90200000000000002,"5min":0.626,"15min":0.626},"max":{"1min":1.927,"5min":4.7560000000000002,"15min":4.7560000000000002},"last":1.798}]}]},{"osd":6,"up_from":13,"seq":55834574914,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":491520,"data_stored":211913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.0409999999999999,"15min":1.0680000000000001},"min":{"1min":0.61799999999999999,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5600000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.073,"5min":1.0449999999999999,"15min":0.97199999999999998},"min":{"1min":0.70499999999999996,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.492,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":1.1990000000000001}]},{"osd":1,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":1.026,"15min":0.999},"min":{"1min":0.66700000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.2909999999999999,"5min":1.74,"15min":1.74},"last":1.143},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.1259999999999999,"15min":1.1419999999999999},"min":{"1min":0.68300000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.486,"5min":1.73,"15min":1.73},"last":1.27}]},{"osd":2,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.222,"15min":1.1899999999999999},"min":{"1min":0.79300000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.355,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.125,"5min":1.143,"15min":1.165},"min":{"1min":0.74399999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5920000000000001,"5min":1.764,"15min":1.764},"last":1.5920000000000001}]},{"osd":3,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.25,"15min":1.268},"min":{"1min":0.83999999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.163,"5min":1.236,"15min":1.2470000000000001},"min":{"1min":0.89800000000000002,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.556,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":0.89800000000000002}]},{"osd":4,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.3440000000000001,"15min":1.3100000000000001},"min":{"1min":0.89700000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.7390000000000001,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.319,"5min":1.3400000000000001,"15min":1.363},"min":{"1min":0.93500000000000005,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.554,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.3260000000000001}]},{"osd":5,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.3140000000000001,"15min":1.3260000000000001},"min":{"1min":0.89100000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.7,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.377,"5min":1.4590000000000001,"15min":1.4410000000000001},"min":{"1min":1.1279999999999999,"5min":0.5,"15min":0.5},"max":{"1min":1.6419999999999999,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":1.369}]},{"osd":7,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.5469999999999999,"15min":1.458},"min":{"1min":0.96499999999999997,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.766,"5min":9.9610000000000003,"15min":9.9610000000000003},"last":1.766},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.4610000000000001,"15min":1.464},"min":{"1min":0.98599999999999999,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.79,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.304}]}]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":217873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":1.0269999999999999,"15min":1.1160000000000001},"min":{"1min":0.71599999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2,"5min":3.6709999999999998,"15min":3.6709999999999998},"last":1.712},{"interface":"front","average":{"1min":0.995,"5min":1.069,"15min":1.1619999999999999},"min":{"1min":0.625,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.359,"5min":3.706,"15min":3.706},"last":0.92600000000000005}]},{"osd":1,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.0600000000000001,"15min":1.077},"min":{"1min":0.73399999999999999,"5min":0.442,"15min":0.442},"max":{"1min":1.3089999999999999,"5min":3.0390000000000001,"15min":3.0390000000000001},"last":1.002},{"interface":"front","average":{"1min":1.165,"5min":1.161,"15min":1.1699999999999999},"min":{"1min":0.81299999999999994,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.407,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.3500000000000001}]},{"osd":2,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.175,"15min":1.1919999999999999},"min":{"1min":0.82599999999999996,"5min":0.498,"15min":0.498},"max":{"1min":1.6499999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.3},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.1990000000000001,"15min":1.242},"min":{"1min":0.879,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4610000000000001,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.2569999999999999}]},{"osd":3,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2709999999999999,"15min":1.349},"min":{"1min":0.82499999999999996,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.478,"5min":3.7440000000000002,"15min":3.7440000000000002},"last":1.177},{"interface":"front","average":{"1min":1.262,"5min":1.2829999999999999,"15min":1.29},"min":{"1min":0.91600000000000004,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.5920000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":1.393}]},{"osd":4,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2569999999999999,"15min":1.2410000000000001},"min":{"1min":0.63200000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.5580000000000001,"5min":1.873,"15min":1.873},"last":0.877},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3280000000000001,"15min":1.294},"min":{"1min":0.67200000000000004,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.6040000000000001,"5min":2,"15min":2},"last":1.498}]},{"osd":6,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.4319999999999999,"15min":1.599},"min":{"1min":0.89300000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.7390000000000001,"5min":8.2080000000000002,"15min":8.2080000000000002},"last":1.6499999999999999},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4419999999999999,"15min":1.4490000000000001},"min":{"1min":0.70299999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.569}]},{"osd":7,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.4850000000000001,"5min":1.4630000000000001,"15min":1.417},"min":{"1min":0.79800000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.7130000000000001,"5min":2.173,"15min":2.173},"last":1.8180000000000001},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.575,"15min":1.764},"min":{"1min":1.0209999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":2.1339999999999999,"5min":8.0139999999999993,"15min":8.0139999999999993},"last":1.46}]}]},{"osd":4,"up_from":13,"seq":55834574916,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":516096,"data_stored":223314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.1020000000000001,"15min":1.1739999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.476,"5min":1.655,"15min":1.655},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.119,"15min":1.1879999999999999},"min":{"1min":0.73399999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.446,"5min":1.724,"15min":1.724},"last":1.484}]},{"osd":1,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.0760000000000001,"15min":1.0149999999999999},"min":{"1min":0.81000000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.504,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.169},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.1479999999999999,"15min":1.1919999999999999},"min":{"1min":0.78100000000000003,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.47,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":0.873}]},{"osd":2,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.1639999999999999,"15min":1.204},"min":{"1min":0.80900000000000005,"5min":0.752,"15min":0.752},"max":{"1min":1.899,"5min":1.982,"15min":1.982},"last":1.069},{"interface":"front","average":{"1min":1.224,"5min":1.1950000000000001,"15min":1.256},"min":{"1min":0.88600000000000001,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.546,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.127}]},{"osd":3,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.161,"15min":1.171},"min":{"1min":0.748,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.6000000000000001,"5min":1.661,"15min":1.661},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.3260000000000001,"15min":1.3979999999999999},"min":{"1min":0.90500000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.821,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.458}]},{"osd":5,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.3280000000000001,"15min":1.3520000000000001},"min":{"1min":0.94799999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.5389999999999999,"5min":2.2589999999999999,"15min":2.2589999999999999},"last":1.238},{"interface":"front","average":{"1min":1.393,"5min":1.3220000000000001,"15min":1.321},"min":{"1min":0.83399999999999996,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.6499999999999999,"5min":1.95,"15min":1.95},"last":1.6060000000000001}]},{"osd":6,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.3979999999999999,"15min":1.389},"min":{"1min":0.94299999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.9470000000000001,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":1.381},{"interface":"front","average":{"1min":1.431,"5min":1.462,"15min":1.512},"min":{"1min":1.093,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.675,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":1.417}]},{"osd":7,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.488,"5min":1.4279999999999999,"15min":1.4259999999999999},"min":{"1min":1.01,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.752,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":1.718},{"interface":"front","average":{"1min":1.52,"5min":1.5469999999999999,"15min":1.615},"min":{"1min":1.1919999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":2.0779999999999998,"5min":3.5720000000000001,"15min":3.5720000000000001},"last":1.6499999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574913,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32224,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1134592,"data_stored":834290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":1.028,"15min":1.22},"min":{"1min":0.47399999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1990000000000001,"5min":5.0039999999999996,"15min":5.0039999999999996},"last":1.2030000000000001},{"interface":"front","average":{"1min":0.81000000000000005,"5min":1.111,"15min":1.262},"min":{"1min":0.58599999999999997,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.3420000000000001,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":0.95999999999999996}]},{"osd":1,"last update":"Tue Mar 12 17:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":1.071,"15min":1.2729999999999999},"min":{"1min":0.52400000000000002,"5min":0.377,"15min":0.377},"max":{"1min":1.26,"5min":4.9080000000000004,"15min":4.9080000000000004},"last":1.131},{"interface":"front","average":{"1min":0.752,"5min":1.0449999999999999,"15min":1.1950000000000001},"min":{"1min":0.54700000000000004,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1779999999999999,"5min":3.2410000000000001,"15min":3.2410000000000001},"last":0.80200000000000005}]},{"osd":2,"last update":"Tue Mar 12 17:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":1.153,"15min":1.341},"min":{"1min":0.57399999999999995,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.155,"5min":3.4990000000000001,"15min":3.4990000000000001},"last":1.2549999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":1.262,"15min":1.488},"min":{"1min":0.65100000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.6439999999999999,"5min":5.1870000000000003,"15min":5.1870000000000003},"last":0.67100000000000004}]},{"osd":4,"last update":"Tue Mar 12 17:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.3240000000000001,"15min":1.5700000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.385,"5min":9.9299999999999997,"15min":9.9299999999999997},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.4179999999999999,"15min":1.597},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.4690000000000001,"5min":5.2450000000000001,"15min":5.2450000000000001},"last":1.2390000000000001}]},{"osd":5,"last update":"Tue Mar 12 17:10:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.2150000000000001,"15min":1.3169999999999999},"min":{"1min":0.74399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.3200000000000001,"5min":2.9550000000000001,"15min":2.9550000000000001},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.3919999999999999,"15min":1.5229999999999999},"min":{"1min":0.79900000000000004,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.5249999999999999,"5min":3.089,"15min":3.089},"last":1.411}]},{"osd":6,"last update":"Tue Mar 12 17:10:30 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.4259999999999999,"15min":1.5780000000000001},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.5800000000000001,"5min":3.3170000000000002,"15min":3.3170000000000002},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.306,"15min":1.417},"min":{"1min":0.78800000000000003,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.226,"5min":2.722,"15min":2.722},"last":1.4970000000000001}]},{"osd":7,"last update":"Tue Mar 12 17:10:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.3600000000000001,"15min":1.494},"min":{"1min":0.91100000000000003,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.3660000000000001,"5min":2.7559999999999998,"15min":2.7559999999999998},"last":1.444},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.454,"15min":1.5680000000000001},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.528,"5min":3.1389999999999998,"15min":3.1389999999999998},"last":1.353}]}]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31608,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":503808,"data_stored":217831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.78100000000000003,"15min":0.81499999999999995},"min":{"1min":0.498,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0780000000000001,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.88900000000000001,"15min":1.0149999999999999},"min":{"1min":0.56200000000000006,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.234,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":1.089}]},{"osd":1,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.84399999999999997,"15min":0.91700000000000004},"min":{"1min":0.49099999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.2370000000000001,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.87},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.92200000000000004,"15min":0.94699999999999995},"min":{"1min":0.61199999999999999,"5min":0.439,"15min":0.439},"max":{"1min":1.573,"5min":1.732,"15min":1.732},"last":0.92400000000000004}]},{"osd":3,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.01,"15min":1.0229999999999999},"min":{"1min":0.56799999999999995,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.611,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.998,"5min":0.95899999999999996,"15min":1.0249999999999999},"min":{"1min":0.57899999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.859,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.98499999999999999}]},{"osd":4,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.23,"15min":1.25},"min":{"1min":0.84799999999999998,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.9730000000000001,"5min":2.9569999999999999,"15min":2.9569999999999999},"last":1.429},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3029999999999999,"15min":1.407},"min":{"1min":0.91700000000000004,"5min":0.751,"15min":0.751},"max":{"1min":1.7390000000000001,"5min":4.4640000000000004,"15min":4.4640000000000004},"last":1.1910000000000001}]},{"osd":5,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.2889999999999999,"15min":1.359},"min":{"1min":0.999,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5900000000000001,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.23},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3049999999999999,"15min":1.353},"min":{"1min":0.90600000000000003,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.077,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":1.2509999999999999}]},{"osd":6,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.3109999999999999,"15min":1.4379999999999999},"min":{"1min":0.89200000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.663,"5min":3.5179999999999998,"15min":3.5179999999999998},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.341,"5min":1.3560000000000001,"15min":1.446},"min":{"1min":1.028,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.7869999999999999,"5min":2.464,"15min":2.464},"last":1.05}]},{"osd":7,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.391,"15min":1.526},"min":{"1min":0.98499999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.7350000000000001,"5min":3.6110000000000002,"15min":3.6110000000000002},"last":1.3799999999999999},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.421,"15min":1.5209999999999999},"min":{"1min":0.872,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.004,"5min":2.8730000000000002,"15min":2.8730000000000002},"last":1.3620000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31592,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340248,"statfs":{"total":96636764160,"available":96604413952,"internally_reserved":0,"allocated":491520,"data_stored":211421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.85999999999999999,"15min":0.90300000000000002},"min":{"1min":0.54800000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.222,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.90200000000000002,"15min":1.034},"min":{"1min":0.56100000000000005,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.2729999999999999,"5min":3.6419999999999999,"15min":3.6419999999999999},"last":1.03}]},{"osd":2,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.94999999999999996,"15min":1.052},"min":{"1min":0.57399999999999995,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4610000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.97099999999999997,"15min":1.1399999999999999},"min":{"1min":0.63300000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1879999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.623}]},{"osd":3,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.0960000000000001,"15min":1.3200000000000001},"min":{"1min":0.69699999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.331,"5min":3.9580000000000002,"15min":3.9580000000000002},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.93000000000000005,"15min":1.04},"min":{"1min":0.52100000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.159,"5min":1.8,"15min":1.8},"last":0.88100000000000001}]},{"osd":4,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.2110000000000001,"15min":1.331},"min":{"1min":0.85199999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.5649999999999999,"5min":2.694,"15min":2.694},"last":1.107},{"interface":"front","average":{"1min":1.238,"5min":1.2789999999999999,"15min":1.403},"min":{"1min":0.89100000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.8999999999999999,"5min":3.5150000000000001,"15min":3.5150000000000001},"last":1.133}]},{"osd":5,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.1819999999999999,"15min":1.25},"min":{"1min":0.78300000000000003,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.7470000000000001,"5min":2.109,"15min":2.109},"last":1.079},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.337,"15min":1.4690000000000001},"min":{"1min":0.95699999999999996,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.677,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":1.26}]},{"osd":6,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3680000000000001,"15min":1.514},"min":{"1min":1.034,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.8240000000000001,"5min":4.1269999999999998,"15min":4.1269999999999998},"last":1.194},{"interface":"front","average":{"1min":1.244,"5min":1.272,"15min":1.417},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.575,"5min":3.7850000000000001,"15min":3.7850000000000001},"last":1.002}]},{"osd":7,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3680000000000001,"15min":1.5049999999999999},"min":{"1min":0.91100000000000003,"5min":0.873,"15min":0.873},"max":{"1min":1.7689999999999999,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.155},{"interface":"front","average":{"1min":1.377,"5min":1.4139999999999999,"15min":1.5529999999999999},"min":{"1min":1.075,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.74,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.175}]}]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1085440,"data_stored":801733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.79800000000000004,"15min":0.91600000000000004},"min":{"1min":0.40400000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.877,"15min":0.91400000000000003},"min":{"1min":0.438,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.3839999999999999,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.84299999999999997}]},{"osd":2,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":0.95199999999999996,"15min":1.052},"min":{"1min":0.58499999999999996,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.9410000000000001,"5min":1.952,"15min":1.952},"last":1.45},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.98199999999999998,"15min":1.1060000000000001},"min":{"1min":0.55300000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.571,"5min":1.915,"15min":1.915},"last":1.0629999999999999}]},{"osd":3,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.94699999999999995,"15min":1.0029999999999999},"min":{"1min":0.53200000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":2.2189999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.071,"5min":1.0940000000000001,"15min":1.242},"min":{"1min":0.63,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.1000000000000001,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.6739999999999999}]},{"osd":4,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.216,"15min":1.3400000000000001},"min":{"1min":0.84799999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.4209999999999998,"5min":3.0499999999999998,"15min":3.0499999999999998},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.298,"5min":1.337,"15min":1.474},"min":{"1min":0.88500000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":2.3730000000000002,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":1.8919999999999999}]},{"osd":5,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.347,"15min":1.4650000000000001},"min":{"1min":0.874,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.859,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":1.109},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.3140000000000001,"15min":1.3939999999999999},"min":{"1min":0.96099999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.9990000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.333}]},{"osd":6,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.361,"15min":1.5329999999999999},"min":{"1min":0.83099999999999996,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.2719999999999998,"5min":2.597,"15min":2.597},"last":1.79},{"interface":"front","average":{"1min":1.234,"5min":1.3420000000000001,"15min":1.4990000000000001},"min":{"1min":0.92200000000000004,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.6850000000000001,"5min":2.452,"15min":2.452},"last":1.3220000000000001}]},{"osd":7,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.3959999999999999,"15min":1.5529999999999999},"min":{"1min":0.86799999999999999,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.3180000000000001,"5min":2.7240000000000002,"15min":2.7240000000000002},"last":1.6020000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.321,"15min":1.518},"min":{"1min":0.86099999999999999,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.4670000000000001,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":1.4119999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-12T17:11:39.085 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-12T17:11:39.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.308+0000 7fc733200640 1 Processor -- start 2024-03-12T17:11:39.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.309+0000 7fc733200640 1 -- start start 2024-03-12T17:11:39.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.310+0000 7fc733200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc72c0cd5a0 0x7fc72c0b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:39.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.310+0000 7fc733200640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc72c0cef00 con 0x7fc72c0b0970 2024-03-12T17:11:39.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.310+0000 7fc733200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc72c0b1480 con 0x7fc72c0af9a0 2024-03-12T17:11:39.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.310+0000 7fc733200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc72c0b11a0 con 0x7fc72c0cd5a0 2024-03-12T17:11:39.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.311+0000 7fc732800640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fc72c0b0970 0x7fc72c15c1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51928/0 (socket says 172.21.15.52:51928) 2024-03-12T17:11:39.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.311+0000 7fc732800640 1 -- 172.21.15.52:0/1598444848 learned_addr learned my addr 172.21.15.52:0/1598444848 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:11:39.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.311+0000 7fc731400640 1 --2- 172.21.15.52:0/1598444848 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc72c0cd5a0 0x7fc72c0b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:39.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.311+0000 7fc730a00640 1 -- 172.21.15.52:0/1598444848 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3768128964 0 0) 0x7fc72c0cef00 con 0x7fc72c0b0970 2024-03-12T17:11:39.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.311+0000 7fc730a00640 1 -- 172.21.15.52:0/1598444848 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc714003520 con 0x7fc72c0b0970 2024-03-12T17:11:39.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.311+0000 7fc730a00640 1 -- 172.21.15.52:0/1598444848 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4179498081 0 0) 0x7fc714003520 con 0x7fc72c0b0970 2024-03-12T17:11:39.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.311+0000 7fc730a00640 1 -- 172.21.15.52:0/1598444848 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc72c0cd5a0 msgr2=0x7fc72c0b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:39.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.311+0000 7fc730a00640 1 --2- 172.21.15.52:0/1598444848 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc72c0cd5a0 0x7fc72c0b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:39.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.311+0000 7fc730a00640 1 -- 172.21.15.52:0/1598444848 >> v1:172.21.15.102:6789/0 conn(0x7fc72c0af9a0 legacy=0x7fc72c0cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:39.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.312+0000 7fc730a00640 1 -- 172.21.15.52:0/1598444848 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc72c0ce8b0 con 0x7fc72c0b0970 2024-03-12T17:11:39.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.312+0000 7fc730a00640 1 -- 172.21.15.52:0/1598444848 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7fc7280019f0 con 0x7fc72c0b0970 2024-03-12T17:11:39.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.312+0000 7fc730a00640 1 -- 172.21.15.52:0/1598444848 >> v1:172.21.15.52:6789/0 conn(0x7fc72c0b0970 legacy=0x7fc72c15c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:39.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.312+0000 7fc730a00640 1 --2- 172.21.15.52:0/1598444848 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc72c0cd5a0 0x7fc7140039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:39.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.312+0000 7fc730a00640 1 --2- 172.21.15.52:0/1598444848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc714003f70 0x7fc7140047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:39.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.312+0000 7fc730a00640 1 --2- 172.21.15.52:0/1598444848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc714004d00 0x7fc714005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:39.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.312+0000 7fc730a00640 1 -- 172.21.15.52:0/1598444848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72c0b11a0 con 0x7fc714003f70 2024-03-12T17:11:39.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.312+0000 7fc730a00640 1 -- 172.21.15.52:0/1598444848 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71400bac0 con 0x7fc714004d00 2024-03-12T17:11:39.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.312+0000 7fc730a00640 1 -- 172.21.15.52:0/1598444848 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc71400bc80 con 0x7fc72c0cd5a0 2024-03-12T17:11:39.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.312+0000 7fc731400640 1 --2- 172.21.15.52:0/1598444848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc714003f70 0x7fc7140047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:39.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.312+0000 7fc731e00640 1 --2- 172.21.15.52:0/1598444848 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc72c0cd5a0 0x7fc7140039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:39.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.312+0000 7fc732800640 1 --2- 172.21.15.52:0/1598444848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc714004d00 0x7fc714005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:39.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.312+0000 7fc731400640 1 -- 172.21.15.52:0/1598444848 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc72c0cd5a0 msgr2=0x7fc7140039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:39.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.312+0000 7fc731400640 1 --2- 172.21.15.52:0/1598444848 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc72c0cd5a0 0x7fc7140039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:39.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.312+0000 7fc731400640 1 -- 172.21.15.52:0/1598444848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc714004d00 msgr2=0x7fc714005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:39.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.312+0000 7fc731400640 1 --2- 172.21.15.52:0/1598444848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc714004d00 0x7fc714005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:39.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.312+0000 7fc731400640 1 -- 172.21.15.52:0/1598444848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc714003520 con 0x7fc714003f70 2024-03-12T17:11:39.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.314+0000 7fc731400640 1 --2- 172.21.15.52:0/1598444848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc714003f70 0x7fc7140047c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009ac0 tx=0x7fc7140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e5cd2948698e7ef server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:39.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.314+0000 7fc730a00640 1 -- 172.21.15.52:0/1598444848 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c02c600 con 0x7fc714003f70 2024-03-12T17:11:39.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.314+0000 7fc730a00640 1 -- 172.21.15.52:0/1598444848 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc71c02c760 con 0x7fc714003f70 2024-03-12T17:11:39.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.314+0000 7fc730a00640 1 -- 172.21.15.52:0/1598444848 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c02caf0 con 0x7fc714003f70 2024-03-12T17:11:39.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.314+0000 7fc733200640 1 -- 172.21.15.52:0/1598444848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc714003f70 msgr2=0x7fc7140047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:39.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.314+0000 7fc733200640 1 --2- 172.21.15.52:0/1598444848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc714003f70 0x7fc7140047c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009ac0 tx=0x7fc7140057f0 comp rx=0 tx=0).stop 2024-03-12T17:11:39.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.314+0000 7fc732800640 1 -- 172.21.15.52:0/1598444848 reap_dead start 2024-03-12T17:11:39.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.315+0000 7fc733200640 1 -- 172.21.15.52:0/1598444848 shutdown_connections 2024-03-12T17:11:39.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.315+0000 7fc733200640 1 -- 172.21.15.52:0/1598444848 >> 172.21.15.52:0/1598444848 conn(0x7fc72c0d0580 msgr2=0x7fc72c0d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:39.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.315+0000 7fc733200640 1 -- 172.21.15.52:0/1598444848 shutdown_connections 2024-03-12T17:11:39.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.315+0000 7fc733200640 1 -- 172.21.15.52:0/1598444848 wait complete. 2024-03-12T17:11:39.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.316+0000 7fc733200640 1 Processor -- start 2024-03-12T17:11:39.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.316+0000 7fc733200640 1 -- start start 2024-03-12T17:11:39.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.317+0000 7fc733200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc714003f70 0x7fc72c1f2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:39.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.317+0000 7fc733200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc72c0c7cb0 0x7fc72c155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:39.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.317+0000 7fc733200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc72c15df70 0x7fc72c15e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:39.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.317+0000 7fc733200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72c15e930 con 0x7fc72c15df70 2024-03-12T17:11:39.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.317+0000 7fc733200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72c0c8ff0 con 0x7fc714003f70 2024-03-12T17:11:39.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.317+0000 7fc733200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc72c0c8470 con 0x7fc72c0c7cb0 2024-03-12T17:11:39.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.318+0000 7fc732800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc72c15df70 0x7fc72c15e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:39.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.318+0000 7fc732800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc72c15df70 0x7fc72c15e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45548/0 (socket says 172.21.15.52:45548) 2024-03-12T17:11:39.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.318+0000 7fc732800640 1 -- 172.21.15.52:0/3925552206 learned_addr learned my addr 172.21.15.52:0/3925552206 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:39.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.318+0000 7fc731400640 1 --2- 172.21.15.52:0/3925552206 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc72c0c7cb0 0x7fc72c155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:39.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.318+0000 7fc731e00640 1 --2- 172.21.15.52:0/3925552206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc714003f70 0x7fc72c1f2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:39.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.318+0000 7fc732800640 1 -- 172.21.15.52:0/3925552206 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc72c0c7cb0 msgr2=0x7fc72c155940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:39.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.318+0000 7fc732800640 1 --2- 172.21.15.52:0/3925552206 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc72c0c7cb0 0x7fc72c155940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:39.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.318+0000 7fc732800640 1 -- 172.21.15.52:0/3925552206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc714003f70 msgr2=0x7fc72c1f2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:39.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.318+0000 7fc732800640 1 --2- 172.21.15.52:0/3925552206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc714003f70 0x7fc72c1f2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:39.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.318+0000 7fc732800640 1 -- 172.21.15.52:0/3925552206 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc72c0c8750 con 0x7fc72c15df70 2024-03-12T17:11:39.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.318+0000 7fc731e00640 1 --2- 172.21.15.52:0/3925552206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc714003f70 0x7fc72c1f2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:11:39.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.319+0000 7fc732800640 1 --2- 172.21.15.52:0/3925552206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc72c15df70 0x7fc72c15e3f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc728007c40 tx=0x7fc72800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:39.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.319+0000 7fc726000640 1 -- 172.21.15.52:0/3925552206 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72800de90 con 0x7fc72c15df70 2024-03-12T17:11:39.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.319+0000 7fc726000640 1 -- 172.21.15.52:0/3925552206 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72800bb20 con 0x7fc72c15df70 2024-03-12T17:11:39.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.319+0000 7fc726000640 1 -- 172.21.15.52:0/3925552206 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc728007110 con 0x7fc72c15df70 2024-03-12T17:11:39.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.319+0000 7fc733200640 1 -- 172.21.15.52:0/3925552206 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc72c0c8a30 con 0x7fc72c15df70 2024-03-12T17:11:39.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.319+0000 7fc733200640 1 -- 172.21.15.52:0/3925552206 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc72c155e80 con 0x7fc72c15df70 2024-03-12T17:11:39.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.320+0000 7fc733200640 1 -- 172.21.15.52:0/3925552206 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc72c04e030 con 0x7fc72c15df70 2024-03-12T17:11:39.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.321+0000 7fc726000640 1 -- 172.21.15.52:0/3925552206 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc728007270 con 0x7fc72c15df70 2024-03-12T17:11:39.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.322+0000 7fc726000640 1 --2- 172.21.15.52:0/3925552206 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc71006ac30 0x7fc71006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:39.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.322+0000 7fc726000640 1 -- 172.21.15.52:0/3925552206 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fc728089590 con 0x7fc72c15df70 2024-03-12T17:11:39.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.323+0000 7fc731e00640 1 --2- 172.21.15.52:0/3925552206 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc71006ac30 0x7fc71006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:39.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.324+0000 7fc731e00640 1 --2- 172.21.15.52:0/3925552206 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc71006ac30 0x7fc71006d0f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc7200016b0 tx=0x7fc720009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:39.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.327+0000 7fc726000640 1 -- 172.21.15.52:0/3925552206 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fc7280549c0 con 0x7fc72c15df70 2024-03-12T17:11:39.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.522+0000 7fc733200640 1 -- 172.21.15.52:0/3925552206 --> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc72c0afd80 con 0x7fc71006ac30 2024-03-12T17:11:39.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.528+0000 7fc726000640 1 -- 172.21.15.52:0/3925552206 <== mgr.4109 v2:172.21.15.102:6832/36622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48618 (secure 0 0 0) 0x7fc72000d050 con 0x7fc71006ac30 2024-03-12T17:11:39.522 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:11:39.523 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-03-12T17:11:39.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.531+0000 7fc733200640 1 -- 172.21.15.52:0/3925552206 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc71006ac30 msgr2=0x7fc71006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:39.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.531+0000 7fc733200640 1 --2- 172.21.15.52:0/3925552206 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc71006ac30 0x7fc71006d0f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc7200016b0 tx=0x7fc720009040 comp rx=0 tx=0).stop 2024-03-12T17:11:39.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.531+0000 7fc733200640 1 -- 172.21.15.52:0/3925552206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc72c15df70 msgr2=0x7fc72c15e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:39.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.531+0000 7fc733200640 1 --2- 172.21.15.52:0/3925552206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc72c15df70 0x7fc72c15e3f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc728007c40 tx=0x7fc72800b4a0 comp rx=0 tx=0).stop 2024-03-12T17:11:39.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.531+0000 7fc733200640 1 -- 172.21.15.52:0/3925552206 shutdown_connections 2024-03-12T17:11:39.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.531+0000 7fc733200640 1 --2- 172.21.15.52:0/3925552206 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fc71006ac30 0x7fc71006d0f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:39.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.531+0000 7fc733200640 1 --2- 172.21.15.52:0/3925552206 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc72c0c7cb0 0x7fc72c155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:39.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.531+0000 7fc733200640 1 --2- 172.21.15.52:0/3925552206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc72c15df70 0x7fc72c15e3f0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:39.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.531+0000 7fc733200640 1 --2- 172.21.15.52:0/3925552206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc714003f70 0x7fc72c1f2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:39.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.531+0000 7fc733200640 1 -- 172.21.15.52:0/3925552206 >> 172.21.15.52:0/3925552206 conn(0x7fc72c0d0580 msgr2=0x7fc72c06b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:39.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.532+0000 7fc733200640 1 -- 172.21.15.52:0/3925552206 shutdown_connections 2024-03-12T17:11:39.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.532+0000 7fc733200640 1 -- 172.21.15.52:0/3925552206 wait complete. 2024-03-12T17:11:39.543 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-03-12T17:11:39.266627+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":237876,"kb_used_data":5320,"kb_used_omap":13,"kb_used_meta":232434,"kb_avail":754736844,"statfs":{"total":773094113280,"available":772850528256,"internally_reserved":0,"allocated":5447680,"data_stored":3020856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":238012504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-617293,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-2,"num_read_kb":-6,"num_write":-129,"num_write_kb":-664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.229053"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":186,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-03-12T17:11:34.219639+0000","last_change":"2024-03-12T17:06:21.714813+0000","last_active":"2024-03-12T17:11:34.219639+0000","last_peered":"2024-03-12T17:11:34.219639+0000","last_clean":"2024-03-12T17:11:34.219639+0000","last_became_active":"2024-03-12T17:06:21.712488+0000","last_became_peered":"2024-03-12T17:06:21.712488+0000","last_unstale":"2024-03-12T17:11:34.219639+0000","last_undegraded":"2024-03-12T17:11:34.219639+0000","last_fullsized":"2024-03-12T17:11:34.219639+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:19.815688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:19.815688+0000","last_clean_scrub_stamp":"2024-03-12T17:06:19.815688+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:38:54.300793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574915,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28124,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343716,"statfs":{"total":96636764160,"available":96607965184,"internally_reserved":0,"allocated":1126400,"data_stored":820383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.1200000000000001,"15min":1.0389999999999999},"min":{"1min":0.68500000000000005,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.8080000000000001,"5min":13.563000000000001,"15min":13.563000000000001},"last":0.63},{"interface":"front","average":{"1min":1.004,"5min":1.1040000000000001,"15min":1.0740000000000001},"min":{"1min":0.68100000000000005,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.776,"5min":6.6779999999999999,"15min":6.6779999999999999},"last":1.1259999999999999}]},{"osd":1,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.107,"15min":1.1519999999999999},"min":{"1min":0.79800000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.488,"5min":2.9990000000000001,"15min":2.9990000000000001},"last":1.02},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.0429999999999999},"min":{"1min":0.748,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.5860000000000001,"5min":5.181,"15min":5.181},"last":1.3680000000000001}]},{"osd":2,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.167,"15min":1.1359999999999999},"min":{"1min":0.754,"5min":0.748,"15min":0.748},"max":{"1min":1.5189999999999999,"5min":5.0819999999999999,"15min":5.0819999999999999},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.2250000000000001,"15min":1.21},"min":{"1min":0.76300000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.425,"5min":6.5289999999999999,"15min":6.5289999999999999},"last":1.3180000000000001}]},{"osd":3,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.1339999999999999,"15min":1.1879999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.5149999999999999,"5min":3.9849999999999999,"15min":3.9849999999999999},"last":1.5209999999999999},{"interface":"front","average":{"1min":1.264,"5min":1.258,"15min":1.26},"min":{"1min":0.98099999999999998,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.657,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":1.4059999999999999}]},{"osd":4,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.234,"15min":1.268},"min":{"1min":0.98299999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.8740000000000001,"5min":3.8580000000000001,"15min":3.8580000000000001},"last":1.718},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.4099999999999999,"15min":1.407},"min":{"1min":1.093,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.5900000000000001,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":1.2370000000000001}]},{"osd":5,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.246,"15min":1.264},"min":{"1min":0.97599999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.5469999999999999,"5min":3.3799999999999999,"15min":3.3799999999999999},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.337,"5min":1.341,"15min":1.3600000000000001},"min":{"1min":0.98399999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.677,"5min":4.0750000000000002,"15min":4.0750000000000002},"last":1.633}]},{"osd":6,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.375,"15min":1.3919999999999999},"min":{"1min":1.01,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.599,"5min":3.7599999999999998,"15min":3.7599999999999998},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4690000000000001,"15min":1.53},"min":{"1min":0.90200000000000002,"5min":0.626,"15min":0.626},"max":{"1min":1.927,"5min":4.7560000000000002,"15min":4.7560000000000002},"last":1.8009999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":230015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.0409999999999999,"15min":1.0680000000000001},"min":{"1min":0.61799999999999999,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5600000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.575},{"interface":"front","average":{"1min":1.073,"5min":1.0449999999999999,"15min":0.97199999999999998},"min":{"1min":0.70499999999999996,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.492,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":1.355}]},{"osd":1,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":1.026,"15min":0.999},"min":{"1min":0.66700000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.2909999999999999,"5min":1.74,"15min":1.74},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.1259999999999999,"15min":1.1419999999999999},"min":{"1min":0.68300000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.486,"5min":1.73,"15min":1.73},"last":1.2330000000000001}]},{"osd":2,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.222,"15min":1.1899999999999999},"min":{"1min":0.79300000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.355,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.125,"5min":1.143,"15min":1.165},"min":{"1min":0.74399999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5920000000000001,"5min":1.764,"15min":1.764},"last":1.7270000000000001}]},{"osd":3,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.25,"15min":1.268},"min":{"1min":0.83999999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":1.655},{"interface":"front","average":{"1min":1.163,"5min":1.236,"15min":1.2470000000000001},"min":{"1min":0.89800000000000002,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.556,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.2809999999999999}]},{"osd":4,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.3440000000000001,"15min":1.3100000000000001},"min":{"1min":0.89700000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.7390000000000001,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.319,"5min":1.3400000000000001,"15min":1.363},"min":{"1min":0.93500000000000005,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.554,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.4910000000000001}]},{"osd":5,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.3140000000000001,"15min":1.3260000000000001},"min":{"1min":0.89100000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.7,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.377,"5min":1.4590000000000001,"15min":1.4410000000000001},"min":{"1min":1.1279999999999999,"5min":0.5,"15min":0.5},"max":{"1min":1.6419999999999999,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":1.319}]},{"osd":7,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.5469999999999999,"15min":1.458},"min":{"1min":0.96499999999999997,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.766,"5min":9.9610000000000003,"15min":9.9610000000000003},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.4610000000000001,"15min":1.464},"min":{"1min":0.98599999999999999,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.79,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.444}]}]},{"osd":5,"up_from":13,"seq":55834574914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":230015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":1.0269999999999999,"15min":1.1160000000000001},"min":{"1min":0.71599999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2,"5min":3.6709999999999998,"15min":3.6709999999999998},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.995,"5min":1.069,"15min":1.1619999999999999},"min":{"1min":0.625,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.359,"5min":3.706,"15min":3.706},"last":1.004}]},{"osd":1,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.0600000000000001,"15min":1.077},"min":{"1min":0.73399999999999999,"5min":0.442,"15min":0.442},"max":{"1min":1.3089999999999999,"5min":3.0390000000000001,"15min":3.0390000000000001},"last":1.169},{"interface":"front","average":{"1min":1.165,"5min":1.161,"15min":1.1699999999999999},"min":{"1min":0.81299999999999994,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.407,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.6439999999999999}]},{"osd":2,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.175,"15min":1.1919999999999999},"min":{"1min":0.82599999999999996,"5min":0.498,"15min":0.498},"max":{"1min":1.6499999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.1990000000000001,"15min":1.242},"min":{"1min":0.879,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4610000000000001,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.589}]},{"osd":3,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2709999999999999,"15min":1.349},"min":{"1min":0.82499999999999996,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.478,"5min":3.7440000000000002,"15min":3.7440000000000002},"last":1.319},{"interface":"front","average":{"1min":1.262,"5min":1.2829999999999999,"15min":1.29},"min":{"1min":0.91600000000000004,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.5920000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":1.266}]},{"osd":4,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2569999999999999,"15min":1.2410000000000001},"min":{"1min":0.63200000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.5580000000000001,"5min":1.873,"15min":1.873},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3280000000000001,"15min":1.294},"min":{"1min":0.67200000000000004,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.6040000000000001,"5min":2,"15min":2},"last":1.5}]},{"osd":6,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.4319999999999999,"15min":1.599},"min":{"1min":0.89300000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.7390000000000001,"5min":8.2080000000000002,"15min":8.2080000000000002},"last":1.464},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4419999999999999,"15min":1.4490000000000001},"min":{"1min":0.70299999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.6799999999999999}]},{"osd":7,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.4850000000000001,"5min":1.4630000000000001,"15min":1.417},"min":{"1min":0.79800000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.7130000000000001,"5min":2.173,"15min":2.173},"last":1.7150000000000001},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.575,"15min":1.764},"min":{"1min":1.0209999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":2.1339999999999999,"5min":8.0139999999999993,"15min":8.0139999999999993},"last":1.5509999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":230015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.1020000000000001,"15min":1.1739999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.476,"5min":1.655,"15min":1.655},"last":1.278},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.119,"15min":1.1879999999999999},"min":{"1min":0.73399999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.446,"5min":1.724,"15min":1.724},"last":1.093}]},{"osd":1,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.0760000000000001,"15min":1.0149999999999999},"min":{"1min":0.81000000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.504,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.877},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.1479999999999999,"15min":1.1919999999999999},"min":{"1min":0.78100000000000003,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.47,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.0149999999999999}]},{"osd":2,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.1639999999999999,"15min":1.204},"min":{"1min":0.80900000000000005,"5min":0.752,"15min":0.752},"max":{"1min":1.899,"5min":1.982,"15min":1.982},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.224,"5min":1.1950000000000001,"15min":1.256},"min":{"1min":0.88600000000000001,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.546,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.4319999999999999}]},{"osd":3,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.161,"15min":1.171},"min":{"1min":0.748,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.6000000000000001,"5min":1.661,"15min":1.661},"last":1.373},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.3260000000000001,"15min":1.3979999999999999},"min":{"1min":0.90500000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.821,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.1659999999999999}]},{"osd":5,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.3280000000000001,"15min":1.3520000000000001},"min":{"1min":0.94799999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.5389999999999999,"5min":2.2589999999999999,"15min":2.2589999999999999},"last":1.575},{"interface":"front","average":{"1min":1.393,"5min":1.3220000000000001,"15min":1.321},"min":{"1min":0.83399999999999996,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.6499999999999999,"5min":1.95,"15min":1.95},"last":1.4770000000000001}]},{"osd":6,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.3979999999999999,"15min":1.389},"min":{"1min":0.94299999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.9470000000000001,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.431,"5min":1.462,"15min":1.512},"min":{"1min":1.093,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.675,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":1.651}]},{"osd":7,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.488,"5min":1.4279999999999999,"15min":1.4259999999999999},"min":{"1min":1.01,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.752,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":1.619},{"interface":"front","average":{"1min":1.52,"5min":1.5469999999999999,"15min":1.615},"min":{"1min":1.1919999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":2.0779999999999998,"5min":3.5720000000000001,"15min":3.5720000000000001},"last":1.524}]}]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31640,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":532480,"data_stored":230015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:11:33 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.92600000000000005,"15min":1.1859999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.369,"5min":5.0039999999999996,"15min":5.0039999999999996},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.97599999999999998,"5min":1.0389999999999999,"15min":1.238},"min":{"1min":0.69699999999999995,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.375,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":0.90600000000000003}]},{"osd":1,"last update":"Tue Mar 12 17:11:33 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.96899999999999997,"15min":1.2390000000000001},"min":{"1min":0.53000000000000003,"5min":0.377,"15min":0.377},"max":{"1min":1.2350000000000001,"5min":4.9080000000000004,"15min":4.9080000000000004},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.98199999999999998,"15min":1.1739999999999999},"min":{"1min":0.64500000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.444,"5min":3.2410000000000001,"15min":3.2410000000000001},"last":1.155}]},{"osd":2,"last update":"Tue Mar 12 17:11:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0700000000000001,"15min":1.3129999999999999},"min":{"1min":0.71799999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.4590000000000001,"5min":3.4990000000000001,"15min":3.4990000000000001},"last":1.28},{"interface":"front","average":{"1min":1.107,"5min":1.163,"15min":1.4550000000000001},"min":{"1min":0.67100000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.6639999999999999,"5min":5.1870000000000003,"15min":5.1870000000000003},"last":1.335}]},{"osd":4,"last update":"Tue Mar 12 17:11:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.2030000000000001,"15min":1.5289999999999999},"min":{"1min":0.78000000000000003,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.5009999999999999,"5min":9.9299999999999997,"15min":9.9299999999999997},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.345,"15min":1.573},"min":{"1min":0.91300000000000003,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.8400000000000001,"5min":5.2450000000000001,"15min":5.2450000000000001},"last":1.415}]},{"osd":5,"last update":"Tue Mar 12 17:11:33 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.1850000000000001,"15min":1.3069999999999999},"min":{"1min":0.88100000000000001,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.5700000000000001,"5min":2.9550000000000001,"15min":2.9550000000000001},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.224,"5min":1.319,"15min":1.4990000000000001},"min":{"1min":0.875,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.9239999999999999,"5min":3.089,"15min":3.089},"last":1.3759999999999999}]},{"osd":6,"last update":"Tue Mar 12 17:11:33 2024","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3660000000000001,"15min":1.5580000000000001},"min":{"1min":0.96199999999999997,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":2.0049999999999999,"5min":3.3170000000000002,"15min":3.3170000000000002},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2569999999999999,"15min":1.401},"min":{"1min":0.85399999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.7509999999999999,"5min":2.722,"15min":2.722},"last":1.2509999999999999}]},{"osd":7,"last update":"Tue Mar 12 17:11:33 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.3,"15min":1.474},"min":{"1min":0.90400000000000003,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.8200000000000001,"5min":2.7559999999999998,"15min":2.7559999999999998},"last":1.206},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.395,"15min":1.5489999999999999},"min":{"1min":0.96699999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2.032,"5min":3.1389999999999998,"15min":3.1389999999999998},"last":1.454}]}]},{"osd":2,"up_from":13,"seq":55834574914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31636,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340204,"statfs":{"total":96636764160,"available":96604368896,"internally_reserved":0,"allocated":532480,"data_stored":230015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.78100000000000003,"15min":0.81499999999999995},"min":{"1min":0.498,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0780000000000001,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.88900000000000001,"15min":1.0149999999999999},"min":{"1min":0.56200000000000006,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.234,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":1.2889999999999999}]},{"osd":1,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.84399999999999997,"15min":0.91700000000000004},"min":{"1min":0.49099999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.2370000000000001,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.92200000000000004,"15min":0.94699999999999995},"min":{"1min":0.61199999999999999,"5min":0.439,"15min":0.439},"max":{"1min":1.573,"5min":1.732,"15min":1.732},"last":1.5309999999999999}]},{"osd":3,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.01,"15min":1.0229999999999999},"min":{"1min":0.56799999999999995,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.611,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.4339999999999999},{"interface":"front","average":{"1min":0.998,"5min":0.95899999999999996,"15min":1.0249999999999999},"min":{"1min":0.57899999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.859,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.90700000000000003}]},{"osd":4,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.23,"15min":1.25},"min":{"1min":0.84799999999999998,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.9730000000000001,"5min":2.9569999999999999,"15min":2.9569999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3029999999999999,"15min":1.407},"min":{"1min":0.91700000000000004,"5min":0.751,"15min":0.751},"max":{"1min":1.7390000000000001,"5min":4.4640000000000004,"15min":4.4640000000000004},"last":1.3899999999999999}]},{"osd":5,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.2889999999999999,"15min":1.359},"min":{"1min":0.999,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5900000000000001,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.345},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3049999999999999,"15min":1.353},"min":{"1min":0.90600000000000003,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.077,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":0.82999999999999996}]},{"osd":6,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.3109999999999999,"15min":1.4379999999999999},"min":{"1min":0.89200000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.663,"5min":3.5179999999999998,"15min":3.5179999999999998},"last":1.454},{"interface":"front","average":{"1min":1.341,"5min":1.3560000000000001,"15min":1.446},"min":{"1min":1.028,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.7869999999999999,"5min":2.464,"15min":2.464},"last":1.496}]},{"osd":7,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.391,"15min":1.526},"min":{"1min":0.98499999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.7350000000000001,"5min":3.6110000000000002,"15min":3.6110000000000002},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.421,"15min":1.5209999999999999},"min":{"1min":0.872,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.004,"5min":2.8730000000000002,"15min":2.8730000000000002},"last":1.571}]}]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31636,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340204,"statfs":{"total":96636764160,"available":96604368896,"internally_reserved":0,"allocated":532480,"data_stored":230015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.85999999999999999,"15min":0.90300000000000002},"min":{"1min":0.54800000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.222,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.90200000000000002,"15min":1.034},"min":{"1min":0.56100000000000005,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.2729999999999999,"5min":3.6419999999999999,"15min":3.6419999999999999},"last":0.69099999999999995}]},{"osd":2,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.94999999999999996,"15min":1.052},"min":{"1min":0.57399999999999995,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4610000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.97099999999999997,"15min":1.1399999999999999},"min":{"1min":0.63300000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1879999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.73699999999999999}]},{"osd":3,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.0960000000000001,"15min":1.3200000000000001},"min":{"1min":0.69699999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.331,"5min":3.9580000000000002,"15min":3.9580000000000002},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.93000000000000005,"15min":1.04},"min":{"1min":0.52100000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.159,"5min":1.8,"15min":1.8},"last":0.96299999999999997}]},{"osd":4,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.2110000000000001,"15min":1.331},"min":{"1min":0.85199999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.5649999999999999,"5min":2.694,"15min":2.694},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.238,"5min":1.2789999999999999,"15min":1.403},"min":{"1min":0.89100000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.8999999999999999,"5min":3.5150000000000001,"15min":3.5150000000000001},"last":1.093}]},{"osd":5,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.1819999999999999,"15min":1.25},"min":{"1min":0.78300000000000003,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.7470000000000001,"5min":2.109,"15min":2.109},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.337,"15min":1.4690000000000001},"min":{"1min":0.95699999999999996,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.677,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":0.95699999999999996}]},{"osd":6,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3680000000000001,"15min":1.514},"min":{"1min":1.034,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.8240000000000001,"5min":4.1269999999999998,"15min":4.1269999999999998},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.244,"5min":1.272,"15min":1.417},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.575,"5min":3.7850000000000001,"15min":3.7850000000000001},"last":1.048}]},{"osd":7,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3680000000000001,"15min":1.5049999999999999},"min":{"1min":0.91100000000000003,"5min":0.873,"15min":0.873},"max":{"1min":1.7689999999999999,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.306},{"interface":"front","average":{"1min":1.377,"5min":1.4139999999999999,"15min":1.5529999999999999},"min":{"1min":1.075,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.74,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":0.93799999999999994}]}]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1126400,"data_stored":820383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.79800000000000004,"15min":0.91600000000000004},"min":{"1min":0.40400000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.877,"15min":0.91400000000000003},"min":{"1min":0.438,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.3839999999999999,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.67400000000000004}]},{"osd":2,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":0.95199999999999996,"15min":1.052},"min":{"1min":0.58499999999999996,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.9410000000000001,"5min":1.952,"15min":1.952},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.98199999999999998,"15min":1.1060000000000001},"min":{"1min":0.55300000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.571,"5min":1.915,"15min":1.915},"last":0.60799999999999998}]},{"osd":3,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.94699999999999995,"15min":1.0029999999999999},"min":{"1min":0.53200000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":2.2189999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.071,"5min":1.0940000000000001,"15min":1.242},"min":{"1min":0.63,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.1000000000000001,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.59399999999999997}]},{"osd":4,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.216,"15min":1.3400000000000001},"min":{"1min":0.84799999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.4209999999999998,"5min":3.0499999999999998,"15min":3.0499999999999998},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.298,"5min":1.337,"15min":1.474},"min":{"1min":0.88500000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":2.3730000000000002,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":1.538}]},{"osd":5,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.347,"15min":1.4650000000000001},"min":{"1min":0.874,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.859,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":4.5780000000000003},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.3140000000000001,"15min":1.3939999999999999},"min":{"1min":0.96099999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.9990000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":4.6319999999999997}]},{"osd":6,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.361,"15min":1.5329999999999999},"min":{"1min":0.83099999999999996,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.2719999999999998,"5min":2.597,"15min":2.597},"last":1.407},{"interface":"front","average":{"1min":1.234,"5min":1.3420000000000001,"15min":1.4990000000000001},"min":{"1min":0.92200000000000004,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.6850000000000001,"5min":2.452,"15min":2.452},"last":0.86199999999999999}]},{"osd":7,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.3959999999999999,"15min":1.5529999999999999},"min":{"1min":0.86799999999999999,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.3180000000000001,"5min":2.7240000000000002,"15min":2.7240000000000002},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.321,"15min":1.518},"min":{"1min":0.86099999999999999,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.4670000000000001,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":2.839}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-12T17:11:39.545 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-12T17:11:39.545 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-12T17:11:39.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.775+0000 7fa163e00640 1 Processor -- start 2024-03-12T17:11:39.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.775+0000 7fa163e00640 1 -- start start 2024-03-12T17:11:39.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.777+0000 7fa163e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa1640cf5c0 0x7fa1640b8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:39.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.777+0000 7fa163e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa1640d0c20 0x7fa1640d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:39.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.777+0000 7fa163e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa16415ce90 con 0x7fa16415cab0 2024-03-12T17:11:39.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.777+0000 7fa163e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1640cddf0 con 0x7fa1640d0c20 2024-03-12T17:11:39.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.777+0000 7fa163e00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1640cd570 con 0x7fa1640cf5c0 2024-03-12T17:11:39.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.777+0000 7fa15be00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa1640cf5c0 0x7fa1640b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:39.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.777+0000 7fa163400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa1640d0c20 0x7fa1640d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:39.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.778+0000 7fa15be00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa1640cf5c0 0x7fa1640b8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.52:57750/0 (socket says 172.21.15.52:57750) 2024-03-12T17:11:39.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.778+0000 7fa15be00640 1 -- 172.21.15.52:0/3035064313 learned_addr learned my addr 172.21.15.52:0/3035064313 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:39.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.778+0000 7fa15be00640 1 -- 172.21.15.52:0/3035064313 >> v2:172.21.15.102:3300/0 conn(0x7fa1640d0c20 msgr2=0x7fa1640d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:39.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.778+0000 7fa15be00640 1 --2- 172.21.15.52:0/3035064313 >> v2:172.21.15.102:3300/0 conn(0x7fa1640d0c20 0x7fa1640d10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:39.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.778+0000 7fa15be00640 1 -- 172.21.15.52:0/3035064313 >> v1:172.21.15.102:6789/0 conn(0x7fa16415cab0 legacy=0x7fa1640ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:39.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.778+0000 7fa15be00640 1 -- 172.21.15.52:0/3035064313 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa164161370 con 0x7fa1640cf5c0 2024-03-12T17:11:39.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.780+0000 7fa15be00640 1 --2- 172.21.15.52:0/3035064313 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa1640cf5c0 0x7fa1640b8050 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009870 tx=0x7fa14c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=167ffca07f4580a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:39.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.780+0000 7fa162000640 1 -- 172.21.15.52:0/3035064313 <== mon.2 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c030840 con 0x7fa1640cf5c0 2024-03-12T17:11:39.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.780+0000 7fa162000640 1 -- 172.21.15.52:0/3035064313 <== mon.2 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14c0309a0 con 0x7fa1640cf5c0 2024-03-12T17:11:39.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.781+0000 7fa162000640 1 -- 172.21.15.52:0/3035064313 <== mon.2 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c030d30 con 0x7fa1640cf5c0 2024-03-12T17:11:39.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.781+0000 7fa163e00640 1 -- 172.21.15.52:0/3035064313 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa1640cf5c0 msgr2=0x7fa1640b8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:39.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.781+0000 7fa163e00640 1 --2- 172.21.15.52:0/3035064313 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa1640cf5c0 0x7fa1640b8050 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009870 tx=0x7fa14c02b980 comp rx=0 tx=0).stop 2024-03-12T17:11:39.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.781+0000 7fa163e00640 1 -- 172.21.15.52:0/3035064313 shutdown_connections 2024-03-12T17:11:39.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.781+0000 7fa163e00640 1 --2- 172.21.15.52:0/3035064313 >> v2:172.21.15.102:3300/0 conn(0x7fa1640d0c20 0x7fa1640d10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:39.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.781+0000 7fa163e00640 1 --2- 172.21.15.52:0/3035064313 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa1640cf5c0 0x7fa1640b8050 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:39.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.781+0000 7fa163e00640 1 -- 172.21.15.52:0/3035064313 >> 172.21.15.52:0/3035064313 conn(0x7fa16406ae90 msgr2=0x7fa16406b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:39.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.781+0000 7fa163e00640 1 -- 172.21.15.52:0/3035064313 shutdown_connections 2024-03-12T17:11:39.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.781+0000 7fa163e00640 1 -- 172.21.15.52:0/3035064313 wait complete. 2024-03-12T17:11:39.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.783+0000 7fa163e00640 1 Processor -- start 2024-03-12T17:11:39.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.783+0000 7fa163e00640 1 -- start start 2024-03-12T17:11:39.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.784+0000 7fa163e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1640cf5c0 0x7fa1640b4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:39.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.784+0000 7fa163e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa1640d0c20 0x7fa1640b46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:39.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.784+0000 7fa163e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa16415cab0 0x7fa1640b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:39.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.784+0000 7fa163e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1640c7410 con 0x7fa1640cf5c0 2024-03-12T17:11:39.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.784+0000 7fa163e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1640b30e0 con 0x7fa16415cab0 2024-03-12T17:11:39.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.784+0000 7fa163e00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1640b33c0 con 0x7fa1640d0c20 2024-03-12T17:11:39.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.784+0000 7fa162a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1640cf5c0 0x7fa1640b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:39.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.785+0000 7fa162a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1640cf5c0 0x7fa1640b4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45552/0 (socket says 172.21.15.52:45552) 2024-03-12T17:11:39.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.785+0000 7fa162a00640 1 -- 172.21.15.52:0/1708008920 learned_addr learned my addr 172.21.15.52:0/1708008920 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:39.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.785+0000 7fa15be00640 1 --2- 172.21.15.52:0/1708008920 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa1640d0c20 0x7fa1640b46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:39.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.785+0000 7fa163400640 1 --2- 172.21.15.52:0/1708008920 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa16415cab0 0x7fa1640b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:39.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.785+0000 7fa162a00640 1 -- 172.21.15.52:0/1708008920 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa1640d0c20 msgr2=0x7fa1640b46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:39.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.785+0000 7fa162a00640 1 --2- 172.21.15.52:0/1708008920 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa1640d0c20 0x7fa1640b46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:39.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.785+0000 7fa162a00640 1 -- 172.21.15.52:0/1708008920 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa16415cab0 msgr2=0x7fa1640b4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:39.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.785+0000 7fa162a00640 1 --2- 172.21.15.52:0/1708008920 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa16415cab0 0x7fa1640b4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:39.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.785+0000 7fa162a00640 1 -- 172.21.15.52:0/1708008920 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14c0094d0 con 0x7fa1640cf5c0 2024-03-12T17:11:39.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.785+0000 7fa162a00640 1 --2- 172.21.15.52:0/1708008920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1640cf5c0 0x7fa1640b4180 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa15400c400 tx=0x7fa15400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:39.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.786+0000 7fa15aa00640 1 -- 172.21.15.52:0/1708008920 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1540052d0 con 0x7fa1640cf5c0 2024-03-12T17:11:39.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.786+0000 7fa15aa00640 1 -- 172.21.15.52:0/1708008920 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa154005430 con 0x7fa1640cf5c0 2024-03-12T17:11:39.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.786+0000 7fa15aa00640 1 -- 172.21.15.52:0/1708008920 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa154003830 con 0x7fa1640cf5c0 2024-03-12T17:11:39.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.786+0000 7fa163e00640 1 -- 172.21.15.52:0/1708008920 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1640b36a0 con 0x7fa1640cf5c0 2024-03-12T17:11:39.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.786+0000 7fa163e00640 1 -- 172.21.15.52:0/1708008920 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1640b3c60 con 0x7fa1640cf5c0 2024-03-12T17:11:39.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.787+0000 7fa163e00640 1 -- 172.21.15.52:0/1708008920 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa16404e030 con 0x7fa1640cf5c0 2024-03-12T17:11:39.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.788+0000 7fa15aa00640 1 -- 172.21.15.52:0/1708008920 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa154003990 con 0x7fa1640cf5c0 2024-03-12T17:11:39.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.789+0000 7fa15aa00640 1 --2- 172.21.15.52:0/1708008920 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa14006ac00 0x7fa14006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:39.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.789+0000 7fa15aa00640 1 -- 172.21.15.52:0/1708008920 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fa1540887b0 con 0x7fa1640cf5c0 2024-03-12T17:11:39.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.790+0000 7fa15be00640 1 --2- 172.21.15.52:0/1708008920 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa14006ac00 0x7fa14006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:39.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.791+0000 7fa15be00640 1 --2- 172.21.15.52:0/1708008920 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa14006ac00 0x7fa14006d0c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa14c036000 tx=0x7fa14c008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:39.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.794+0000 7fa15aa00640 1 -- 172.21.15.52:0/1708008920 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fa154053ab0 con 0x7fa1640cf5c0 2024-03-12T17:11:39.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.987+0000 7fa163e00640 1 -- 172.21.15.52:0/1708008920 --> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa1640cdb10 con 0x7fa14006ac00 2024-03-12T17:11:39.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.994+0000 7fa15aa00640 1 -- 172.21.15.52:0/1708008920 <== mgr.4109 v2:172.21.15.102:6832/36622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48618 (secure 0 0 0) 0x7fa14c044470 con 0x7fa14006ac00 2024-03-12T17:11:39.988 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:11:39.989 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-03-12T17:11:39.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.997+0000 7fa163e00640 1 -- 172.21.15.52:0/1708008920 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa14006ac00 msgr2=0x7fa14006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:39.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.997+0000 7fa163e00640 1 --2- 172.21.15.52:0/1708008920 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa14006ac00 0x7fa14006d0c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa14c036000 tx=0x7fa14c008030 comp rx=0 tx=0).stop 2024-03-12T17:11:39.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.997+0000 7fa163e00640 1 -- 172.21.15.52:0/1708008920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1640cf5c0 msgr2=0x7fa1640b4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:39.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.997+0000 7fa163e00640 1 --2- 172.21.15.52:0/1708008920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1640cf5c0 0x7fa1640b4180 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa15400c400 tx=0x7fa15400c8c0 comp rx=0 tx=0).stop 2024-03-12T17:11:39.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.997+0000 7fa163e00640 1 -- 172.21.15.52:0/1708008920 shutdown_connections 2024-03-12T17:11:39.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.997+0000 7fa163e00640 1 --2- 172.21.15.52:0/1708008920 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fa14006ac00 0x7fa14006d0c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:39.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.997+0000 7fa163e00640 1 --2- 172.21.15.52:0/1708008920 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa1640d0c20 0x7fa1640b46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:39.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.997+0000 7fa163e00640 1 --2- 172.21.15.52:0/1708008920 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa16415cab0 0x7fa1640b4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:39.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.997+0000 7fa163e00640 1 --2- 172.21.15.52:0/1708008920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1640cf5c0 0x7fa1640b4180 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:39.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.997+0000 7fa163e00640 1 -- 172.21.15.52:0/1708008920 >> 172.21.15.52:0/1708008920 conn(0x7fa16406ae90 msgr2=0x7fa1640b7b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:39.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.998+0000 7fa163e00640 1 -- 172.21.15.52:0/1708008920 shutdown_connections 2024-03-12T17:11:39.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:39.998+0000 7fa163e00640 1 -- 172.21.15.52:0/1708008920 wait complete. 2024-03-12T17:11:40.010 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-03-12T17:11:39.266627+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":237876,"kb_used_data":5320,"kb_used_omap":13,"kb_used_meta":232434,"kb_avail":754736844,"statfs":{"total":773094113280,"available":772850528256,"internally_reserved":0,"allocated":5447680,"data_stored":3020856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":238012504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-617293,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-2,"num_read_kb":-6,"num_write":-129,"num_write_kb":-664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.229053"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":186,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-03-12T17:11:34.219639+0000","last_change":"2024-03-12T17:06:21.714813+0000","last_active":"2024-03-12T17:11:34.219639+0000","last_peered":"2024-03-12T17:11:34.219639+0000","last_clean":"2024-03-12T17:11:34.219639+0000","last_became_active":"2024-03-12T17:06:21.712488+0000","last_became_peered":"2024-03-12T17:06:21.712488+0000","last_unstale":"2024-03-12T17:11:34.219639+0000","last_undegraded":"2024-03-12T17:11:34.219639+0000","last_fullsized":"2024-03-12T17:11:34.219639+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:19.815688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:19.815688+0000","last_clean_scrub_stamp":"2024-03-12T17:06:19.815688+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:38:54.300793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574915,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28124,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343716,"statfs":{"total":96636764160,"available":96607965184,"internally_reserved":0,"allocated":1126400,"data_stored":820383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.1200000000000001,"15min":1.0389999999999999},"min":{"1min":0.68500000000000005,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.8080000000000001,"5min":13.563000000000001,"15min":13.563000000000001},"last":0.63},{"interface":"front","average":{"1min":1.004,"5min":1.1040000000000001,"15min":1.0740000000000001},"min":{"1min":0.68100000000000005,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.776,"5min":6.6779999999999999,"15min":6.6779999999999999},"last":1.1259999999999999}]},{"osd":1,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.107,"15min":1.1519999999999999},"min":{"1min":0.79800000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.488,"5min":2.9990000000000001,"15min":2.9990000000000001},"last":1.02},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.0429999999999999},"min":{"1min":0.748,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.5860000000000001,"5min":5.181,"15min":5.181},"last":1.3680000000000001}]},{"osd":2,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.167,"15min":1.1359999999999999},"min":{"1min":0.754,"5min":0.748,"15min":0.748},"max":{"1min":1.5189999999999999,"5min":5.0819999999999999,"15min":5.0819999999999999},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.2250000000000001,"15min":1.21},"min":{"1min":0.76300000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.425,"5min":6.5289999999999999,"15min":6.5289999999999999},"last":1.3180000000000001}]},{"osd":3,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.1339999999999999,"15min":1.1879999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.5149999999999999,"5min":3.9849999999999999,"15min":3.9849999999999999},"last":1.5209999999999999},{"interface":"front","average":{"1min":1.264,"5min":1.258,"15min":1.26},"min":{"1min":0.98099999999999998,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.657,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":1.4059999999999999}]},{"osd":4,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.234,"15min":1.268},"min":{"1min":0.98299999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.8740000000000001,"5min":3.8580000000000001,"15min":3.8580000000000001},"last":1.718},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.4099999999999999,"15min":1.407},"min":{"1min":1.093,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.5900000000000001,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":1.2370000000000001}]},{"osd":5,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.246,"15min":1.264},"min":{"1min":0.97599999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.5469999999999999,"5min":3.3799999999999999,"15min":3.3799999999999999},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.337,"5min":1.341,"15min":1.3600000000000001},"min":{"1min":0.98399999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.677,"5min":4.0750000000000002,"15min":4.0750000000000002},"last":1.633}]},{"osd":6,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.375,"15min":1.3919999999999999},"min":{"1min":1.01,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.599,"5min":3.7599999999999998,"15min":3.7599999999999998},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4690000000000001,"15min":1.53},"min":{"1min":0.90200000000000002,"5min":0.626,"15min":0.626},"max":{"1min":1.927,"5min":4.7560000000000002,"15min":4.7560000000000002},"last":1.8009999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":230015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.0409999999999999,"15min":1.0680000000000001},"min":{"1min":0.61799999999999999,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5600000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.575},{"interface":"front","average":{"1min":1.073,"5min":1.0449999999999999,"15min":0.97199999999999998},"min":{"1min":0.70499999999999996,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.492,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":1.355}]},{"osd":1,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":1.026,"15min":0.999},"min":{"1min":0.66700000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.2909999999999999,"5min":1.74,"15min":1.74},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.1259999999999999,"15min":1.1419999999999999},"min":{"1min":0.68300000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.486,"5min":1.73,"15min":1.73},"last":1.2330000000000001}]},{"osd":2,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.222,"15min":1.1899999999999999},"min":{"1min":0.79300000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.355,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.125,"5min":1.143,"15min":1.165},"min":{"1min":0.74399999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5920000000000001,"5min":1.764,"15min":1.764},"last":1.7270000000000001}]},{"osd":3,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.25,"15min":1.268},"min":{"1min":0.83999999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":1.655},{"interface":"front","average":{"1min":1.163,"5min":1.236,"15min":1.2470000000000001},"min":{"1min":0.89800000000000002,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.556,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.2809999999999999}]},{"osd":4,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.3440000000000001,"15min":1.3100000000000001},"min":{"1min":0.89700000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.7390000000000001,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.319,"5min":1.3400000000000001,"15min":1.363},"min":{"1min":0.93500000000000005,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.554,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.4910000000000001}]},{"osd":5,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.3140000000000001,"15min":1.3260000000000001},"min":{"1min":0.89100000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.7,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.377,"5min":1.4590000000000001,"15min":1.4410000000000001},"min":{"1min":1.1279999999999999,"5min":0.5,"15min":0.5},"max":{"1min":1.6419999999999999,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":1.319}]},{"osd":7,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.5469999999999999,"15min":1.458},"min":{"1min":0.96499999999999997,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.766,"5min":9.9610000000000003,"15min":9.9610000000000003},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.4610000000000001,"15min":1.464},"min":{"1min":0.98599999999999999,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.79,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.444}]}]},{"osd":5,"up_from":13,"seq":55834574914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":230015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":1.0269999999999999,"15min":1.1160000000000001},"min":{"1min":0.71599999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2,"5min":3.6709999999999998,"15min":3.6709999999999998},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.995,"5min":1.069,"15min":1.1619999999999999},"min":{"1min":0.625,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.359,"5min":3.706,"15min":3.706},"last":1.004}]},{"osd":1,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.0600000000000001,"15min":1.077},"min":{"1min":0.73399999999999999,"5min":0.442,"15min":0.442},"max":{"1min":1.3089999999999999,"5min":3.0390000000000001,"15min":3.0390000000000001},"last":1.169},{"interface":"front","average":{"1min":1.165,"5min":1.161,"15min":1.1699999999999999},"min":{"1min":0.81299999999999994,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.407,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.6439999999999999}]},{"osd":2,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.175,"15min":1.1919999999999999},"min":{"1min":0.82599999999999996,"5min":0.498,"15min":0.498},"max":{"1min":1.6499999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.1990000000000001,"15min":1.242},"min":{"1min":0.879,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4610000000000001,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.589}]},{"osd":3,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2709999999999999,"15min":1.349},"min":{"1min":0.82499999999999996,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.478,"5min":3.7440000000000002,"15min":3.7440000000000002},"last":1.319},{"interface":"front","average":{"1min":1.262,"5min":1.2829999999999999,"15min":1.29},"min":{"1min":0.91600000000000004,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.5920000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":1.266}]},{"osd":4,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2569999999999999,"15min":1.2410000000000001},"min":{"1min":0.63200000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.5580000000000001,"5min":1.873,"15min":1.873},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3280000000000001,"15min":1.294},"min":{"1min":0.67200000000000004,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.6040000000000001,"5min":2,"15min":2},"last":1.5}]},{"osd":6,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.4319999999999999,"15min":1.599},"min":{"1min":0.89300000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.7390000000000001,"5min":8.2080000000000002,"15min":8.2080000000000002},"last":1.464},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4419999999999999,"15min":1.4490000000000001},"min":{"1min":0.70299999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.6799999999999999}]},{"osd":7,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.4850000000000001,"5min":1.4630000000000001,"15min":1.417},"min":{"1min":0.79800000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.7130000000000001,"5min":2.173,"15min":2.173},"last":1.7150000000000001},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.575,"15min":1.764},"min":{"1min":1.0209999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":2.1339999999999999,"5min":8.0139999999999993,"15min":8.0139999999999993},"last":1.5509999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":230015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.1020000000000001,"15min":1.1739999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.476,"5min":1.655,"15min":1.655},"last":1.278},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.119,"15min":1.1879999999999999},"min":{"1min":0.73399999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.446,"5min":1.724,"15min":1.724},"last":1.093}]},{"osd":1,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.0760000000000001,"15min":1.0149999999999999},"min":{"1min":0.81000000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.504,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.877},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.1479999999999999,"15min":1.1919999999999999},"min":{"1min":0.78100000000000003,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.47,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.0149999999999999}]},{"osd":2,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.1639999999999999,"15min":1.204},"min":{"1min":0.80900000000000005,"5min":0.752,"15min":0.752},"max":{"1min":1.899,"5min":1.982,"15min":1.982},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.224,"5min":1.1950000000000001,"15min":1.256},"min":{"1min":0.88600000000000001,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.546,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.4319999999999999}]},{"osd":3,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.161,"15min":1.171},"min":{"1min":0.748,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.6000000000000001,"5min":1.661,"15min":1.661},"last":1.373},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.3260000000000001,"15min":1.3979999999999999},"min":{"1min":0.90500000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.821,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.1659999999999999}]},{"osd":5,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.3280000000000001,"15min":1.3520000000000001},"min":{"1min":0.94799999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.5389999999999999,"5min":2.2589999999999999,"15min":2.2589999999999999},"last":1.575},{"interface":"front","average":{"1min":1.393,"5min":1.3220000000000001,"15min":1.321},"min":{"1min":0.83399999999999996,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.6499999999999999,"5min":1.95,"15min":1.95},"last":1.4770000000000001}]},{"osd":6,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.3979999999999999,"15min":1.389},"min":{"1min":0.94299999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.9470000000000001,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.431,"5min":1.462,"15min":1.512},"min":{"1min":1.093,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.675,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":1.651}]},{"osd":7,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.488,"5min":1.4279999999999999,"15min":1.4259999999999999},"min":{"1min":1.01,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.752,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":1.619},{"interface":"front","average":{"1min":1.52,"5min":1.5469999999999999,"15min":1.615},"min":{"1min":1.1919999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":2.0779999999999998,"5min":3.5720000000000001,"15min":3.5720000000000001},"last":1.524}]}]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31640,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":532480,"data_stored":230015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:11:33 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.92600000000000005,"15min":1.1859999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.369,"5min":5.0039999999999996,"15min":5.0039999999999996},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.97599999999999998,"5min":1.0389999999999999,"15min":1.238},"min":{"1min":0.69699999999999995,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.375,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":0.90600000000000003}]},{"osd":1,"last update":"Tue Mar 12 17:11:33 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.96899999999999997,"15min":1.2390000000000001},"min":{"1min":0.53000000000000003,"5min":0.377,"15min":0.377},"max":{"1min":1.2350000000000001,"5min":4.9080000000000004,"15min":4.9080000000000004},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.98199999999999998,"15min":1.1739999999999999},"min":{"1min":0.64500000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.444,"5min":3.2410000000000001,"15min":3.2410000000000001},"last":1.155}]},{"osd":2,"last update":"Tue Mar 12 17:11:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0700000000000001,"15min":1.3129999999999999},"min":{"1min":0.71799999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.4590000000000001,"5min":3.4990000000000001,"15min":3.4990000000000001},"last":1.28},{"interface":"front","average":{"1min":1.107,"5min":1.163,"15min":1.4550000000000001},"min":{"1min":0.67100000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.6639999999999999,"5min":5.1870000000000003,"15min":5.1870000000000003},"last":1.335}]},{"osd":4,"last update":"Tue Mar 12 17:11:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.2030000000000001,"15min":1.5289999999999999},"min":{"1min":0.78000000000000003,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.5009999999999999,"5min":9.9299999999999997,"15min":9.9299999999999997},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.345,"15min":1.573},"min":{"1min":0.91300000000000003,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.8400000000000001,"5min":5.2450000000000001,"15min":5.2450000000000001},"last":1.415}]},{"osd":5,"last update":"Tue Mar 12 17:11:33 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.1850000000000001,"15min":1.3069999999999999},"min":{"1min":0.88100000000000001,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.5700000000000001,"5min":2.9550000000000001,"15min":2.9550000000000001},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.224,"5min":1.319,"15min":1.4990000000000001},"min":{"1min":0.875,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.9239999999999999,"5min":3.089,"15min":3.089},"last":1.3759999999999999}]},{"osd":6,"last update":"Tue Mar 12 17:11:33 2024","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3660000000000001,"15min":1.5580000000000001},"min":{"1min":0.96199999999999997,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":2.0049999999999999,"5min":3.3170000000000002,"15min":3.3170000000000002},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2569999999999999,"15min":1.401},"min":{"1min":0.85399999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.7509999999999999,"5min":2.722,"15min":2.722},"last":1.2509999999999999}]},{"osd":7,"last update":"Tue Mar 12 17:11:33 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.3,"15min":1.474},"min":{"1min":0.90400000000000003,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.8200000000000001,"5min":2.7559999999999998,"15min":2.7559999999999998},"last":1.206},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.395,"15min":1.5489999999999999},"min":{"1min":0.96699999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2.032,"5min":3.1389999999999998,"15min":3.1389999999999998},"last":1.454}]}]},{"osd":2,"up_from":13,"seq":55834574914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31636,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340204,"statfs":{"total":96636764160,"available":96604368896,"internally_reserved":0,"allocated":532480,"data_stored":230015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.78100000000000003,"15min":0.81499999999999995},"min":{"1min":0.498,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0780000000000001,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.88900000000000001,"15min":1.0149999999999999},"min":{"1min":0.56200000000000006,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.234,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":1.2889999999999999}]},{"osd":1,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.84399999999999997,"15min":0.91700000000000004},"min":{"1min":0.49099999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.2370000000000001,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.92200000000000004,"15min":0.94699999999999995},"min":{"1min":0.61199999999999999,"5min":0.439,"15min":0.439},"max":{"1min":1.573,"5min":1.732,"15min":1.732},"last":1.5309999999999999}]},{"osd":3,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.01,"15min":1.0229999999999999},"min":{"1min":0.56799999999999995,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.611,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.4339999999999999},{"interface":"front","average":{"1min":0.998,"5min":0.95899999999999996,"15min":1.0249999999999999},"min":{"1min":0.57899999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.859,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.90700000000000003}]},{"osd":4,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.23,"15min":1.25},"min":{"1min":0.84799999999999998,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.9730000000000001,"5min":2.9569999999999999,"15min":2.9569999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3029999999999999,"15min":1.407},"min":{"1min":0.91700000000000004,"5min":0.751,"15min":0.751},"max":{"1min":1.7390000000000001,"5min":4.4640000000000004,"15min":4.4640000000000004},"last":1.3899999999999999}]},{"osd":5,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.2889999999999999,"15min":1.359},"min":{"1min":0.999,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5900000000000001,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.345},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3049999999999999,"15min":1.353},"min":{"1min":0.90600000000000003,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.077,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":0.82999999999999996}]},{"osd":6,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.3109999999999999,"15min":1.4379999999999999},"min":{"1min":0.89200000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.663,"5min":3.5179999999999998,"15min":3.5179999999999998},"last":1.454},{"interface":"front","average":{"1min":1.341,"5min":1.3560000000000001,"15min":1.446},"min":{"1min":1.028,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.7869999999999999,"5min":2.464,"15min":2.464},"last":1.496}]},{"osd":7,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.391,"15min":1.526},"min":{"1min":0.98499999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.7350000000000001,"5min":3.6110000000000002,"15min":3.6110000000000002},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.421,"15min":1.5209999999999999},"min":{"1min":0.872,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.004,"5min":2.8730000000000002,"15min":2.8730000000000002},"last":1.571}]}]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31636,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340204,"statfs":{"total":96636764160,"available":96604368896,"internally_reserved":0,"allocated":532480,"data_stored":230015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.85999999999999999,"15min":0.90300000000000002},"min":{"1min":0.54800000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.222,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.90200000000000002,"15min":1.034},"min":{"1min":0.56100000000000005,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.2729999999999999,"5min":3.6419999999999999,"15min":3.6419999999999999},"last":0.69099999999999995}]},{"osd":2,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.94999999999999996,"15min":1.052},"min":{"1min":0.57399999999999995,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4610000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.97099999999999997,"15min":1.1399999999999999},"min":{"1min":0.63300000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1879999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.73699999999999999}]},{"osd":3,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.0960000000000001,"15min":1.3200000000000001},"min":{"1min":0.69699999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.331,"5min":3.9580000000000002,"15min":3.9580000000000002},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.93000000000000005,"15min":1.04},"min":{"1min":0.52100000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.159,"5min":1.8,"15min":1.8},"last":0.96299999999999997}]},{"osd":4,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.2110000000000001,"15min":1.331},"min":{"1min":0.85199999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.5649999999999999,"5min":2.694,"15min":2.694},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.238,"5min":1.2789999999999999,"15min":1.403},"min":{"1min":0.89100000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.8999999999999999,"5min":3.5150000000000001,"15min":3.5150000000000001},"last":1.093}]},{"osd":5,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.1819999999999999,"15min":1.25},"min":{"1min":0.78300000000000003,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.7470000000000001,"5min":2.109,"15min":2.109},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.337,"15min":1.4690000000000001},"min":{"1min":0.95699999999999996,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.677,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":0.95699999999999996}]},{"osd":6,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3680000000000001,"15min":1.514},"min":{"1min":1.034,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.8240000000000001,"5min":4.1269999999999998,"15min":4.1269999999999998},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.244,"5min":1.272,"15min":1.417},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.575,"5min":3.7850000000000001,"15min":3.7850000000000001},"last":1.048}]},{"osd":7,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3680000000000001,"15min":1.5049999999999999},"min":{"1min":0.91100000000000003,"5min":0.873,"15min":0.873},"max":{"1min":1.7689999999999999,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.306},{"interface":"front","average":{"1min":1.377,"5min":1.4139999999999999,"15min":1.5529999999999999},"min":{"1min":1.075,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.74,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":0.93799999999999994}]}]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1126400,"data_stored":820383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.79800000000000004,"15min":0.91600000000000004},"min":{"1min":0.40400000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.877,"15min":0.91400000000000003},"min":{"1min":0.438,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.3839999999999999,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.67400000000000004}]},{"osd":2,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":0.95199999999999996,"15min":1.052},"min":{"1min":0.58499999999999996,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.9410000000000001,"5min":1.952,"15min":1.952},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.98199999999999998,"15min":1.1060000000000001},"min":{"1min":0.55300000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.571,"5min":1.915,"15min":1.915},"last":0.60799999999999998}]},{"osd":3,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.94699999999999995,"15min":1.0029999999999999},"min":{"1min":0.53200000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":2.2189999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.071,"5min":1.0940000000000001,"15min":1.242},"min":{"1min":0.63,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.1000000000000001,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.59399999999999997}]},{"osd":4,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.216,"15min":1.3400000000000001},"min":{"1min":0.84799999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.4209999999999998,"5min":3.0499999999999998,"15min":3.0499999999999998},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.298,"5min":1.337,"15min":1.474},"min":{"1min":0.88500000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":2.3730000000000002,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":1.538}]},{"osd":5,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.347,"15min":1.4650000000000001},"min":{"1min":0.874,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.859,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":4.5780000000000003},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.3140000000000001,"15min":1.3939999999999999},"min":{"1min":0.96099999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.9990000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":4.6319999999999997}]},{"osd":6,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.361,"15min":1.5329999999999999},"min":{"1min":0.83099999999999996,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.2719999999999998,"5min":2.597,"15min":2.597},"last":1.407},{"interface":"front","average":{"1min":1.234,"5min":1.3420000000000001,"15min":1.4990000000000001},"min":{"1min":0.92200000000000004,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.6850000000000001,"5min":2.452,"15min":2.452},"last":0.86199999999999999}]},{"osd":7,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.3959999999999999,"15min":1.5529999999999999},"min":{"1min":0.86799999999999999,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.3180000000000001,"5min":2.7240000000000002,"15min":2.7240000000000002},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.321,"15min":1.518},"min":{"1min":0.86099999999999999,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.4670000000000001,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":2.839}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-12T17:11:40.012 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-12T17:11:40.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.240+0000 7ff0e0a00640 1 Processor -- start 2024-03-12T17:11:40.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.240+0000 7ff0e0a00640 1 -- start start 2024-03-12T17:11:40.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.242+0000 7ff0e0a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff0dc1571d0 0x7ff0dc0b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:40.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.242+0000 7ff0e0a00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff0dc0cdc00 0x7ff0dc15aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:40.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.242+0000 7ff0e0a00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0dc1575b0 con 0x7ff0dc15afe0 2024-03-12T17:11:40.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.242+0000 7ff0e0a00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0dc159f20 con 0x7ff0dc0cdc00 2024-03-12T17:11:40.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.242+0000 7ff0e0a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0dc15a200 con 0x7ff0dc1571d0 2024-03-12T17:11:40.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.242+0000 7ff0d3e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff0dc0cdc00 0x7ff0dc15aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:40.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.242+0000 7ff0d3e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff0dc0cdc00 0x7ff0dc15aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45562/0 (socket says 172.21.15.52:45562) 2024-03-12T17:11:40.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.242+0000 7ff0d3e00640 1 -- 172.21.15.52:0/3075443221 learned_addr learned my addr 172.21.15.52:0/3075443221 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:40.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.243+0000 7ff0d3e00640 1 -- 172.21.15.52:0/3075443221 >> v2:172.21.15.102:3300/0 conn(0x7ff0dc1571d0 msgr2=0x7ff0dc0b35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:40.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.243+0000 7ff0d3e00640 1 --2- 172.21.15.52:0/3075443221 >> v2:172.21.15.102:3300/0 conn(0x7ff0dc1571d0 0x7ff0dc0b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:40.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.243+0000 7ff0d3e00640 1 -- 172.21.15.52:0/3075443221 >> v1:172.21.15.102:6789/0 conn(0x7ff0dc15afe0 legacy=0x7ff0dc0cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:40.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.243+0000 7ff0d3e00640 1 -- 172.21.15.52:0/3075443221 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0dc15a4e0 con 0x7ff0dc0cdc00 2024-03-12T17:11:40.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.244+0000 7ff0d3e00640 1 --2- 172.21.15.52:0/3075443221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0dc0cdc00 0x7ff0dc15aaa0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc0712e0 tx=0x7ff0c402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a77d85d5a94975 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:40.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.246+0000 7ff0daa00640 1 -- 172.21.15.52:0/3075443221 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4031870 con 0x7ff0dc0cdc00 2024-03-12T17:11:40.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.246+0000 7ff0daa00640 1 -- 172.21.15.52:0/3075443221 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c40319d0 con 0x7ff0dc0cdc00 2024-03-12T17:11:40.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.246+0000 7ff0daa00640 1 -- 172.21.15.52:0/3075443221 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4031d60 con 0x7ff0dc0cdc00 2024-03-12T17:11:40.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.246+0000 7ff0e0a00640 1 -- 172.21.15.52:0/3075443221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0dc0cdc00 msgr2=0x7ff0dc15aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:40.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.246+0000 7ff0e0a00640 1 --2- 172.21.15.52:0/3075443221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0dc0cdc00 0x7ff0dc15aaa0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc0712e0 tx=0x7ff0c402b910 comp rx=0 tx=0).stop 2024-03-12T17:11:40.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.247+0000 7ff0e0a00640 1 -- 172.21.15.52:0/3075443221 shutdown_connections 2024-03-12T17:11:40.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.247+0000 7ff0e0a00640 1 --2- 172.21.15.52:0/3075443221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0dc0cdc00 0x7ff0dc15aaa0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:40.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.247+0000 7ff0e0a00640 1 --2- 172.21.15.52:0/3075443221 >> v2:172.21.15.102:3300/0 conn(0x7ff0dc1571d0 0x7ff0dc0b35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:40.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.247+0000 7ff0e0a00640 1 -- 172.21.15.52:0/3075443221 >> 172.21.15.52:0/3075443221 conn(0x7ff0dc065df0 msgr2=0x7ff0dc069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:40.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.247+0000 7ff0e0a00640 1 -- 172.21.15.52:0/3075443221 shutdown_connections 2024-03-12T17:11:40.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.247+0000 7ff0e0a00640 1 -- 172.21.15.52:0/3075443221 wait complete. 2024-03-12T17:11:40.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.248+0000 7ff0e0a00640 1 Processor -- start 2024-03-12T17:11:40.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.249+0000 7ff0e0a00640 1 -- start start 2024-03-12T17:11:40.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.249+0000 7ff0e0a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0dc0cdc00 0x7ff0dc137b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:40.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.249+0000 7ff0e0a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0dc1571d0 0x7ff0dc14fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:40.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.249+0000 7ff0e0a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0dc15afe0 0x7ff0dc1503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:40.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.249+0000 7ff0e0a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0dc14df70 con 0x7ff0dc1571d0 2024-03-12T17:11:40.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.249+0000 7ff0e0a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0dc0b0cf0 con 0x7ff0dc15afe0 2024-03-12T17:11:40.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.249+0000 7ff0e0a00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0dc14d920 con 0x7ff0dc0cdc00 2024-03-12T17:11:40.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.250+0000 7ff0d3e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0dc1571d0 0x7ff0dc14fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:40.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.250+0000 7ff0d3e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0dc1571d0 0x7ff0dc14fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45566/0 (socket says 172.21.15.52:45566) 2024-03-12T17:11:40.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.250+0000 7ff0d3e00640 1 -- 172.21.15.52:0/3023690292 learned_addr learned my addr 172.21.15.52:0/3023690292 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:40.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.250+0000 7ff0db400640 1 --2- 172.21.15.52:0/3023690292 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0dc0cdc00 0x7ff0dc137b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:40.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.250+0000 7ff0dbe00640 1 --2- 172.21.15.52:0/3023690292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0dc15afe0 0x7ff0dc1503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:40.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.250+0000 7ff0d3e00640 1 -- 172.21.15.52:0/3023690292 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0dc0cdc00 msgr2=0x7ff0dc137b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:40.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.250+0000 7ff0d3e00640 1 --2- 172.21.15.52:0/3023690292 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0dc0cdc00 0x7ff0dc137b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:40.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.250+0000 7ff0d3e00640 1 -- 172.21.15.52:0/3023690292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0dc15afe0 msgr2=0x7ff0dc1503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:40.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.250+0000 7ff0d3e00640 1 --2- 172.21.15.52:0/3023690292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0dc15afe0 0x7ff0dc1503d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:40.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.250+0000 7ff0d3e00640 1 -- 172.21.15.52:0/3023690292 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c40094d0 con 0x7ff0dc1571d0 2024-03-12T17:11:40.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.250+0000 7ff0dbe00640 1 --2- 172.21.15.52:0/3023690292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0dc15afe0 0x7ff0dc1503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:11:40.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.251+0000 7ff0d3e00640 1 --2- 172.21.15.52:0/3023690292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0dc1571d0 0x7ff0dc14fe90 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4000c00 tx=0x7ff0c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:40.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.251+0000 7ff0d3400640 1 -- 172.21.15.52:0/3023690292 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c403ad10 con 0x7ff0dc1571d0 2024-03-12T17:11:40.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.251+0000 7ff0d3400640 1 -- 172.21.15.52:0/3023690292 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c403ae70 con 0x7ff0dc1571d0 2024-03-12T17:11:40.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.251+0000 7ff0d3400640 1 -- 172.21.15.52:0/3023690292 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4044610 con 0x7ff0dc1571d0 2024-03-12T17:11:40.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.251+0000 7ff0e0a00640 1 -- 172.21.15.52:0/3023690292 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0dc14dc00 con 0x7ff0dc1571d0 2024-03-12T17:11:40.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.251+0000 7ff0e0a00640 1 -- 172.21.15.52:0/3023690292 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0dc0aff70 con 0x7ff0dc1571d0 2024-03-12T17:11:40.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.252+0000 7ff0e0a00640 1 -- 172.21.15.52:0/3023690292 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0dc04e030 con 0x7ff0dc1571d0 2024-03-12T17:11:40.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.253+0000 7ff0d3400640 1 -- 172.21.15.52:0/3023690292 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff0c404d450 con 0x7ff0dc1571d0 2024-03-12T17:11:40.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.254+0000 7ff0d3400640 1 --2- 172.21.15.52:0/3023690292 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff0b806ac00 0x7ff0b806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:40.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.254+0000 7ff0d3400640 1 -- 172.21.15.52:0/3023690292 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5263+0+0 (secure 0 0 0) 0x7ff0c402d040 con 0x7ff0dc1571d0 2024-03-12T17:11:40.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.255+0000 7ff0db400640 1 --2- 172.21.15.52:0/3023690292 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff0b806ac00 0x7ff0b806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:40.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.256+0000 7ff0db400640 1 --2- 172.21.15.52:0/3023690292 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff0b806ac00 0x7ff0b806d0c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc009890 tx=0x7ff0cc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:40.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.258+0000 7ff0d3400640 1 -- 172.21.15.52:0/3023690292 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7ff0c4076cc0 con 0x7ff0dc1571d0 2024-03-12T17:11:40.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.454+0000 7ff0e0a00640 1 -- 172.21.15.52:0/3023690292 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0dc164e90 con 0x7ff0dc1571d0 2024-03-12T17:11:40.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.456+0000 7ff0d3400640 1 -- 172.21.15.52:0/3023690292 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+12721 (secure 0 0 0) 0x7ff0c407a2e0 con 0x7ff0dc1571d0 2024-03-12T17:11:40.449 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:11:40.449 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":45,"fsid":"524cdbb7-4b9b-4ef9-b44d-37d458f5ffd3","created":"2024-03-12T17:06:05.953844+0000","modified":"2024-03-12T17:11:35.212359+0000","last_up_change":"2024-03-12T17:06:17.807936+0000","last_in_change":"2024-03-12T17:06:09.480010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T17:06:19.218543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce602581-c510-46f1-9792-691ff2a839a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":37214},{"type":"v1","addr":"172.21.15.52:6801","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":37214},{"type":"v1","addr":"172.21.15.52:6803","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":37214},{"type":"v1","addr":"172.21.15.52:6807","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":37214},{"type":"v1","addr":"172.21.15.52:6805","nonce":37214}]},"public_addr":"172.21.15.52:6801/37214","cluster_addr":"172.21.15.52:6803/37214","heartbeat_back_addr":"172.21.15.52:6807/37214","heartbeat_front_addr":"172.21.15.52:6805/37214","state":["exists","up"]},{"osd":1,"uuid":"91ba0742-065a-4c84-8000-bbda30993c50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6817","nonce":37218},{"type":"v1","addr":"172.21.15.52:6819","nonce":37218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6821","nonce":37218},{"type":"v1","addr":"172.21.15.52:6823","nonce":37218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":37218},{"type":"v1","addr":"172.21.15.52:6830","nonce":37218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6825","nonce":37218},{"type":"v1","addr":"172.21.15.52:6827","nonce":37218}]},"public_addr":"172.21.15.52:6819/37218","cluster_addr":"172.21.15.52:6823/37218","heartbeat_back_addr":"172.21.15.52:6830/37218","heartbeat_front_addr":"172.21.15.52:6827/37218","state":["exists","up"]},{"osd":2,"uuid":"c166769b-d6f9-4435-a0ae-203fb1fb8743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":37216},{"type":"v1","addr":"172.21.15.52:6809","nonce":37216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":37216},{"type":"v1","addr":"172.21.15.52:6811","nonce":37216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":37216},{"type":"v1","addr":"172.21.15.52:6815","nonce":37216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":37216},{"type":"v1","addr":"172.21.15.52:6813","nonce":37216}]},"public_addr":"172.21.15.52:6809/37216","cluster_addr":"172.21.15.52:6811/37216","heartbeat_back_addr":"172.21.15.52:6815/37216","heartbeat_front_addr":"172.21.15.52:6813/37216","state":["exists","up"]},{"osd":3,"uuid":"179df347-56fa-409a-a308-ba2a64d598be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":37217},{"type":"v1","addr":"172.21.15.52:6818","nonce":37217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":37217},{"type":"v1","addr":"172.21.15.52:6822","nonce":37217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6829","nonce":37217},{"type":"v1","addr":"172.21.15.52:6831","nonce":37217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":37217},{"type":"v1","addr":"172.21.15.52:6826","nonce":37217}]},"public_addr":"172.21.15.52:6818/37217","cluster_addr":"172.21.15.52:6822/37217","heartbeat_back_addr":"172.21.15.52:6831/37217","heartbeat_front_addr":"172.21.15.52:6826/37217","state":["exists","up"]},{"osd":4,"uuid":"a7f97e2c-aeeb-49cd-afa1-9accb1e86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":37191},{"type":"v1","addr":"172.21.15.102:6801","nonce":37191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":37191},{"type":"v1","addr":"172.21.15.102:6803","nonce":37191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":37191},{"type":"v1","addr":"172.21.15.102:6807","nonce":37191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":37191},{"type":"v1","addr":"172.21.15.102:6805","nonce":37191}]},"public_addr":"172.21.15.102:6801/37191","cluster_addr":"172.21.15.102:6803/37191","heartbeat_back_addr":"172.21.15.102:6807/37191","heartbeat_front_addr":"172.21.15.102:6805/37191","state":["exists","up"]},{"osd":5,"uuid":"a0270815-1af1-4a1a-b74d-3e9241179739","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":37201},{"type":"v1","addr":"172.21.15.102:6817","nonce":37201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":37201},{"type":"v1","addr":"172.21.15.102:6819","nonce":37201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":37201},{"type":"v1","addr":"172.21.15.102:6823","nonce":37201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":37201},{"type":"v1","addr":"172.21.15.102:6821","nonce":37201}]},"public_addr":"172.21.15.102:6817/37201","cluster_addr":"172.21.15.102:6819/37201","heartbeat_back_addr":"172.21.15.102:6823/37201","heartbeat_front_addr":"172.21.15.102:6821/37201","state":["exists","up"]},{"osd":6,"uuid":"7992e177-6c68-400b-b7d7-5be8d226812e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":37192},{"type":"v1","addr":"172.21.15.102:6809","nonce":37192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":37192},{"type":"v1","addr":"172.21.15.102:6811","nonce":37192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":37192},{"type":"v1","addr":"172.21.15.102:6815","nonce":37192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":37192},{"type":"v1","addr":"172.21.15.102:6813","nonce":37192}]},"public_addr":"172.21.15.102:6809/37192","cluster_addr":"172.21.15.102:6811/37192","heartbeat_back_addr":"172.21.15.102:6815/37192","heartbeat_front_addr":"172.21.15.102:6813/37192","state":["exists","up"]},{"osd":7,"uuid":"a0ef534d-c6a0-4b86-a2f1-cb6ebb9f1b5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":37214},{"type":"v1","addr":"172.21.15.102:6825","nonce":37214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":37214},{"type":"v1","addr":"172.21.15.102:6827","nonce":37214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":37214},{"type":"v1","addr":"172.21.15.102:6831","nonce":37214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":37214},{"type":"v1","addr":"172.21.15.102:6829","nonce":37214}]},"public_addr":"172.21.15.102:6825/37214","cluster_addr":"172.21.15.102:6827/37214","heartbeat_back_addr":"172.21.15.102:6831/37214","heartbeat_front_addr":"172.21.15.102:6829/37214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.733031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.775178+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.591522+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.785110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.820838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.829252+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T17:06:14.582481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:6835/543567396":"2024-03-13T17:11:32.033545+0000","172.21.15.52:6834/543567396":"2024-03-13T17:11:32.033545+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T17:11:40.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.458+0000 7ff0e0a00640 1 -- 172.21.15.52:0/3023690292 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff0b806ac00 msgr2=0x7ff0b806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:40.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.458+0000 7ff0e0a00640 1 --2- 172.21.15.52:0/3023690292 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff0b806ac00 0x7ff0b806d0c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc009890 tx=0x7ff0cc008040 comp rx=0 tx=0).stop 2024-03-12T17:11:40.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.458+0000 7ff0e0a00640 1 -- 172.21.15.52:0/3023690292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0dc1571d0 msgr2=0x7ff0dc14fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:40.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.458+0000 7ff0e0a00640 1 --2- 172.21.15.52:0/3023690292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0dc1571d0 0x7ff0dc14fe90 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4000c00 tx=0x7ff0c40046c0 comp rx=0 tx=0).stop 2024-03-12T17:11:40.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.458+0000 7ff0e0a00640 1 -- 172.21.15.52:0/3023690292 shutdown_connections 2024-03-12T17:11:40.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.458+0000 7ff0e0a00640 1 --2- 172.21.15.52:0/3023690292 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ff0b806ac00 0x7ff0b806d0c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:40.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.458+0000 7ff0e0a00640 1 --2- 172.21.15.52:0/3023690292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0dc15afe0 0x7ff0dc1503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:40.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.458+0000 7ff0e0a00640 1 --2- 172.21.15.52:0/3023690292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0dc1571d0 0x7ff0dc14fe90 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:40.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.458+0000 7ff0e0a00640 1 --2- 172.21.15.52:0/3023690292 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff0dc0cdc00 0x7ff0dc137b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:40.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.458+0000 7ff0e0a00640 1 -- 172.21.15.52:0/3023690292 >> 172.21.15.52:0/3023690292 conn(0x7ff0dc065df0 msgr2=0x7ff0dc0cc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:40.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.459+0000 7ff0e0a00640 1 -- 172.21.15.52:0/3023690292 shutdown_connections 2024-03-12T17:11:40.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:40.459+0000 7ff0e0a00640 1 -- 172.21.15.52:0/3023690292 wait complete. 2024-03-12T17:11:41.472 INFO:tasks.ceph:Scrubbing osd.0 2024-03-12T17:11:41.473 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-12T17:11:41.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.716+0000 7feea2600640 1 Processor -- start 2024-03-12T17:11:41.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.716+0000 7feea2600640 1 -- start start 2024-03-12T17:11:41.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.717+0000 7feea2600640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fee9c0cd830 0x7fee9c160e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:41.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.717+0000 7feea2600640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee9c1613d0 con 0x7fee9c15c820 2024-03-12T17:11:41.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.717+0000 7feea2600640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee9c0ccd90 con 0x7fee9c0cf7c0 2024-03-12T17:11:41.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.717+0000 7feea2600640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c0cdcc0 con 0x7fee9c0cd830 2024-03-12T17:11:41.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.718+0000 7feea1200640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fee9c15c820 0x7fee9c0cf0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51932/0 (socket says 172.21.15.52:51932) 2024-03-12T17:11:41.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.718+0000 7feea1200640 1 -- 172.21.15.52:0/4032647585 learned_addr learned my addr 172.21.15.52:0/4032647585 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:11:41.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.719+0000 7feea1c00640 1 --2- 172.21.15.52:0/4032647585 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fee9c0cd830 0x7fee9c160e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:41.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.719+0000 7fee9b400640 1 -- 172.21.15.52:0/4032647585 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2981646974 0 0) 0x7fee8c001110 con 0x7fee9c15c820 2024-03-12T17:11:41.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.719+0000 7fee9b400640 1 -- 172.21.15.52:0/4032647585 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee80003520 con 0x7fee9c15c820 2024-03-12T17:11:41.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.719+0000 7fee9b400640 1 -- 172.21.15.52:0/4032647585 <== mon.1 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2075751577 0 0) 0x7fee9c0ccd90 con 0x7fee9c0cf7c0 2024-03-12T17:11:41.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.719+0000 7fee9b400640 1 -- 172.21.15.52:0/4032647585 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee8c001110 con 0x7fee9c0cf7c0 2024-03-12T17:11:41.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.719+0000 7fee9b400640 1 -- 172.21.15.52:0/4032647585 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2029071569 0 0) 0x7fee80003520 con 0x7fee9c15c820 2024-03-12T17:11:41.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.719+0000 7fee9b400640 1 -- 172.21.15.52:0/4032647585 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fee9c0cd830 msgr2=0x7fee9c160e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:41.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.719+0000 7fee9b400640 1 --2- 172.21.15.52:0/4032647585 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fee9c0cd830 0x7fee9c160e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:41.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.719+0000 7fee9b400640 1 -- 172.21.15.52:0/4032647585 >> v1:172.21.15.102:6789/0 conn(0x7fee9c0cf7c0 legacy=0x7fee9c0d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:41.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.719+0000 7fee9b400640 1 -- 172.21.15.52:0/4032647585 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee9c0b7fc0 con 0x7fee9c15c820 2024-03-12T17:11:41.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.719+0000 7fee9b400640 1 -- 172.21.15.52:0/4032647585 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7fee8c001800 con 0x7fee9c15c820 2024-03-12T17:11:41.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.719+0000 7fee9b400640 1 -- 172.21.15.52:0/4032647585 >> v1:172.21.15.52:6789/0 conn(0x7fee9c15c820 legacy=0x7fee9c0cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:41.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.719+0000 7fee9b400640 1 --2- 172.21.15.52:0/4032647585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee80003b30 0x7fee80003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:41.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.719+0000 7fee9b400640 1 --2- 172.21.15.52:0/4032647585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee800047c0 0x7fee80004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:41.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.719+0000 7fee9b400640 1 --2- 172.21.15.52:0/4032647585 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fee9c0cd830 0x7fee80005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:41.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.719+0000 7fee9b400640 1 -- 172.21.15.52:0/4032647585 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c0cdcc0 con 0x7fee80003b30 2024-03-12T17:11:41.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.719+0000 7fee9b400640 1 -- 172.21.15.52:0/4032647585 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8000ba30 con 0x7fee800047c0 2024-03-12T17:11:41.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.719+0000 7fee9b400640 1 -- 172.21.15.52:0/4032647585 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee8000bbf0 con 0x7fee9c0cd830 2024-03-12T17:11:41.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.720+0000 7feea1200640 1 --2- 172.21.15.52:0/4032647585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee80003b30 0x7fee80003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:41.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.720+0000 7feea1c00640 1 --2- 172.21.15.52:0/4032647585 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fee9c0cd830 0x7fee80005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:41.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.720+0000 7fee9be00640 1 --2- 172.21.15.52:0/4032647585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee800047c0 0x7fee80004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:41.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.720+0000 7feea1200640 1 -- 172.21.15.52:0/4032647585 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fee9c0cd830 msgr2=0x7fee80005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:41.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.720+0000 7feea1200640 1 --2- 172.21.15.52:0/4032647585 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fee9c0cd830 0x7fee80005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:41.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.720+0000 7feea1200640 1 -- 172.21.15.52:0/4032647585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee800047c0 msgr2=0x7fee80004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:41.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.720+0000 7feea1200640 1 --2- 172.21.15.52:0/4032647585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee800047c0 0x7fee80004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:41.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.720+0000 7feea1200640 1 -- 172.21.15.52:0/4032647585 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee80003520 con 0x7fee80003b30 2024-03-12T17:11:41.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.720+0000 7fee9be00640 1 --2- 172.21.15.52:0/4032647585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee800047c0 0x7fee80004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:11:41.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.721+0000 7feea1200640 1 --2- 172.21.15.52:0/4032647585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee80003b30 0x7fee80003f30 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fee8c007fa0 tx=0x7fee800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=990da2e7b850379a server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:41.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.722+0000 7fee9b400640 1 -- 172.21.15.52:0/4032647585 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c005ad0 con 0x7fee80003b30 2024-03-12T17:11:41.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.722+0000 7fee9b400640 1 -- 172.21.15.52:0/4032647585 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee8c005c30 con 0x7fee80003b30 2024-03-12T17:11:41.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.722+0000 7fee9b400640 1 -- 172.21.15.52:0/4032647585 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c02c790 con 0x7fee80003b30 2024-03-12T17:11:41.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.722+0000 7feea2600640 1 -- 172.21.15.52:0/4032647585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee80003b30 msgr2=0x7fee80003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:41.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.722+0000 7feea2600640 1 --2- 172.21.15.52:0/4032647585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee80003b30 0x7fee80003f30 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fee8c007fa0 tx=0x7fee800057f0 comp rx=0 tx=0).stop 2024-03-12T17:11:41.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.722+0000 7feea1c00640 1 -- 172.21.15.52:0/4032647585 reap_dead start 2024-03-12T17:11:41.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.722+0000 7feea2600640 1 -- 172.21.15.52:0/4032647585 shutdown_connections 2024-03-12T17:11:41.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.722+0000 7feea2600640 1 -- 172.21.15.52:0/4032647585 >> 172.21.15.52:0/4032647585 conn(0x7fee9c06b050 msgr2=0x7fee9c06b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:41.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.723+0000 7feea2600640 1 -- 172.21.15.52:0/4032647585 shutdown_connections 2024-03-12T17:11:41.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.723+0000 7feea2600640 1 -- 172.21.15.52:0/4032647585 wait complete. 2024-03-12T17:11:41.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.724+0000 7feea2600640 1 Processor -- start 2024-03-12T17:11:41.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.725+0000 7feea2600640 1 -- start start 2024-03-12T17:11:41.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.725+0000 7feea2600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee80003b30 0x7fee9c0b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:41.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.726+0000 7feea2600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee9c0b2b00 0x7fee9c0b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:41.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.726+0000 7feea2600640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fee9c15ef30 0x7fee9c14d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:41.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.726+0000 7feea2600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c0b3050 con 0x7fee80003b30 2024-03-12T17:11:41.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.726+0000 7feea2600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c0b3330 con 0x7fee9c0b2b00 2024-03-12T17:11:41.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.726+0000 7feea2600640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c0b3610 con 0x7fee9c15ef30 2024-03-12T17:11:41.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.726+0000 7feea1200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee80003b30 0x7fee9c0b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:41.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.726+0000 7feea1200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee80003b30 0x7fee9c0b4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45584/0 (socket says 172.21.15.52:45584) 2024-03-12T17:11:41.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.726+0000 7feea1200640 1 -- 172.21.15.52:0/1542118887 learned_addr learned my addr 172.21.15.52:0/1542118887 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:41.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.726+0000 7fee9be00640 1 --2- 172.21.15.52:0/1542118887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee9c0b2b00 0x7fee9c0b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:41.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.726+0000 7feea1200640 1 -- 172.21.15.52:0/1542118887 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fee9c15ef30 msgr2=0x7fee9c14d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:41.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.726+0000 7feea1200640 1 --2- 172.21.15.52:0/1542118887 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fee9c15ef30 0x7fee9c14d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:41.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.726+0000 7feea1200640 1 -- 172.21.15.52:0/1542118887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee9c0b2b00 msgr2=0x7fee9c0b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:41.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.726+0000 7feea1200640 1 --2- 172.21.15.52:0/1542118887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee9c0b2b00 0x7fee9c0b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:41.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.726+0000 7feea1200640 1 -- 172.21.15.52:0/1542118887 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee9c0b38f0 con 0x7fee80003b30 2024-03-12T17:11:41.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.726+0000 7fee9be00640 1 --2- 172.21.15.52:0/1542118887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee9c0b2b00 0x7fee9c0b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:11:41.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.727+0000 7feea1200640 1 --2- 172.21.15.52:0/1542118887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee80003b30 0x7fee9c0b4bc0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fee8c007fa0 tx=0x7fee8c001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:41.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.727+0000 7fee98c00640 1 -- 172.21.15.52:0/1542118887 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c039070 con 0x7fee80003b30 2024-03-12T17:11:41.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.727+0000 7fee98c00640 1 -- 172.21.15.52:0/1542118887 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee8c02cba0 con 0x7fee80003b30 2024-03-12T17:11:41.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.727+0000 7fee98c00640 1 -- 172.21.15.52:0/1542118887 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c03c040 con 0x7fee80003b30 2024-03-12T17:11:41.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.727+0000 7feea2600640 1 -- 172.21.15.52:0/1542118887 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee9c0b3bd0 con 0x7fee80003b30 2024-03-12T17:11:41.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.727+0000 7feea2600640 1 -- 172.21.15.52:0/1542118887 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee9c15f3b0 con 0x7fee80003b30 2024-03-12T17:11:41.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.728+0000 7feea2600640 1 -- 172.21.15.52:0/1542118887 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fee9c0cd310 con 0x7fee80003b30 2024-03-12T17:11:41.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.729+0000 7fee98c00640 1 -- 172.21.15.52:0/1542118887 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fee8c032070 con 0x7fee80003b30 2024-03-12T17:11:41.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.730+0000 7fee98c00640 1 --2- 172.21.15.52:0/1542118887 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fee7806ac00 0x7fee7806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:41.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.730+0000 7fee98c00640 1 -- 172.21.15.52:0/1542118887 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fee8c0aa4c0 con 0x7fee80003b30 2024-03-12T17:11:41.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.731+0000 7fee98c00640 1 --2- 172.21.15.52:0/1542118887 >> [v2:172.21.15.52:6800/37214,v1:172.21.15.52:6801/37214] conn(0x7fee78070f00 0x7fee78073340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:41.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.731+0000 7fee98c00640 1 -- 172.21.15.52:0/1542118887 --> [v2:172.21.15.52:6800/37214,v1:172.21.15.52:6801/37214] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fee78073a10 con 0x7fee78070f00 2024-03-12T17:11:41.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.731+0000 7fee98c00640 1 -- 172.21.15.52:0/1542118887 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee8c0aca40 con 0x7fee80003b30 2024-03-12T17:11:41.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.731+0000 7feea1c00640 1 --2- 172.21.15.52:0/1542118887 >> [v2:172.21.15.52:6800/37214,v1:172.21.15.52:6801/37214] conn(0x7fee78070f00 0x7fee78073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:41.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.731+0000 7fee9be00640 1 --2- 172.21.15.52:0/1542118887 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fee7806ac00 0x7fee7806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:41.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.732+0000 7feea1c00640 1 --2- 172.21.15.52:0/1542118887 >> [v2:172.21.15.52:6800/37214,v1:172.21.15.52:6801/37214] conn(0x7fee78070f00 0x7fee78073340 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:41.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.732+0000 7fee9be00640 1 --2- 172.21.15.52:0/1542118887 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fee7806ac00 0x7fee7806d0c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fee9c157a20 tx=0x7fee84009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:41.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.732+0000 7fee98c00640 1 -- 172.21.15.52:0/1542118887 <== osd.0 v2:172.21.15.52:6800/37214 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fee78073a10 con 0x7fee78070f00 2024-03-12T17:11:41.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.747+0000 7feea2600640 1 -- 172.21.15.52:0/1542118887 --> [v2:172.21.15.52:6800/37214,v1:172.21.15.52:6801/37214] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fee9c0cd030 con 0x7fee78070f00 2024-03-12T17:11:41.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.747+0000 7fee98c00640 1 -- 172.21.15.52:0/1542118887 <== osd.0 v2:172.21.15.52:6800/37214 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fee9400f040 con 0x7fee78070f00 2024-03-12T17:11:41.741 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-03-12T17:11:41.741 INFO:teuthology.orchestra.run.smithi052.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-12T17:11:41.741 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-03-12T17:11:41.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.748+0000 7feea2600640 1 -- 172.21.15.52:0/1542118887 >> [v2:172.21.15.52:6800/37214,v1:172.21.15.52:6801/37214] conn(0x7fee78070f00 msgr2=0x7fee78073340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:41.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.748+0000 7feea2600640 1 --2- 172.21.15.52:0/1542118887 >> [v2:172.21.15.52:6800/37214,v1:172.21.15.52:6801/37214] conn(0x7fee78070f00 0x7fee78073340 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:41.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.748+0000 7feea2600640 1 -- 172.21.15.52:0/1542118887 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fee7806ac00 msgr2=0x7fee7806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:41.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.748+0000 7feea2600640 1 --2- 172.21.15.52:0/1542118887 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fee7806ac00 0x7fee7806d0c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fee9c157a20 tx=0x7fee84009040 comp rx=0 tx=0).stop 2024-03-12T17:11:41.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.748+0000 7feea2600640 1 -- 172.21.15.52:0/1542118887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee80003b30 msgr2=0x7fee9c0b4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:41.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.748+0000 7feea2600640 1 --2- 172.21.15.52:0/1542118887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee80003b30 0x7fee9c0b4bc0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fee8c007fa0 tx=0x7fee8c001ac0 comp rx=0 tx=0).stop 2024-03-12T17:11:41.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.748+0000 7feea1c00640 1 -- 172.21.15.52:0/1542118887 reap_dead start 2024-03-12T17:11:41.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.748+0000 7feea2600640 1 -- 172.21.15.52:0/1542118887 shutdown_connections 2024-03-12T17:11:41.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.748+0000 7feea2600640 1 -- 172.21.15.52:0/1542118887 >> 172.21.15.52:0/1542118887 conn(0x7fee9c06b050 msgr2=0x7fee9c0d0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:41.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.748+0000 7feea2600640 1 -- 172.21.15.52:0/1542118887 shutdown_connections 2024-03-12T17:11:41.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.748+0000 7feea2600640 1 -- 172.21.15.52:0/1542118887 wait complete. 2024-03-12T17:11:41.758 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-12T17:11:41.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.984+0000 7ffb5fe00640 1 Processor -- start 2024-03-12T17:11:41.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.984+0000 7ffb5fe00640 1 -- start start 2024-03-12T17:11:41.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.986+0000 7ffb5fe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ffb6015c890 0x7ffb60160480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:41.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.986+0000 7ffb5fe00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffb600b80a0 0x7ffb600cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:41.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.986+0000 7ffb5fe00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb60161390 con 0x7ffb600cd9b0 2024-03-12T17:11:41.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.986+0000 7ffb5fe00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb600d0ca0 con 0x7ffb6015c890 2024-03-12T17:11:41.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.986+0000 7ffb5fe00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb600cde60 con 0x7ffb600b80a0 2024-03-12T17:11:41.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.986+0000 7ffb5ea00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ffb6015c890 0x7ffb60160480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:41.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.986+0000 7ffb5ea00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ffb6015c890 0x7ffb60160480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45588/0 (socket says 172.21.15.52:45588) 2024-03-12T17:11:41.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.986+0000 7ffb5ea00640 1 -- 172.21.15.52:0/2870140250 learned_addr learned my addr 172.21.15.52:0/2870140250 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:41.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.986+0000 7ffb56000640 1 --2- 172.21.15.52:0/2870140250 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffb600b80a0 0x7ffb600cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:41.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.987+0000 7ffb5ea00640 1 -- 172.21.15.52:0/2870140250 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffb600b80a0 msgr2=0x7ffb600cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:41.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.987+0000 7ffb5ea00640 1 --2- 172.21.15.52:0/2870140250 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffb600b80a0 0x7ffb600cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:41.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.987+0000 7ffb5ea00640 1 -- 172.21.15.52:0/2870140250 >> v1:172.21.15.102:6789/0 conn(0x7ffb600cd9b0 legacy=0x7ffb600b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:41.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.987+0000 7ffb5ea00640 1 -- 172.21.15.52:0/2870140250 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb60160dd0 con 0x7ffb6015c890 2024-03-12T17:11:41.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.988+0000 7ffb5ea00640 1 --2- 172.21.15.52:0/2870140250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb6015c890 0x7ffb60160480 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ffb50009e50 tx=0x7ffb5002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c061e5f9b50faaf server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:41.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.989+0000 7ffb5e000640 1 -- 172.21.15.52:0/2870140250 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5002ca70 con 0x7ffb6015c890 2024-03-12T17:11:41.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.989+0000 7ffb5e000640 1 -- 172.21.15.52:0/2870140250 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb5002cbd0 con 0x7ffb6015c890 2024-03-12T17:11:41.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.989+0000 7ffb5e000640 1 -- 172.21.15.52:0/2870140250 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb500343f0 con 0x7ffb6015c890 2024-03-12T17:11:41.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.989+0000 7ffb5fe00640 1 -- 172.21.15.52:0/2870140250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb6015c890 msgr2=0x7ffb60160480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:41.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.989+0000 7ffb5fe00640 1 --2- 172.21.15.52:0/2870140250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb6015c890 0x7ffb60160480 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ffb50009e50 tx=0x7ffb5002bed0 comp rx=0 tx=0).stop 2024-03-12T17:11:41.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.989+0000 7ffb5fe00640 1 -- 172.21.15.52:0/2870140250 shutdown_connections 2024-03-12T17:11:41.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.989+0000 7ffb5fe00640 1 --2- 172.21.15.52:0/2870140250 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffb600b80a0 0x7ffb600cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:41.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.989+0000 7ffb5fe00640 1 --2- 172.21.15.52:0/2870140250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb6015c890 0x7ffb60160480 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:41.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.989+0000 7ffb5fe00640 1 -- 172.21.15.52:0/2870140250 >> 172.21.15.52:0/2870140250 conn(0x7ffb6006ae90 msgr2=0x7ffb6006b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:41.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.989+0000 7ffb5fe00640 1 -- 172.21.15.52:0/2870140250 shutdown_connections 2024-03-12T17:11:41.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.989+0000 7ffb5fe00640 1 -- 172.21.15.52:0/2870140250 wait complete. 2024-03-12T17:11:41.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.991+0000 7ffb5fe00640 1 Processor -- start 2024-03-12T17:11:41.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.991+0000 7ffb5fe00640 1 -- start start 2024-03-12T17:11:41.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.992+0000 7ffb5fe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb600b80a0 0x7ffb600b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:41.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.992+0000 7ffb5fe00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffb600cd9b0 0x7ffb600b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:41.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.992+0000 7ffb5fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb6015c890 0x7ffb601f5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:41.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.992+0000 7ffb5fe00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb600c65d0 con 0x7ffb600b80a0 2024-03-12T17:11:41.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.992+0000 7ffb5fe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb600b2ff0 con 0x7ffb6015c890 2024-03-12T17:11:41.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.992+0000 7ffb5fe00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb600b3250 con 0x7ffb600cd9b0 2024-03-12T17:11:41.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.992+0000 7ffb5ea00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb600b80a0 0x7ffb600b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:41.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.992+0000 7ffb5ea00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb600b80a0 0x7ffb600b23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45598/0 (socket says 172.21.15.52:45598) 2024-03-12T17:11:41.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.993+0000 7ffb5ea00640 1 -- 172.21.15.52:0/3844274890 learned_addr learned my addr 172.21.15.52:0/3844274890 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:41.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.993+0000 7ffb5f400640 1 --2- 172.21.15.52:0/3844274890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb6015c890 0x7ffb601f5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:41.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.993+0000 7ffb56000640 1 --2- 172.21.15.52:0/3844274890 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffb600cd9b0 0x7ffb600b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:41.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.993+0000 7ffb5ea00640 1 -- 172.21.15.52:0/3844274890 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffb600cd9b0 msgr2=0x7ffb600b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:41.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.993+0000 7ffb5ea00640 1 --2- 172.21.15.52:0/3844274890 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffb600cd9b0 0x7ffb600b2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:41.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.993+0000 7ffb5ea00640 1 -- 172.21.15.52:0/3844274890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb6015c890 msgr2=0x7ffb601f5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:41.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.993+0000 7ffb5ea00640 1 --2- 172.21.15.52:0/3844274890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb6015c890 0x7ffb601f5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:41.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.993+0000 7ffb5ea00640 1 -- 172.21.15.52:0/3844274890 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb50009b30 con 0x7ffb600b80a0 2024-03-12T17:11:41.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.993+0000 7ffb5ea00640 1 --2- 172.21.15.52:0/3844274890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb600b80a0 0x7ffb600b23d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ffb50000f80 tx=0x7ffb50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:41.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.994+0000 7ffb57400640 1 -- 172.21.15.52:0/3844274890 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb50009510 con 0x7ffb600b80a0 2024-03-12T17:11:41.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.994+0000 7ffb57400640 1 -- 172.21.15.52:0/3844274890 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb5003e450 con 0x7ffb600b80a0 2024-03-12T17:11:41.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.994+0000 7ffb5fe00640 1 -- 172.21.15.52:0/3844274890 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb6015cdf0 con 0x7ffb600b80a0 2024-03-12T17:11:41.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.994+0000 7ffb57400640 1 -- 172.21.15.52:0/3844274890 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5003e800 con 0x7ffb600b80a0 2024-03-12T17:11:41.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.994+0000 7ffb5fe00640 1 -- 172.21.15.52:0/3844274890 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb600b3810 con 0x7ffb600b80a0 2024-03-12T17:11:41.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.995+0000 7ffb5fe00640 1 -- 172.21.15.52:0/3844274890 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb600735a0 con 0x7ffb600b80a0 2024-03-12T17:11:41.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.996+0000 7ffb57400640 1 -- 172.21.15.52:0/3844274890 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ffb5003e960 con 0x7ffb600b80a0 2024-03-12T17:11:41.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.997+0000 7ffb57400640 1 --2- 172.21.15.52:0/3844274890 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ffb3c06ac00 0x7ffb3c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:41.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.997+0000 7ffb57400640 1 -- 172.21.15.52:0/3844274890 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5263+0+0 (secure 0 0 0) 0x7ffb500abf80 con 0x7ffb600b80a0 2024-03-12T17:11:41.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.998+0000 7ffb56000640 1 --2- 172.21.15.52:0/3844274890 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ffb3c06ac00 0x7ffb3c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:41.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:41.999+0000 7ffb56000640 1 --2- 172.21.15.52:0/3844274890 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ffb3c06ac00 0x7ffb3c06d0c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ffb601f5b40 tx=0x7ffb48008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:41.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.002+0000 7ffb57400640 1 -- 172.21.15.52:0/3844274890 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7ffb50077310 con 0x7ffb600b80a0 2024-03-12T17:11:42.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.199+0000 7ffb5fe00640 1 -- 172.21.15.52:0/3844274890 --> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb6015cc70 con 0x7ffb3c06ac00 2024-03-12T17:11:42.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.201+0000 7ffb57400640 1 -- 172.21.15.52:0/3844274890 <== mgr.4109 v2:172.21.15.102:6832/36622 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ffb4800f830 con 0x7ffb3c06ac00 2024-03-12T17:11:42.194 INFO:teuthology.orchestra.run.smithi052.stderr:instructed osd(s) 0 to deep-scrub 2024-03-12T17:11:42.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.203+0000 7ffb5fe00640 1 -- 172.21.15.52:0/3844274890 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ffb3c06ac00 msgr2=0x7ffb3c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.203+0000 7ffb5fe00640 1 --2- 172.21.15.52:0/3844274890 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ffb3c06ac00 0x7ffb3c06d0c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ffb601f5b40 tx=0x7ffb48008040 comp rx=0 tx=0).stop 2024-03-12T17:11:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.203+0000 7ffb5fe00640 1 -- 172.21.15.52:0/3844274890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb600b80a0 msgr2=0x7ffb600b23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.203+0000 7ffb5fe00640 1 --2- 172.21.15.52:0/3844274890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb600b80a0 0x7ffb600b23d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ffb50000f80 tx=0x7ffb50004a40 comp rx=0 tx=0).stop 2024-03-12T17:11:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.204+0000 7ffb5fe00640 1 -- 172.21.15.52:0/3844274890 shutdown_connections 2024-03-12T17:11:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.204+0000 7ffb5fe00640 1 --2- 172.21.15.52:0/3844274890 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7ffb3c06ac00 0x7ffb3c06d0c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.204+0000 7ffb5fe00640 1 --2- 172.21.15.52:0/3844274890 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffb600cd9b0 0x7ffb600b2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.204+0000 7ffb5fe00640 1 --2- 172.21.15.52:0/3844274890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb6015c890 0x7ffb601f5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.204+0000 7ffb5fe00640 1 --2- 172.21.15.52:0/3844274890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb600b80a0 0x7ffb600b23d0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.204+0000 7ffb5fe00640 1 -- 172.21.15.52:0/3844274890 >> 172.21.15.52:0/3844274890 conn(0x7ffb6006ae90 msgr2=0x7ffb6015cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.204+0000 7ffb5fe00640 1 -- 172.21.15.52:0/3844274890 shutdown_connections 2024-03-12T17:11:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.204+0000 7ffb5fe00640 1 -- 172.21.15.52:0/3844274890 wait complete. 2024-03-12T17:11:42.217 INFO:tasks.ceph:Scrubbing osd.1 2024-03-12T17:11:42.217 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-12T17:11:42.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.452+0000 7feb56600640 1 Processor -- start 2024-03-12T17:11:42.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.453+0000 7feb56600640 1 -- start start 2024-03-12T17:11:42.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.454+0000 7feb56600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7feb500cd740 0x7feb500b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:42.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.454+0000 7feb56600640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7feb500b5aa0 0x7feb500b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:42.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.454+0000 7feb56600640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb500d20c0 con 0x7feb500d0d40 2024-03-12T17:11:42.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.454+0000 7feb56600640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb500d14b0 con 0x7feb500b5aa0 2024-03-12T17:11:42.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.454+0000 7feb56600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb500d1790 con 0x7feb500cd740 2024-03-12T17:11:42.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.454+0000 7feb4fe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7feb500b5aa0 0x7feb500b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:42.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.455+0000 7feb4fe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7feb500b5aa0 0x7feb500b5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45610/0 (socket says 172.21.15.52:45610) 2024-03-12T17:11:42.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.455+0000 7feb4ea00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7feb500d0d40 0x7feb500cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51934/0 (socket says 172.21.15.52:51934) 2024-03-12T17:11:42.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.455+0000 7feb4fe00640 1 -- 172.21.15.52:0/292708571 learned_addr learned my addr 172.21.15.52:0/292708571 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:42.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.455+0000 7feb4f400640 1 --2- 172.21.15.52:0/292708571 >> v2:172.21.15.102:3300/0 conn(0x7feb500cd740 0x7feb500b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:42.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.455+0000 7feb4fe00640 1 -- 172.21.15.52:0/292708571 >> v2:172.21.15.102:3300/0 conn(0x7feb500cd740 msgr2=0x7feb500b84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:42.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.455+0000 7feb4fe00640 1 --2- 172.21.15.52:0/292708571 >> v2:172.21.15.102:3300/0 conn(0x7feb500cd740 0x7feb500b84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:42.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.455+0000 7feb4fe00640 1 -- 172.21.15.52:0/292708571 >> v1:172.21.15.52:6789/0 conn(0x7feb500d0d40 legacy=0x7feb500cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:42.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.455+0000 7feb4fe00640 1 -- 172.21.15.52:0/292708571 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb500d1a70 con 0x7feb500b5aa0 2024-03-12T17:11:42.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.455+0000 7feb4e000640 1 -- 172.21.15.52:0/292708571 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1945592883 0 0) 0x7feb500d20c0 con 0x7feb500d0d40 2024-03-12T17:11:42.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.456+0000 7feb4fe00640 1 --2- 172.21.15.52:0/292708571 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb500b5aa0 0x7feb500b5f30 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7feb5008a100 tx=0x7feb4802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8499fced2129d98 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:42.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.457+0000 7feb4e000640 1 -- 172.21.15.52:0/292708571 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb48030860 con 0x7feb500b5aa0 2024-03-12T17:11:42.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.457+0000 7feb4e000640 1 -- 172.21.15.52:0/292708571 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb480309c0 con 0x7feb500b5aa0 2024-03-12T17:11:42.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.457+0000 7feb4e000640 1 -- 172.21.15.52:0/292708571 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb48030d50 con 0x7feb500b5aa0 2024-03-12T17:11:42.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.457+0000 7feb56600640 1 -- 172.21.15.52:0/292708571 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb500b5aa0 msgr2=0x7feb500b5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:42.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.457+0000 7feb56600640 1 --2- 172.21.15.52:0/292708571 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb500b5aa0 0x7feb500b5f30 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7feb5008a100 tx=0x7feb4802b9a0 comp rx=0 tx=0).stop 2024-03-12T17:11:42.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.457+0000 7feb56600640 1 -- 172.21.15.52:0/292708571 shutdown_connections 2024-03-12T17:11:42.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.457+0000 7feb56600640 1 --2- 172.21.15.52:0/292708571 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb500b5aa0 0x7feb500b5f30 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:42.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.457+0000 7feb56600640 1 --2- 172.21.15.52:0/292708571 >> v2:172.21.15.102:3300/0 conn(0x7feb500cd740 0x7feb500b84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:42.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.457+0000 7feb56600640 1 -- 172.21.15.52:0/292708571 >> 172.21.15.52:0/292708571 conn(0x7feb500cf5d0 msgr2=0x7feb500cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:42.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.457+0000 7feb56600640 1 -- 172.21.15.52:0/292708571 shutdown_connections 2024-03-12T17:11:42.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.457+0000 7feb56600640 1 -- 172.21.15.52:0/292708571 wait complete. 2024-03-12T17:11:42.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.459+0000 7feb56600640 1 Processor -- start 2024-03-12T17:11:42.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.459+0000 7feb56600640 1 -- start start 2024-03-12T17:11:42.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.460+0000 7feb56600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb500b5aa0 0x7feb500c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:42.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.460+0000 7feb56600640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feb500cd740 0x7feb500c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:42.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.460+0000 7feb56600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb500d0d40 0x7feb500cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:42.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.460+0000 7feb56600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb500cb0a0 con 0x7feb500d0d40 2024-03-12T17:11:42.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.460+0000 7feb56600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb500c83d0 con 0x7feb500b5aa0 2024-03-12T17:11:42.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.460+0000 7feb56600640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb500ca420 con 0x7feb500cd740 2024-03-12T17:11:42.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.461+0000 7feb4fe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb500d0d40 0x7feb500cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:42.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.461+0000 7feb4fe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb500d0d40 0x7feb500cbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45620/0 (socket says 172.21.15.52:45620) 2024-03-12T17:11:42.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.461+0000 7feb4fe00640 1 -- 172.21.15.52:0/3314962150 learned_addr learned my addr 172.21.15.52:0/3314962150 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:42.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.461+0000 7feb4ea00640 1 --2- 172.21.15.52:0/3314962150 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feb500cd740 0x7feb500c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:42.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.461+0000 7feb4f400640 1 --2- 172.21.15.52:0/3314962150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb500b5aa0 0x7feb500c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:42.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.461+0000 7feb4fe00640 1 -- 172.21.15.52:0/3314962150 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feb500cd740 msgr2=0x7feb500c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:42.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.461+0000 7feb4fe00640 1 --2- 172.21.15.52:0/3314962150 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feb500cd740 0x7feb500c9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:42.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.461+0000 7feb4fe00640 1 -- 172.21.15.52:0/3314962150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb500b5aa0 msgr2=0x7feb500c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:42.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.461+0000 7feb4fe00640 1 --2- 172.21.15.52:0/3314962150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb500b5aa0 0x7feb500c7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:42.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.461+0000 7feb4fe00640 1 -- 172.21.15.52:0/3314962150 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb480094d0 con 0x7feb500d0d40 2024-03-12T17:11:42.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.461+0000 7feb4f400640 1 --2- 172.21.15.52:0/3314962150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb500b5aa0 0x7feb500c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:11:42.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.462+0000 7feb4fe00640 1 --2- 172.21.15.52:0/3314962150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb500d0d40 0x7feb500cbd30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7feb480090b0 tx=0x7feb48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:42.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.462+0000 7feb47400640 1 -- 172.21.15.52:0/3314962150 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb48039c90 con 0x7feb500d0d40 2024-03-12T17:11:42.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.462+0000 7feb47400640 1 -- 172.21.15.52:0/3314962150 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb48039df0 con 0x7feb500d0d40 2024-03-12T17:11:42.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.462+0000 7feb47400640 1 -- 172.21.15.52:0/3314962150 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb48043640 con 0x7feb500d0d40 2024-03-12T17:11:42.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.462+0000 7feb56600640 1 -- 172.21.15.52:0/3314962150 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb500ca700 con 0x7feb500d0d40 2024-03-12T17:11:42.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.462+0000 7feb56600640 1 -- 172.21.15.52:0/3314962150 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb500cacc0 con 0x7feb500d0d40 2024-03-12T17:11:42.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.463+0000 7feb56600640 1 -- 172.21.15.52:0/3314962150 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7feb50160300 con 0x7feb500d0d40 2024-03-12T17:11:42.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.464+0000 7feb47400640 1 -- 172.21.15.52:0/3314962150 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7feb480437a0 con 0x7feb500d0d40 2024-03-12T17:11:42.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.465+0000 7feb47400640 1 --2- 172.21.15.52:0/3314962150 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7feb2c06ac30 0x7feb2c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:42.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.465+0000 7feb47400640 1 -- 172.21.15.52:0/3314962150 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5263+0+0 (secure 0 0 0) 0x7feb4803e070 con 0x7feb500d0d40 2024-03-12T17:11:42.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.465+0000 7feb47400640 1 --2- 172.21.15.52:0/3314962150 >> [v2:172.21.15.52:6817/37218,v1:172.21.15.52:6819/37218] conn(0x7feb2c070f30 0x7feb2c073370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:42.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.465+0000 7feb47400640 1 -- 172.21.15.52:0/3314962150 --> [v2:172.21.15.52:6817/37218,v1:172.21.15.52:6819/37218] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feb2c073a40 con 0x7feb2c070f30 2024-03-12T17:11:42.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.465+0000 7feb47400640 1 -- 172.21.15.52:0/3314962150 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb480ae740 con 0x7feb500d0d40 2024-03-12T17:11:42.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.466+0000 7feb4ea00640 1 --2- 172.21.15.52:0/3314962150 >> [v2:172.21.15.52:6817/37218,v1:172.21.15.52:6819/37218] conn(0x7feb2c070f30 0x7feb2c073370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:42.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.466+0000 7feb4f400640 1 --2- 172.21.15.52:0/3314962150 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7feb2c06ac30 0x7feb2c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:42.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.466+0000 7feb4ea00640 1 --2- 172.21.15.52:0/3314962150 >> [v2:172.21.15.52:6817/37218,v1:172.21.15.52:6819/37218] conn(0x7feb2c070f30 0x7feb2c073370 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:42.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.467+0000 7feb4f400640 1 --2- 172.21.15.52:0/3314962150 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7feb2c06ac30 0x7feb2c06d0f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7feb40007330 tx=0x7feb4000b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:42.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.467+0000 7feb47400640 1 -- 172.21.15.52:0/3314962150 <== osd.1 v2:172.21.15.52:6817/37218 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7feb2c073a40 con 0x7feb2c070f30 2024-03-12T17:11:42.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.481+0000 7feb56600640 1 -- 172.21.15.52:0/3314962150 --> [v2:172.21.15.52:6817/37218,v1:172.21.15.52:6819/37218] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7feb500d11d0 con 0x7feb2c070f30 2024-03-12T17:11:42.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.481+0000 7feb47400640 1 -- 172.21.15.52:0/3314962150 <== osd.1 v2:172.21.15.52:6817/37218 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7feb38007270 con 0x7feb2c070f30 2024-03-12T17:11:42.475 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-03-12T17:11:42.475 INFO:teuthology.orchestra.run.smithi052.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-12T17:11:42.475 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-03-12T17:11:42.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.482+0000 7feb56600640 1 -- 172.21.15.52:0/3314962150 >> [v2:172.21.15.52:6817/37218,v1:172.21.15.52:6819/37218] conn(0x7feb2c070f30 msgr2=0x7feb2c073370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:42.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.482+0000 7feb56600640 1 --2- 172.21.15.52:0/3314962150 >> [v2:172.21.15.52:6817/37218,v1:172.21.15.52:6819/37218] conn(0x7feb2c070f30 0x7feb2c073370 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:42.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.482+0000 7feb56600640 1 -- 172.21.15.52:0/3314962150 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7feb2c06ac30 msgr2=0x7feb2c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:42.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.482+0000 7feb56600640 1 --2- 172.21.15.52:0/3314962150 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7feb2c06ac30 0x7feb2c06d0f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7feb40007330 tx=0x7feb4000b040 comp rx=0 tx=0).stop 2024-03-12T17:11:42.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.482+0000 7feb56600640 1 -- 172.21.15.52:0/3314962150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb500d0d40 msgr2=0x7feb500cbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:42.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.482+0000 7feb56600640 1 --2- 172.21.15.52:0/3314962150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb500d0d40 0x7feb500cbd30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7feb480090b0 tx=0x7feb48004730 comp rx=0 tx=0).stop 2024-03-12T17:11:42.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.482+0000 7feb4fe00640 1 -- 172.21.15.52:0/3314962150 reap_dead start 2024-03-12T17:11:42.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.482+0000 7feb56600640 1 -- 172.21.15.52:0/3314962150 shutdown_connections 2024-03-12T17:11:42.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.482+0000 7feb56600640 1 -- 172.21.15.52:0/3314962150 >> 172.21.15.52:0/3314962150 conn(0x7feb500cf5d0 msgr2=0x7feb500b42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:42.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.482+0000 7feb56600640 1 -- 172.21.15.52:0/3314962150 shutdown_connections 2024-03-12T17:11:42.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.482+0000 7feb56600640 1 -- 172.21.15.52:0/3314962150 wait complete. 2024-03-12T17:11:42.496 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-12T17:11:42.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.724+0000 7fe239800640 1 Processor -- start 2024-03-12T17:11:42.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.724+0000 7fe239800640 1 -- start start 2024-03-12T17:11:42.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.726+0000 7fe239800640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe2340b1db0 0x7fe234167880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:42.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.726+0000 7fe239800640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe234161640 con 0x7fe23413faa0 2024-03-12T17:11:42.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.726+0000 7fe239800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2340b38a0 con 0x7fe234160800 2024-03-12T17:11:42.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.726+0000 7fe239800640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2340b3b80 con 0x7fe2340b1db0 2024-03-12T17:11:42.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.726+0000 7fe233400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe2340b1db0 0x7fe234167880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:42.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.726+0000 7fe232a00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fe23413faa0 0x7fe234160f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51946/0 (socket says 172.21.15.52:51946) 2024-03-12T17:11:42.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.726+0000 7fe232a00640 1 -- 172.21.15.52:0/2772060907 learned_addr learned my addr 172.21.15.52:0/2772060907 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:11:42.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.727+0000 7fe233400640 1 -- 172.21.15.52:0/2772060907 >> v1:172.21.15.102:6789/0 conn(0x7fe234160800 legacy=0x7fe2340b16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:42.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.727+0000 7fe233400640 1 -- 172.21.15.52:0/2772060907 >> v1:172.21.15.52:6789/0 conn(0x7fe23413faa0 legacy=0x7fe234160f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:42.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.727+0000 7fe233400640 1 -- 172.21.15.52:0/2772060907 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2340b3e60 con 0x7fe2340b1db0 2024-03-12T17:11:42.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.727+0000 7fe232000640 1 -- 172.21.15.52:0/2772060907 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2593131269 0 0) 0x7fe234161640 con 0x7fe23413faa0 2024-03-12T17:11:42.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.728+0000 7fe233400640 1 --2- 172.21.15.52:0/2772060907 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2340b1db0 0x7fe234167880 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe22c009d90 tx=0x7fe22c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d3ea8d624990f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:42.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.728+0000 7fe232000640 1 -- 172.21.15.52:0/2772060907 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c02cd00 con 0x7fe2340b1db0 2024-03-12T17:11:42.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.728+0000 7fe232000640 1 -- 172.21.15.52:0/2772060907 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22c02ce60 con 0x7fe2340b1db0 2024-03-12T17:11:42.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.728+0000 7fe232000640 1 -- 172.21.15.52:0/2772060907 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c034660 con 0x7fe2340b1db0 2024-03-12T17:11:42.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.729+0000 7fe239800640 1 -- 172.21.15.52:0/2772060907 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2340b1db0 msgr2=0x7fe234167880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:42.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.729+0000 7fe239800640 1 --2- 172.21.15.52:0/2772060907 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2340b1db0 0x7fe234167880 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe22c009d90 tx=0x7fe22c02bf50 comp rx=0 tx=0).stop 2024-03-12T17:11:42.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.729+0000 7fe239800640 1 -- 172.21.15.52:0/2772060907 shutdown_connections 2024-03-12T17:11:42.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.729+0000 7fe239800640 1 --2- 172.21.15.52:0/2772060907 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2340b1db0 0x7fe234167880 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:42.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.729+0000 7fe239800640 1 -- 172.21.15.52:0/2772060907 >> 172.21.15.52:0/2772060907 conn(0x7fe23406a3d0 msgr2=0x7fe2340b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:42.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.729+0000 7fe239800640 1 -- 172.21.15.52:0/2772060907 shutdown_connections 2024-03-12T17:11:42.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.729+0000 7fe239800640 1 -- 172.21.15.52:0/2772060907 wait complete. 2024-03-12T17:11:42.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.731+0000 7fe239800640 1 Processor -- start 2024-03-12T17:11:42.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.731+0000 7fe239800640 1 -- start start 2024-03-12T17:11:42.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.732+0000 7fe239800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2340b1db0 0x7fe23415db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:42.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.732+0000 7fe239800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe23413faa0 0x7fe2341f2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:42.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.732+0000 7fe239800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe234160800 0x7fe2341f2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:42.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.732+0000 7fe239800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23415e490 con 0x7fe2340b1db0 2024-03-12T17:11:42.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.732+0000 7fe239800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23415e770 con 0x7fe23413faa0 2024-03-12T17:11:42.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.732+0000 7fe239800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe23412d680 con 0x7fe234160800 2024-03-12T17:11:42.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.732+0000 7fe232a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2340b1db0 0x7fe23415db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:42.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.732+0000 7fe232a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2340b1db0 0x7fe23415db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45634/0 (socket says 172.21.15.52:45634) 2024-03-12T17:11:42.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.732+0000 7fe232a00640 1 -- 172.21.15.52:0/2720390195 learned_addr learned my addr 172.21.15.52:0/2720390195 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:42.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.732+0000 7fe233400640 1 --2- 172.21.15.52:0/2720390195 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe234160800 0x7fe2341f2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:42.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.732+0000 7fe232a00640 1 -- 172.21.15.52:0/2720390195 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe234160800 msgr2=0x7fe2341f2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:42.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.732+0000 7fe232a00640 1 --2- 172.21.15.52:0/2720390195 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe234160800 0x7fe2341f2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:42.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.732+0000 7fe22be00640 1 --2- 172.21.15.52:0/2720390195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe23413faa0 0x7fe2341f2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:42.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.732+0000 7fe232a00640 1 -- 172.21.15.52:0/2720390195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe23413faa0 msgr2=0x7fe2341f2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:42.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.733+0000 7fe232a00640 1 --2- 172.21.15.52:0/2720390195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe23413faa0 0x7fe2341f2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:42.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.733+0000 7fe232a00640 1 -- 172.21.15.52:0/2720390195 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22c009a70 con 0x7fe2340b1db0 2024-03-12T17:11:42.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.733+0000 7fe232a00640 1 --2- 172.21.15.52:0/2720390195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2340b1db0 0x7fe23415db90 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe22400c4a0 tx=0x7fe22400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:42.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.733+0000 7fe22aa00640 1 -- 172.21.15.52:0/2720390195 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2240074e0 con 0x7fe2340b1db0 2024-03-12T17:11:42.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.733+0000 7fe22aa00640 1 -- 172.21.15.52:0/2720390195 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe224007640 con 0x7fe2340b1db0 2024-03-12T17:11:42.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.733+0000 7fe22aa00640 1 -- 172.21.15.52:0/2720390195 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe224004a00 con 0x7fe2340b1db0 2024-03-12T17:11:42.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.733+0000 7fe239800640 1 -- 172.21.15.52:0/2720390195 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe23412d960 con 0x7fe2340b1db0 2024-03-12T17:11:42.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.733+0000 7fe239800640 1 -- 172.21.15.52:0/2720390195 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe23415e0d0 con 0x7fe2340b1db0 2024-03-12T17:11:42.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.735+0000 7fe239800640 1 -- 172.21.15.52:0/2720390195 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe23412db50 con 0x7fe2340b1db0 2024-03-12T17:11:42.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.736+0000 7fe22aa00640 1 -- 172.21.15.52:0/2720390195 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe224004b60 con 0x7fe2340b1db0 2024-03-12T17:11:42.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.736+0000 7fe22aa00640 1 --2- 172.21.15.52:0/2720390195 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe21006ac00 0x7fe21006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:42.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.737+0000 7fe22aa00640 1 -- 172.21.15.52:0/2720390195 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fe224088270 con 0x7fe2340b1db0 2024-03-12T17:11:42.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.737+0000 7fe22be00640 1 --2- 172.21.15.52:0/2720390195 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe21006ac00 0x7fe21006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:42.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.738+0000 7fe22be00640 1 --2- 172.21.15.52:0/2720390195 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe21006ac00 0x7fe21006d0c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe234070e60 tx=0x7fe21c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:42.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.741+0000 7fe22aa00640 1 -- 172.21.15.52:0/2720390195 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fe2240536a0 con 0x7fe2340b1db0 2024-03-12T17:11:42.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.940+0000 7fe239800640 1 -- 172.21.15.52:0/2720390195 --> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe23413fe80 con 0x7fe21006ac00 2024-03-12T17:11:42.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.941+0000 7fe22aa00640 1 -- 172.21.15.52:0/2720390195 <== mgr.4109 v2:172.21.15.102:6832/36622 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe21c0012f0 con 0x7fe21006ac00 2024-03-12T17:11:42.934 INFO:teuthology.orchestra.run.smithi052.stderr:instructed osd(s) 1 to deep-scrub 2024-03-12T17:11:42.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.944+0000 7fe239800640 1 -- 172.21.15.52:0/2720390195 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe21006ac00 msgr2=0x7fe21006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:42.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.944+0000 7fe239800640 1 --2- 172.21.15.52:0/2720390195 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe21006ac00 0x7fe21006d0c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe234070e60 tx=0x7fe21c009040 comp rx=0 tx=0).stop 2024-03-12T17:11:42.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.944+0000 7fe239800640 1 -- 172.21.15.52:0/2720390195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2340b1db0 msgr2=0x7fe23415db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:42.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.944+0000 7fe239800640 1 --2- 172.21.15.52:0/2720390195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2340b1db0 0x7fe23415db90 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe22400c4a0 tx=0x7fe22400c960 comp rx=0 tx=0).stop 2024-03-12T17:11:42.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.944+0000 7fe239800640 1 -- 172.21.15.52:0/2720390195 shutdown_connections 2024-03-12T17:11:42.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.944+0000 7fe239800640 1 --2- 172.21.15.52:0/2720390195 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fe21006ac00 0x7fe21006d0c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:42.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.944+0000 7fe239800640 1 --2- 172.21.15.52:0/2720390195 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe234160800 0x7fe2341f2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:42.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.944+0000 7fe239800640 1 --2- 172.21.15.52:0/2720390195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe23413faa0 0x7fe2341f2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:42.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.944+0000 7fe239800640 1 --2- 172.21.15.52:0/2720390195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2340b1db0 0x7fe23415db90 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:42.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.944+0000 7fe239800640 1 -- 172.21.15.52:0/2720390195 >> 172.21.15.52:0/2720390195 conn(0x7fe23406a3d0 msgr2=0x7fe2340acf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:42.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.944+0000 7fe239800640 1 -- 172.21.15.52:0/2720390195 shutdown_connections 2024-03-12T17:11:42.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:42.944+0000 7fe239800640 1 -- 172.21.15.52:0/2720390195 wait complete. 2024-03-12T17:11:42.956 INFO:tasks.ceph:Scrubbing osd.2 2024-03-12T17:11:42.956 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-12T17:11:43.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.176+0000 7f7aa2000640 1 Processor -- start 2024-03-12T17:11:43.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.176+0000 7f7aa2000640 1 -- start start 2024-03-12T17:11:43.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.177+0000 7f7aa2000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f7a9c0cd740 0x7f7a9c0b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:43.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.177+0000 7f7aa2000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7a9c0b5aa0 0x7f7a9c0b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:43.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.177+0000 7f7aa2000640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a9c0d20c0 con 0x7f7a9c0d0d40 2024-03-12T17:11:43.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.177+0000 7f7aa2000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a9c0d14b0 con 0x7f7a9c0cd740 2024-03-12T17:11:43.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.177+0000 7f7aa2000640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c0d1790 con 0x7f7a9c0b5aa0 2024-03-12T17:11:43.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.178+0000 7f7a9aa00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f7a9c0d0d40 0x7f7a9c0cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51956/0 (socket says 172.21.15.52:51956) 2024-03-12T17:11:43.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.178+0000 7f7a9aa00640 1 -- 172.21.15.52:0/2867220582 learned_addr learned my addr 172.21.15.52:0/2867220582 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:11:43.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.178+0000 7f7a9b400640 1 --2- 172.21.15.52:0/2867220582 >> v2:172.21.15.102:3300/0 conn(0x7f7a9c0cd740 0x7f7a9c0b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:43.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.178+0000 7f7a9be00640 1 --2- 172.21.15.52:0/2867220582 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7a9c0b5aa0 0x7f7a9c0b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:43.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.178+0000 7f7a9a000640 1 -- 172.21.15.52:0/2867220582 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2390773247 0 0) 0x7f7a9c0d20c0 con 0x7f7a9c0d0d40 2024-03-12T17:11:43.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.178+0000 7f7a9a000640 1 -- 172.21.15.52:0/2867220582 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a80003520 con 0x7f7a9c0d0d40 2024-03-12T17:11:43.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.179+0000 7f7a9a000640 1 -- 172.21.15.52:0/2867220582 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 996649792 0 0) 0x7f7a80003520 con 0x7f7a9c0d0d40 2024-03-12T17:11:43.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.179+0000 7f7a9a000640 1 -- 172.21.15.52:0/2867220582 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7a9c0b5aa0 msgr2=0x7f7a9c0b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:43.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.179+0000 7f7a9a000640 1 --2- 172.21.15.52:0/2867220582 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7a9c0b5aa0 0x7f7a9c0b5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:43.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.179+0000 7f7a9a000640 1 -- 172.21.15.52:0/2867220582 >> v2:172.21.15.102:3300/0 conn(0x7f7a9c0cd740 msgr2=0x7f7a9c0b84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:43.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.179+0000 7f7a9a000640 1 --2- 172.21.15.52:0/2867220582 >> v2:172.21.15.102:3300/0 conn(0x7f7a9c0cd740 0x7f7a9c0b84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:43.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.179+0000 7f7a9a000640 1 -- 172.21.15.52:0/2867220582 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a9c0d1a70 con 0x7f7a9c0d0d40 2024-03-12T17:11:43.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.179+0000 7f7a9a000640 1 -- 172.21.15.52:0/2867220582 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f7a88001670 con 0x7f7a9c0d0d40 2024-03-12T17:11:43.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.179+0000 7f7a9a000640 1 -- 172.21.15.52:0/2867220582 >> v1:172.21.15.52:6789/0 conn(0x7f7a9c0d0d40 legacy=0x7f7a9c0cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:43.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.179+0000 7f7a9a000640 1 --2- 172.21.15.52:0/2867220582 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a80003970 0x7f7a80003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:43.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.179+0000 7f7a9a000640 1 --2- 172.21.15.52:0/2867220582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a800047c0 0x7f7a80004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:43.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.179+0000 7f7a9a000640 1 --2- 172.21.15.52:0/2867220582 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7a9c0b5aa0 0x7f7a80005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:43.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.179+0000 7f7a9a000640 1 -- 172.21.15.52:0/2867220582 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c0d14b0 con 0x7f7a800047c0 2024-03-12T17:11:43.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.179+0000 7f7a9a000640 1 -- 172.21.15.52:0/2867220582 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c0d1790 con 0x7f7a80003970 2024-03-12T17:11:43.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.179+0000 7f7a9a000640 1 -- 172.21.15.52:0/2867220582 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8000b980 con 0x7f7a9c0b5aa0 2024-03-12T17:11:43.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.179+0000 7f7a9aa00640 1 --2- 172.21.15.52:0/2867220582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a800047c0 0x7f7a80004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:43.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.180+0000 7f7a9b400640 1 --2- 172.21.15.52:0/2867220582 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a80003970 0x7f7a80003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:43.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.180+0000 7f7a9be00640 1 --2- 172.21.15.52:0/2867220582 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7a9c0b5aa0 0x7f7a80005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:43.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.180+0000 7f7a9aa00640 1 -- 172.21.15.52:0/2867220582 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7a9c0b5aa0 msgr2=0x7f7a80005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:43.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.180+0000 7f7a9aa00640 1 --2- 172.21.15.52:0/2867220582 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7a9c0b5aa0 0x7f7a80005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:43.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.180+0000 7f7a9aa00640 1 -- 172.21.15.52:0/2867220582 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a80003970 msgr2=0x7f7a80003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:43.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.180+0000 7f7a9aa00640 1 --2- 172.21.15.52:0/2867220582 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a80003970 0x7f7a80003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:43.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.180+0000 7f7a9aa00640 1 -- 172.21.15.52:0/2867220582 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a80003520 con 0x7f7a800047c0 2024-03-12T17:11:43.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.180+0000 7f7a9be00640 1 --2- 172.21.15.52:0/2867220582 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7a9c0b5aa0 0x7f7a80005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:11:43.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.181+0000 7f7a9aa00640 1 --2- 172.21.15.52:0/2867220582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a800047c0 0x7f7a80004c40 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800c110 tx=0x7f7a800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d7518be8fbf564 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:43.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.181+0000 7f7a9a000640 1 -- 172.21.15.52:0/2867220582 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a88005ad0 con 0x7f7a800047c0 2024-03-12T17:11:43.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.181+0000 7f7a9a000640 1 -- 172.21.15.52:0/2867220582 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a88005c50 con 0x7f7a800047c0 2024-03-12T17:11:43.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.181+0000 7f7a9a000640 1 -- 172.21.15.52:0/2867220582 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8802c830 con 0x7f7a800047c0 2024-03-12T17:11:43.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.182+0000 7f7aa2000640 1 -- 172.21.15.52:0/2867220582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a800047c0 msgr2=0x7f7a80004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:43.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.182+0000 7f7aa2000640 1 --2- 172.21.15.52:0/2867220582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a800047c0 0x7f7a80004c40 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800c110 tx=0x7f7a800057f0 comp rx=0 tx=0).stop 2024-03-12T17:11:43.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.182+0000 7f7a9be00640 1 -- 172.21.15.52:0/2867220582 reap_dead start 2024-03-12T17:11:43.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.182+0000 7f7aa2000640 1 -- 172.21.15.52:0/2867220582 shutdown_connections 2024-03-12T17:11:43.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.182+0000 7f7aa2000640 1 -- 172.21.15.52:0/2867220582 >> 172.21.15.52:0/2867220582 conn(0x7f7a9c0cf5d0 msgr2=0x7f7a9c0cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:43.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.182+0000 7f7aa2000640 1 -- 172.21.15.52:0/2867220582 shutdown_connections 2024-03-12T17:11:43.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.182+0000 7f7aa2000640 1 -- 172.21.15.52:0/2867220582 wait complete. 2024-03-12T17:11:43.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.184+0000 7f7aa2000640 1 Processor -- start 2024-03-12T17:11:43.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.184+0000 7f7aa2000640 1 -- start start 2024-03-12T17:11:43.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.185+0000 7f7aa2000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a800047c0 0x7f7a9c14c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:43.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.185+0000 7f7aa2000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7a9c1f2760 0x7f7a9c0cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:43.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.185+0000 7f7aa2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a9c0cc150 0x7f7a9c1f53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:43.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.185+0000 7f7aa2000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c0af6c0 con 0x7f7a800047c0 2024-03-12T17:11:43.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.185+0000 7f7aa2000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c0a9270 con 0x7f7a9c0cc150 2024-03-12T17:11:43.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.185+0000 7f7aa2000640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c0cc5a0 con 0x7f7a9c1f2760 2024-03-12T17:11:43.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.185+0000 7f7a9b400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a800047c0 0x7f7a9c14c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:43.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.185+0000 7f7a9b400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a800047c0 0x7f7a9c14c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45650/0 (socket says 172.21.15.52:45650) 2024-03-12T17:11:43.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.185+0000 7f7a9b400640 1 -- 172.21.15.52:0/857972813 learned_addr learned my addr 172.21.15.52:0/857972813 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:43.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.185+0000 7f7a9b400640 1 -- 172.21.15.52:0/857972813 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7a9c1f2760 msgr2=0x7f7a9c0cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:43.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.185+0000 7f7a9b400640 1 --2- 172.21.15.52:0/857972813 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7a9c1f2760 0x7f7a9c0cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:43.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.185+0000 7f7a9b400640 1 -- 172.21.15.52:0/857972813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a9c0cc150 msgr2=0x7f7a9c1f53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:43.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.185+0000 7f7a9b400640 1 --2- 172.21.15.52:0/857972813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a9c0cc150 0x7f7a9c1f53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:43.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.185+0000 7f7a9b400640 1 -- 172.21.15.52:0/857972813 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a9c14cbe0 con 0x7f7a800047c0 2024-03-12T17:11:43.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.186+0000 7f7a9b400640 1 --2- 172.21.15.52:0/857972813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a800047c0 0x7f7a9c14c6a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7a94007ea0 tx=0x7f7a94007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:43.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.186+0000 7f7a93400640 1 -- 172.21.15.52:0/857972813 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9400e030 con 0x7f7a800047c0 2024-03-12T17:11:43.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.186+0000 7f7a93400640 1 -- 172.21.15.52:0/857972813 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a94011e90 con 0x7f7a800047c0 2024-03-12T17:11:43.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.186+0000 7f7a93400640 1 -- 172.21.15.52:0/857972813 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a94016690 con 0x7f7a800047c0 2024-03-12T17:11:43.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.186+0000 7f7aa2000640 1 -- 172.21.15.52:0/857972813 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a9c0a9450 con 0x7f7a800047c0 2024-03-12T17:11:43.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.186+0000 7f7aa2000640 1 -- 172.21.15.52:0/857972813 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a9c0a9a10 con 0x7f7a800047c0 2024-03-12T17:11:43.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.187+0000 7f7aa2000640 1 -- 172.21.15.52:0/857972813 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7a9c160300 con 0x7f7a800047c0 2024-03-12T17:11:43.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.188+0000 7f7a93400640 1 -- 172.21.15.52:0/857972813 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7a940167f0 con 0x7f7a800047c0 2024-03-12T17:11:43.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.189+0000 7f7a93400640 1 --2- 172.21.15.52:0/857972813 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f7a7c06ac00 0x7f7a7c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:43.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.189+0000 7f7a93400640 1 -- 172.21.15.52:0/857972813 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f7a9400f070 con 0x7f7a800047c0 2024-03-12T17:11:43.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.189+0000 7f7a93400640 1 --2- 172.21.15.52:0/857972813 >> [v2:172.21.15.52:6808/37216,v1:172.21.15.52:6809/37216] conn(0x7f7a7c070f00 0x7f7a7c073340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:43.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.189+0000 7f7a93400640 1 -- 172.21.15.52:0/857972813 --> [v2:172.21.15.52:6808/37216,v1:172.21.15.52:6809/37216] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7a7c073a10 con 0x7f7a7c070f00 2024-03-12T17:11:43.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.189+0000 7f7a93400640 1 -- 172.21.15.52:0/857972813 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a9408c4b0 con 0x7f7a800047c0 2024-03-12T17:11:43.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.190+0000 7f7a9be00640 1 --2- 172.21.15.52:0/857972813 >> [v2:172.21.15.52:6808/37216,v1:172.21.15.52:6809/37216] conn(0x7f7a7c070f00 0x7f7a7c073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:43.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.190+0000 7f7a9aa00640 1 --2- 172.21.15.52:0/857972813 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f7a7c06ac00 0x7f7a7c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:43.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.190+0000 7f7a9be00640 1 --2- 172.21.15.52:0/857972813 >> [v2:172.21.15.52:6808/37216,v1:172.21.15.52:6809/37216] conn(0x7f7a7c070f00 0x7f7a7c073340 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:43.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.191+0000 7f7a9aa00640 1 --2- 172.21.15.52:0/857972813 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f7a7c06ac00 0x7f7a7c06d0c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7a88034000 tx=0x7f7a8802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:43.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.191+0000 7f7a93400640 1 -- 172.21.15.52:0/857972813 <== osd.2 v2:172.21.15.52:6808/37216 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f7a7c073a10 con 0x7f7a7c070f00 2024-03-12T17:11:43.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.205+0000 7f7aa2000640 1 -- 172.21.15.52:0/857972813 --> [v2:172.21.15.52:6808/37216,v1:172.21.15.52:6809/37216] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7a9c0d11d0 con 0x7f7a7c070f00 2024-03-12T17:11:43.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.205+0000 7f7a93400640 1 -- 172.21.15.52:0/857972813 <== osd.2 v2:172.21.15.52:6808/37216 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f7a8c005990 con 0x7f7a7c070f00 2024-03-12T17:11:43.199 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-03-12T17:11:43.199 INFO:teuthology.orchestra.run.smithi052.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-12T17:11:43.199 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-03-12T17:11:43.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.206+0000 7f7aa2000640 1 -- 172.21.15.52:0/857972813 >> [v2:172.21.15.52:6808/37216,v1:172.21.15.52:6809/37216] conn(0x7f7a7c070f00 msgr2=0x7f7a7c073340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:43.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.206+0000 7f7aa2000640 1 --2- 172.21.15.52:0/857972813 >> [v2:172.21.15.52:6808/37216,v1:172.21.15.52:6809/37216] conn(0x7f7a7c070f00 0x7f7a7c073340 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:43.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.206+0000 7f7aa2000640 1 -- 172.21.15.52:0/857972813 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f7a7c06ac00 msgr2=0x7f7a7c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:43.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.206+0000 7f7aa2000640 1 --2- 172.21.15.52:0/857972813 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f7a7c06ac00 0x7f7a7c06d0c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7a88034000 tx=0x7f7a8802e000 comp rx=0 tx=0).stop 2024-03-12T17:11:43.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.206+0000 7f7aa2000640 1 -- 172.21.15.52:0/857972813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a800047c0 msgr2=0x7f7a9c14c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:43.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.206+0000 7f7aa2000640 1 --2- 172.21.15.52:0/857972813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a800047c0 0x7f7a9c14c6a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7a94007ea0 tx=0x7f7a94007f90 comp rx=0 tx=0).stop 2024-03-12T17:11:43.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.206+0000 7f7a9be00640 1 -- 172.21.15.52:0/857972813 reap_dead start 2024-03-12T17:11:43.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.206+0000 7f7aa2000640 1 -- 172.21.15.52:0/857972813 shutdown_connections 2024-03-12T17:11:43.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.206+0000 7f7aa2000640 1 -- 172.21.15.52:0/857972813 >> 172.21.15.52:0/857972813 conn(0x7f7a9c0cf5d0 msgr2=0x7f7a9c15ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:43.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.206+0000 7f7aa2000640 1 -- 172.21.15.52:0/857972813 shutdown_connections 2024-03-12T17:11:43.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.206+0000 7f7aa2000640 1 -- 172.21.15.52:0/857972813 wait complete. 2024-03-12T17:11:43.216 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-12T17:11:43.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.430+0000 7f1aa1000640 1 Processor -- start 2024-03-12T17:11:43.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.431+0000 7f1aa1000640 1 -- start start 2024-03-12T17:11:43.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.432+0000 7f1aa1000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1a9c15cce0 0x7f1a9c0cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:43.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.432+0000 7f1aa1000640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a9c15d0c0 con 0x7f1a9c0b1740 2024-03-12T17:11:43.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.432+0000 7f1aa1000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a9c0d0ae0 con 0x7f1a9c0b4350 2024-03-12T17:11:43.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.432+0000 7f1aa1000640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c0d0d80 con 0x7f1a9c15cce0 2024-03-12T17:11:43.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.433+0000 7f1a99400640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f1a9c0b1740 0x7f1a9c0b5ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51960/0 (socket says 172.21.15.52:51960) 2024-03-12T17:11:43.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.433+0000 7f1a99400640 1 -- 172.21.15.52:0/2999323413 learned_addr learned my addr 172.21.15.52:0/2999323413 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:11:43.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.433+0000 7f1a99e00640 1 --2- 172.21.15.52:0/2999323413 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1a9c15cce0 0x7f1a9c0cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:43.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.433+0000 7f1a98a00640 1 -- 172.21.15.52:0/2999323413 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2660542007 0 0) 0x7f1a9c15d0c0 con 0x7f1a9c0b1740 2024-03-12T17:11:43.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.433+0000 7f1a98a00640 1 -- 172.21.15.52:0/2999323413 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a80003520 con 0x7f1a9c0b1740 2024-03-12T17:11:43.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.434+0000 7f1a98a00640 1 -- 172.21.15.52:0/2999323413 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1471768105 0 0) 0x7f1a80003520 con 0x7f1a9c0b1740 2024-03-12T17:11:43.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.434+0000 7f1a98a00640 1 -- 172.21.15.52:0/2999323413 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1a9c15cce0 msgr2=0x7f1a9c0cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:43.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.434+0000 7f1a98a00640 1 --2- 172.21.15.52:0/2999323413 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1a9c15cce0 0x7f1a9c0cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:43.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.434+0000 7f1a98a00640 1 -- 172.21.15.52:0/2999323413 >> v1:172.21.15.102:6789/0 conn(0x7f1a9c0b4350 legacy=0x7f1a9c0d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:43.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.434+0000 7f1a98a00640 1 -- 172.21.15.52:0/2999323413 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a9c0d1060 con 0x7f1a9c0b1740 2024-03-12T17:11:43.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.434+0000 7f1a98a00640 1 -- 172.21.15.52:0/2999323413 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7f1a88001670 con 0x7f1a9c0b1740 2024-03-12T17:11:43.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.434+0000 7f1a98a00640 1 -- 172.21.15.52:0/2999323413 >> v1:172.21.15.52:6789/0 conn(0x7f1a9c0b1740 legacy=0x7f1a9c0b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:43.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.434+0000 7f1a98a00640 1 --2- 172.21.15.52:0/2999323413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a800039a0 0x7f1a80003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:43.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.434+0000 7f1a98a00640 1 --2- 172.21.15.52:0/2999323413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a800047c0 0x7f1a80004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:43.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.434+0000 7f1a98a00640 1 --2- 172.21.15.52:0/2999323413 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1a9c15cce0 0x7f1a80005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:43.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.434+0000 7f1a98a00640 1 -- 172.21.15.52:0/2999323413 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c0d0d80 con 0x7f1a800039a0 2024-03-12T17:11:43.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.434+0000 7f1a98a00640 1 -- 172.21.15.52:0/2999323413 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8000ba30 con 0x7f1a800047c0 2024-03-12T17:11:43.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.434+0000 7f1a98a00640 1 -- 172.21.15.52:0/2999323413 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8000bbf0 con 0x7f1a9c15cce0 2024-03-12T17:11:43.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.434+0000 7f1a99e00640 1 --2- 172.21.15.52:0/2999323413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a800039a0 0x7f1a80003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:43.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.434+0000 7f1a9a800640 1 --2- 172.21.15.52:0/2999323413 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1a9c15cce0 0x7f1a80005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:43.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.434+0000 7f1a99400640 1 --2- 172.21.15.52:0/2999323413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a800047c0 0x7f1a80004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:43.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.435+0000 7f1a99e00640 1 -- 172.21.15.52:0/2999323413 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1a9c15cce0 msgr2=0x7f1a80005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:43.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.435+0000 7f1a99e00640 1 --2- 172.21.15.52:0/2999323413 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1a9c15cce0 0x7f1a80005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:43.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.435+0000 7f1a99e00640 1 -- 172.21.15.52:0/2999323413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a800047c0 msgr2=0x7f1a80004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:43.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.435+0000 7f1a99e00640 1 --2- 172.21.15.52:0/2999323413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a800047c0 0x7f1a80004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:43.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.435+0000 7f1a99e00640 1 -- 172.21.15.52:0/2999323413 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a80003520 con 0x7f1a800039a0 2024-03-12T17:11:43.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.435+0000 7f1a99400640 1 --2- 172.21.15.52:0/2999323413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a800047c0 0x7f1a80004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:11:43.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.436+0000 7f1a99e00640 1 --2- 172.21.15.52:0/2999323413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a800039a0 0x7f1a80003dc0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c009a90 tx=0x7f1a800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d8f01a979320088 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:43.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.436+0000 7f1a98a00640 1 -- 172.21.15.52:0/2999323413 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c02c660 con 0x7f1a800039a0 2024-03-12T17:11:43.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.436+0000 7f1a98a00640 1 -- 172.21.15.52:0/2999323413 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8c02c7c0 con 0x7f1a800039a0 2024-03-12T17:11:43.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.436+0000 7f1a98a00640 1 -- 172.21.15.52:0/2999323413 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c02cb50 con 0x7f1a800039a0 2024-03-12T17:11:43.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.437+0000 7f1aa1000640 1 -- 172.21.15.52:0/2999323413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a800039a0 msgr2=0x7f1a80003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:43.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.437+0000 7f1aa1000640 1 --2- 172.21.15.52:0/2999323413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a800039a0 0x7f1a80003dc0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c009a90 tx=0x7f1a800057f0 comp rx=0 tx=0).stop 2024-03-12T17:11:43.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.437+0000 7f1a9a800640 1 -- 172.21.15.52:0/2999323413 reap_dead start 2024-03-12T17:11:43.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.437+0000 7f1aa1000640 1 -- 172.21.15.52:0/2999323413 shutdown_connections 2024-03-12T17:11:43.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.437+0000 7f1aa1000640 1 -- 172.21.15.52:0/2999323413 >> 172.21.15.52:0/2999323413 conn(0x7f1a9c0cf350 msgr2=0x7f1a9c0cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:43.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.437+0000 7f1aa1000640 1 -- 172.21.15.52:0/2999323413 shutdown_connections 2024-03-12T17:11:43.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.437+0000 7f1aa1000640 1 -- 172.21.15.52:0/2999323413 wait complete. 2024-03-12T17:11:43.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.438+0000 7f1aa1000640 1 Processor -- start 2024-03-12T17:11:43.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.439+0000 7f1aa1000640 1 -- start start 2024-03-12T17:11:43.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.439+0000 7f1aa1000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1a800039a0 0x7f1a9c0b7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:43.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.439+0000 7f1aa1000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a9c0b5ce0 0x7f1a9c1f21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:43.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.439+0000 7f1aa1000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a9c0ca200 0x7f1a9c0cc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:43.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.439+0000 7f1aa1000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c0b7c40 con 0x7f1a9c0b5ce0 2024-03-12T17:11:43.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.439+0000 7f1aa1000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c15e510 con 0x7f1a9c0ca200 2024-03-12T17:11:43.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.439+0000 7f1aa1000640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c15e7f0 con 0x7f1a800039a0 2024-03-12T17:11:43.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.440+0000 7f1a99400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a9c0b5ce0 0x7f1a9c1f21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:43.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.440+0000 7f1a99400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a9c0b5ce0 0x7f1a9c1f21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45680/0 (socket says 172.21.15.52:45680) 2024-03-12T17:11:43.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.440+0000 7f1a99400640 1 -- 172.21.15.52:0/3870042855 learned_addr learned my addr 172.21.15.52:0/3870042855 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:43.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.440+0000 7f1a99e00640 1 --2- 172.21.15.52:0/3870042855 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1a800039a0 0x7f1a9c0b7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:43.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.440+0000 7f1a9a800640 1 --2- 172.21.15.52:0/3870042855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a9c0ca200 0x7f1a9c0cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:43.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.440+0000 7f1a99400640 1 -- 172.21.15.52:0/3870042855 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1a800039a0 msgr2=0x7f1a9c0b7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:43.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.440+0000 7f1a99400640 1 --2- 172.21.15.52:0/3870042855 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1a800039a0 0x7f1a9c0b7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:43.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.440+0000 7f1a99400640 1 -- 172.21.15.52:0/3870042855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a9c0ca200 msgr2=0x7f1a9c0cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:43.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.440+0000 7f1a99400640 1 --2- 172.21.15.52:0/3870042855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a9c0ca200 0x7f1a9c0cc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:43.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.440+0000 7f1a99400640 1 -- 172.21.15.52:0/3870042855 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a9c15ead0 con 0x7f1a9c0b5ce0 2024-03-12T17:11:43.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.440+0000 7f1a99e00640 1 --2- 172.21.15.52:0/3870042855 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1a800039a0 0x7f1a9c0b7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:11:43.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.440+0000 7f1a99400640 1 --2- 172.21.15.52:0/3870042855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a9c0b5ce0 0x7f1a9c1f21d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1a8800d860 tx=0x7f1a8800dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:43.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.441+0000 7f1a92000640 1 -- 172.21.15.52:0/3870042855 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a880074e0 con 0x7f1a9c0b5ce0 2024-03-12T17:11:43.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.441+0000 7f1a92000640 1 -- 172.21.15.52:0/3870042855 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a88007640 con 0x7f1a9c0b5ce0 2024-03-12T17:11:43.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.441+0000 7f1a92000640 1 -- 172.21.15.52:0/3870042855 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a880049e0 con 0x7f1a9c0b5ce0 2024-03-12T17:11:43.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.441+0000 7f1aa1000640 1 -- 172.21.15.52:0/3870042855 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a9c15edb0 con 0x7f1a9c0b5ce0 2024-03-12T17:11:43.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.441+0000 7f1aa1000640 1 -- 172.21.15.52:0/3870042855 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a9c1f2710 con 0x7f1a9c0b5ce0 2024-03-12T17:11:43.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.442+0000 7f1aa1000640 1 -- 172.21.15.52:0/3870042855 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a9c073390 con 0x7f1a9c0b5ce0 2024-03-12T17:11:43.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.443+0000 7f1a92000640 1 -- 172.21.15.52:0/3870042855 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1a88004b40 con 0x7f1a9c0b5ce0 2024-03-12T17:11:43.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.444+0000 7f1a92000640 1 --2- 172.21.15.52:0/3870042855 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1a7c06ac00 0x7f1a7c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:43.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.444+0000 7f1a92000640 1 -- 172.21.15.52:0/3870042855 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f1a88088be0 con 0x7f1a9c0b5ce0 2024-03-12T17:11:43.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.444+0000 7f1a99e00640 1 --2- 172.21.15.52:0/3870042855 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1a7c06ac00 0x7f1a7c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:43.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.445+0000 7f1a99e00640 1 --2- 172.21.15.52:0/3870042855 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1a7c06ac00 0x7f1a7c06d0c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c036000 tx=0x7f1a8c030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:43.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.448+0000 7f1a92000640 1 -- 172.21.15.52:0/3870042855 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f1a88053ee0 con 0x7f1a9c0b5ce0 2024-03-12T17:11:43.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.644+0000 7f1aa1000640 1 -- 172.21.15.52:0/3870042855 --> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a9c0d1620 con 0x7f1a7c06ac00 2024-03-12T17:11:43.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.645+0000 7f1a92000640 1 -- 172.21.15.52:0/3870042855 <== mgr.4109 v2:172.21.15.102:6832/36622 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1a8c004140 con 0x7f1a7c06ac00 2024-03-12T17:11:43.639 INFO:teuthology.orchestra.run.smithi052.stderr:instructed osd(s) 2 to deep-scrub 2024-03-12T17:11:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.649+0000 7f1aa1000640 1 -- 172.21.15.52:0/3870042855 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1a7c06ac00 msgr2=0x7f1a7c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.649+0000 7f1aa1000640 1 --2- 172.21.15.52:0/3870042855 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1a7c06ac00 0x7f1a7c06d0c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c036000 tx=0x7f1a8c030000 comp rx=0 tx=0).stop 2024-03-12T17:11:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.649+0000 7f1aa1000640 1 -- 172.21.15.52:0/3870042855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a9c0b5ce0 msgr2=0x7f1a9c1f21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.649+0000 7f1aa1000640 1 --2- 172.21.15.52:0/3870042855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a9c0b5ce0 0x7f1a9c1f21d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1a8800d860 tx=0x7f1a8800dd20 comp rx=0 tx=0).stop 2024-03-12T17:11:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.649+0000 7f1aa1000640 1 -- 172.21.15.52:0/3870042855 shutdown_connections 2024-03-12T17:11:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.649+0000 7f1aa1000640 1 --2- 172.21.15.52:0/3870042855 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1a7c06ac00 0x7f1a7c06d0c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.649+0000 7f1aa1000640 1 --2- 172.21.15.52:0/3870042855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a9c0ca200 0x7f1a9c0cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.649+0000 7f1aa1000640 1 --2- 172.21.15.52:0/3870042855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a9c0b5ce0 0x7f1a9c1f21d0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.649+0000 7f1aa1000640 1 --2- 172.21.15.52:0/3870042855 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1a800039a0 0x7f1a9c0b7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.649+0000 7f1aa1000640 1 -- 172.21.15.52:0/3870042855 >> 172.21.15.52:0/3870042855 conn(0x7f1a9c0cf350 msgr2=0x7f1a9c0b47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.649+0000 7f1aa1000640 1 -- 172.21.15.52:0/3870042855 shutdown_connections 2024-03-12T17:11:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.649+0000 7f1aa1000640 1 -- 172.21.15.52:0/3870042855 wait complete. 2024-03-12T17:11:43.661 INFO:tasks.ceph:Scrubbing osd.3 2024-03-12T17:11:43.661 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-12T17:11:43.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.885+0000 7f2dd5e00640 1 Processor -- start 2024-03-12T17:11:43.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.886+0000 7f2dd5e00640 1 -- start start 2024-03-12T17:11:43.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.887+0000 7f2dd5e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2dd015c820 0x7f2dd00cd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:43.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.887+0000 7f2dd5e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2dd0160e90 0x7f2dd00cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:43.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.887+0000 7f2dd5e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dd00ccd90 con 0x7f2dd00cf5f0 2024-03-12T17:11:43.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.887+0000 7f2dd5e00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dd00cdd70 con 0x7f2dd015c820 2024-03-12T17:11:43.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.887+0000 7f2dd5e00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd00cfa60 con 0x7f2dd0160e90 2024-03-12T17:11:43.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.887+0000 7f2dd4a00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2dd015c820 0x7f2dd00cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:43.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.887+0000 7f2dd4a00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2dd015c820 0x7f2dd00cd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45686/0 (socket says 172.21.15.52:45686) 2024-03-12T17:11:43.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.887+0000 7f2dd4a00640 1 -- 172.21.15.52:0/3093557204 learned_addr learned my addr 172.21.15.52:0/3093557204 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:43.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.888+0000 7f2dd4a00640 1 -- 172.21.15.52:0/3093557204 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2dd0160e90 msgr2=0x7f2dd00cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:43.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.888+0000 7f2dd4a00640 1 --2- 172.21.15.52:0/3093557204 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2dd0160e90 0x7f2dd00cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:43.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.888+0000 7f2dd4a00640 1 -- 172.21.15.52:0/3093557204 >> v1:172.21.15.102:6789/0 conn(0x7f2dd00cf5f0 legacy=0x7f2dd00d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:43.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.888+0000 7f2dd4a00640 1 -- 172.21.15.52:0/3093557204 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd00b7fc0 con 0x7f2dd015c820 2024-03-12T17:11:43.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.889+0000 7f2dd4a00640 1 --2- 172.21.15.52:0/3093557204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2dd015c820 0x7f2dd00cd830 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4009870 tx=0x7f2dc402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e28f9841786f41f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:43.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.889+0000 7f2dcf400640 1 -- 172.21.15.52:0/3093557204 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4030840 con 0x7f2dd015c820 2024-03-12T17:11:43.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.889+0000 7f2dcf400640 1 -- 172.21.15.52:0/3093557204 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc40309a0 con 0x7f2dd015c820 2024-03-12T17:11:43.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.889+0000 7f2dcf400640 1 -- 172.21.15.52:0/3093557204 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4030d30 con 0x7f2dd015c820 2024-03-12T17:11:43.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.890+0000 7f2dd5e00640 1 -- 172.21.15.52:0/3093557204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2dd015c820 msgr2=0x7f2dd00cd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:43.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.890+0000 7f2dd5e00640 1 --2- 172.21.15.52:0/3093557204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2dd015c820 0x7f2dd00cd830 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4009870 tx=0x7f2dc402b980 comp rx=0 tx=0).stop 2024-03-12T17:11:43.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.890+0000 7f2dd5e00640 1 -- 172.21.15.52:0/3093557204 shutdown_connections 2024-03-12T17:11:43.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.890+0000 7f2dd5e00640 1 --2- 172.21.15.52:0/3093557204 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2dd0160e90 0x7f2dd00cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:43.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.890+0000 7f2dd5e00640 1 --2- 172.21.15.52:0/3093557204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2dd015c820 0x7f2dd00cd830 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:43.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.890+0000 7f2dd5e00640 1 -- 172.21.15.52:0/3093557204 >> 172.21.15.52:0/3093557204 conn(0x7f2dd006b050 msgr2=0x7f2dd006b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:43.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.890+0000 7f2dd5e00640 1 -- 172.21.15.52:0/3093557204 shutdown_connections 2024-03-12T17:11:43.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.890+0000 7f2dd5e00640 1 -- 172.21.15.52:0/3093557204 wait complete. 2024-03-12T17:11:43.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.893+0000 7f2dd5e00640 1 Processor -- start 2024-03-12T17:11:43.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.893+0000 7f2dd5e00640 1 -- start start 2024-03-12T17:11:43.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.894+0000 7f2dd5e00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2dd00cf5f0 0x7f2dd00b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:43.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.894+0000 7f2dd5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2dd015c820 0x7f2dd0155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:43.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.894+0000 7f2dd5e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2dd0160e90 0x7f2dd01554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:43.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.894+0000 7f2dd5e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd0156320 con 0x7f2dd0160e90 2024-03-12T17:11:43.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.894+0000 7f2dd5e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd00b31a0 con 0x7f2dd015c820 2024-03-12T17:11:43.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.894+0000 7f2dd5e00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd00b2100 con 0x7f2dd00cf5f0 2024-03-12T17:11:43.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.895+0000 7f2dd5400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2dd0160e90 0x7f2dd01554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:43.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.895+0000 7f2dd5400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2dd0160e90 0x7f2dd01554d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45692/0 (socket says 172.21.15.52:45692) 2024-03-12T17:11:43.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.895+0000 7f2dd5400640 1 -- 172.21.15.52:0/3367144582 learned_addr learned my addr 172.21.15.52:0/3367144582 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:43.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.895+0000 7f2dd4a00640 1 --2- 172.21.15.52:0/3367144582 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2dd00cf5f0 0x7f2dd00b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:43.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.895+0000 7f2dcfe00640 1 --2- 172.21.15.52:0/3367144582 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2dd015c820 0x7f2dd0155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:43.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.895+0000 7f2dd5400640 1 -- 172.21.15.52:0/3367144582 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2dd00cf5f0 msgr2=0x7f2dd00b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:43.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.895+0000 7f2dd5400640 1 --2- 172.21.15.52:0/3367144582 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2dd00cf5f0 0x7f2dd00b1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:43.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.895+0000 7f2dd5400640 1 -- 172.21.15.52:0/3367144582 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2dd015c820 msgr2=0x7f2dd0155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:43.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.895+0000 7f2dd5400640 1 --2- 172.21.15.52:0/3367144582 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2dd015c820 0x7f2dd0155de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:43.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.895+0000 7f2dd5400640 1 -- 172.21.15.52:0/3367144582 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc40094d0 con 0x7f2dd0160e90 2024-03-12T17:11:43.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.895+0000 7f2dd5400640 1 --2- 172.21.15.52:0/3367144582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2dd0160e90 0x7f2dd01554d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f2dc800c780 tx=0x7f2dc800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:43.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.896+0000 7f2dccc00640 1 -- 172.21.15.52:0/3367144582 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8003d80 con 0x7f2dd0160e90 2024-03-12T17:11:43.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.896+0000 7f2dccc00640 1 -- 172.21.15.52:0/3367144582 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc800ce90 con 0x7f2dd0160e90 2024-03-12T17:11:43.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.896+0000 7f2dd5e00640 1 -- 172.21.15.52:0/3367144582 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dd00b23e0 con 0x7f2dd0160e90 2024-03-12T17:11:43.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.896+0000 7f2dccc00640 1 -- 172.21.15.52:0/3367144582 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc80052b0 con 0x7f2dd0160e90 2024-03-12T17:11:43.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.896+0000 7f2dd5e00640 1 -- 172.21.15.52:0/3367144582 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd00b29a0 con 0x7f2dd0160e90 2024-03-12T17:11:43.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.897+0000 7f2dd5e00640 1 -- 172.21.15.52:0/3367144582 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2dd00cd310 con 0x7f2dd0160e90 2024-03-12T17:11:43.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.898+0000 7f2dccc00640 1 -- 172.21.15.52:0/3367144582 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2dc8005410 con 0x7f2dd0160e90 2024-03-12T17:11:43.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.899+0000 7f2dccc00640 1 --2- 172.21.15.52:0/3367144582 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2db006ac50 0x7f2db006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:43.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.900+0000 7f2dccc00640 1 -- 172.21.15.52:0/3367144582 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f2dc8088a70 con 0x7f2dd0160e90 2024-03-12T17:11:43.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.900+0000 7f2dccc00640 1 --2- 172.21.15.52:0/3367144582 >> [v2:172.21.15.52:6816/37217,v1:172.21.15.52:6818/37217] conn(0x7f2db0070f50 0x7f2db0073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:43.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.900+0000 7f2dccc00640 1 -- 172.21.15.52:0/3367144582 --> [v2:172.21.15.52:6816/37217,v1:172.21.15.52:6818/37217] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2db0073a60 con 0x7f2db0070f50 2024-03-12T17:11:43.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.900+0000 7f2dccc00640 1 -- 172.21.15.52:0/3367144582 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2dc808d2a0 con 0x7f2dd0160e90 2024-03-12T17:11:43.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.900+0000 7f2dd4a00640 1 --2- 172.21.15.52:0/3367144582 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2db006ac50 0x7f2db006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:43.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.900+0000 7f2dcfe00640 1 --2- 172.21.15.52:0/3367144582 >> [v2:172.21.15.52:6816/37217,v1:172.21.15.52:6818/37217] conn(0x7f2db0070f50 0x7f2db0073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:43.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.901+0000 7f2dcfe00640 1 --2- 172.21.15.52:0/3367144582 >> [v2:172.21.15.52:6816/37217,v1:172.21.15.52:6818/37217] conn(0x7f2db0070f50 0x7f2db0073390 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:43.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.901+0000 7f2dd4a00640 1 --2- 172.21.15.52:0/3367144582 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2db006ac50 0x7f2db006d110 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4036000 tx=0x7f2dc4008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:43.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.902+0000 7f2dccc00640 1 -- 172.21.15.52:0/3367144582 <== osd.3 v2:172.21.15.52:6816/37217 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f2db0073a60 con 0x7f2db0070f50 2024-03-12T17:11:43.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.916+0000 7f2dd5e00640 1 -- 172.21.15.52:0/3367144582 --> [v2:172.21.15.52:6816/37217,v1:172.21.15.52:6818/37217] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2dd00cd030 con 0x7f2db0070f50 2024-03-12T17:11:43.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.916+0000 7f2dccc00640 1 -- 172.21.15.52:0/3367144582 <== osd.3 v2:172.21.15.52:6816/37217 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f2dbc00b940 con 0x7f2db0070f50 2024-03-12T17:11:43.909 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-03-12T17:11:43.910 INFO:teuthology.orchestra.run.smithi052.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-12T17:11:43.910 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-03-12T17:11:43.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.916+0000 7f2dd5e00640 1 -- 172.21.15.52:0/3367144582 >> [v2:172.21.15.52:6816/37217,v1:172.21.15.52:6818/37217] conn(0x7f2db0070f50 msgr2=0x7f2db0073390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:43.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.916+0000 7f2dd5e00640 1 --2- 172.21.15.52:0/3367144582 >> [v2:172.21.15.52:6816/37217,v1:172.21.15.52:6818/37217] conn(0x7f2db0070f50 0x7f2db0073390 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:43.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.916+0000 7f2dd5e00640 1 -- 172.21.15.52:0/3367144582 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2db006ac50 msgr2=0x7f2db006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:43.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.916+0000 7f2dd5e00640 1 --2- 172.21.15.52:0/3367144582 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2db006ac50 0x7f2db006d110 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4036000 tx=0x7f2dc4008030 comp rx=0 tx=0).stop 2024-03-12T17:11:43.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.916+0000 7f2dd5e00640 1 -- 172.21.15.52:0/3367144582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2dd0160e90 msgr2=0x7f2dd01554d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:43.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.916+0000 7f2dd5e00640 1 --2- 172.21.15.52:0/3367144582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2dd0160e90 0x7f2dd01554d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f2dc800c780 tx=0x7f2dc800cc40 comp rx=0 tx=0).stop 2024-03-12T17:11:43.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.916+0000 7f2dd5400640 1 -- 172.21.15.52:0/3367144582 reap_dead start 2024-03-12T17:11:43.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.917+0000 7f2dd5e00640 1 -- 172.21.15.52:0/3367144582 shutdown_connections 2024-03-12T17:11:43.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.917+0000 7f2dd5e00640 1 -- 172.21.15.52:0/3367144582 >> 172.21.15.52:0/3367144582 conn(0x7f2dd006b050 msgr2=0x7f2dd00b5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:43.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.917+0000 7f2dd5e00640 1 -- 172.21.15.52:0/3367144582 shutdown_connections 2024-03-12T17:11:43.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:43.917+0000 7f2dd5e00640 1 -- 172.21.15.52:0/3367144582 wait complete. 2024-03-12T17:11:43.928 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-12T17:11:44.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.152+0000 7fd9b4a00640 1 Processor -- start 2024-03-12T17:11:44.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.152+0000 7fd9b4a00640 1 -- start start 2024-03-12T17:11:44.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.154+0000 7fd9b4a00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd9b00cbab0 0x7fd9b00b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:44.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.154+0000 7fd9b4a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9b006d390 0x7fd9b006d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:44.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.154+0000 7fd9b4a00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9b00b16a0 con 0x7fd9b00adbc0 2024-03-12T17:11:44.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.154+0000 7fd9b4a00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9b006dd60 con 0x7fd9b00cbab0 2024-03-12T17:11:44.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.154+0000 7fd9b4a00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b00cbfb0 con 0x7fd9b006d390 2024-03-12T17:11:44.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.154+0000 7fd9ad400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd9b00cbab0 0x7fd9b00b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:44.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.154+0000 7fd9ade00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fd9b00adbc0 0x7fd9b00b41c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51962/0 (socket says 172.21.15.52:51962) 2024-03-12T17:11:44.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.154+0000 7fd9ade00640 1 -- 172.21.15.52:0/678334404 learned_addr learned my addr 172.21.15.52:0/678334404 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:11:44.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.154+0000 7fd9ae800640 1 --2- 172.21.15.52:0/678334404 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9b006d390 0x7fd9b006d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:44.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.155+0000 7fd9ad400640 1 -- 172.21.15.52:0/678334404 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9b006d390 msgr2=0x7fd9b006d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:44.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.155+0000 7fd9ad400640 1 --2- 172.21.15.52:0/678334404 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9b006d390 0x7fd9b006d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:44.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.155+0000 7fd9ad400640 1 -- 172.21.15.52:0/678334404 >> v1:172.21.15.52:6789/0 conn(0x7fd9b00adbc0 legacy=0x7fd9b00b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:44.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.155+0000 7fd9ad400640 1 -- 172.21.15.52:0/678334404 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b00cc290 con 0x7fd9b00cbab0 2024-03-12T17:11:44.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.155+0000 7fd9aca00640 1 -- 172.21.15.52:0/678334404 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1110350791 0 0) 0x7fd9b00b16a0 con 0x7fd9b00adbc0 2024-03-12T17:11:44.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.156+0000 7fd9ad400640 1 --2- 172.21.15.52:0/678334404 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9b00cbab0 0x7fd9b00b20f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0097a0 tx=0x7fd99c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d455e29ed3ce4941 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:44.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.156+0000 7fd9aca00640 1 -- 172.21.15.52:0/678334404 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c031780 con 0x7fd9b00cbab0 2024-03-12T17:11:44.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.156+0000 7fd9aca00640 1 -- 172.21.15.52:0/678334404 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd99c0318e0 con 0x7fd9b00cbab0 2024-03-12T17:11:44.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.156+0000 7fd9aca00640 1 -- 172.21.15.52:0/678334404 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c031c70 con 0x7fd9b00cbab0 2024-03-12T17:11:44.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.157+0000 7fd9b4a00640 1 -- 172.21.15.52:0/678334404 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9b00cbab0 msgr2=0x7fd9b00b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:44.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.157+0000 7fd9b4a00640 1 --2- 172.21.15.52:0/678334404 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9b00cbab0 0x7fd9b00b20f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0097a0 tx=0x7fd99c02b820 comp rx=0 tx=0).stop 2024-03-12T17:11:44.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.157+0000 7fd9b4a00640 1 -- 172.21.15.52:0/678334404 shutdown_connections 2024-03-12T17:11:44.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.157+0000 7fd9b4a00640 1 --2- 172.21.15.52:0/678334404 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9b00cbab0 0x7fd9b00b20f0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:44.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.157+0000 7fd9b4a00640 1 --2- 172.21.15.52:0/678334404 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9b006d390 0x7fd9b006d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:44.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.157+0000 7fd9b4a00640 1 -- 172.21.15.52:0/678334404 >> 172.21.15.52:0/678334404 conn(0x7fd9b006a3d0 msgr2=0x7fd9b00b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:44.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.157+0000 7fd9b4a00640 1 -- 172.21.15.52:0/678334404 shutdown_connections 2024-03-12T17:11:44.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.157+0000 7fd9b4a00640 1 -- 172.21.15.52:0/678334404 wait complete. 2024-03-12T17:11:44.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.159+0000 7fd9b4a00640 1 Processor -- start 2024-03-12T17:11:44.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.159+0000 7fd9b4a00640 1 -- start start 2024-03-12T17:11:44.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.160+0000 7fd9b4a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9b006d390 0x7fd9b00acc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:44.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.160+0000 7fd9b4a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9b00adbc0 0x7fd9b00ad170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:44.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.160+0000 7fd9b4a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9b00cbab0 0x7fd9b01ef380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:44.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.160+0000 7fd9b4a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b013f000 con 0x7fd9b006d390 2024-03-12T17:11:44.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.160+0000 7fd9b4a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b013f2e0 con 0x7fd9b00cbab0 2024-03-12T17:11:44.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.160+0000 7fd9b4a00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b013f5c0 con 0x7fd9b00adbc0 2024-03-12T17:11:44.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.160+0000 7fd9ade00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9b006d390 0x7fd9b00acc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:44.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.160+0000 7fd9ade00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9b006d390 0x7fd9b00acc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45716/0 (socket says 172.21.15.52:45716) 2024-03-12T17:11:44.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.160+0000 7fd9ade00640 1 -- 172.21.15.52:0/2913628302 learned_addr learned my addr 172.21.15.52:0/2913628302 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:44.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.161+0000 7fd9ad400640 1 --2- 172.21.15.52:0/2913628302 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9b00adbc0 0x7fd9b00ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:44.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.161+0000 7fd9ade00640 1 -- 172.21.15.52:0/2913628302 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9b00adbc0 msgr2=0x7fd9b00ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:44.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.161+0000 7fd9ade00640 1 --2- 172.21.15.52:0/2913628302 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9b00adbc0 0x7fd9b00ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:44.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.161+0000 7fd9ade00640 1 -- 172.21.15.52:0/2913628302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9b00cbab0 msgr2=0x7fd9b01ef380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:44.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.161+0000 7fd9ae800640 1 --2- 172.21.15.52:0/2913628302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9b00cbab0 0x7fd9b01ef380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:44.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.161+0000 7fd9ade00640 1 --2- 172.21.15.52:0/2913628302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9b00cbab0 0x7fd9b01ef380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:44.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.161+0000 7fd9ade00640 1 -- 172.21.15.52:0/2913628302 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99c009400 con 0x7fd9b006d390 2024-03-12T17:11:44.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.161+0000 7fd9ad400640 1 --2- 172.21.15.52:0/2913628302 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9b00adbc0 0x7fd9b00ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:11:44.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.161+0000 7fd9ade00640 1 --2- 172.21.15.52:0/2913628302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9b006d390 0x7fd9b00acc30 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd9a000c4d0 tx=0x7fd9a000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:44.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.161+0000 7fd9a6000640 1 -- 172.21.15.52:0/2913628302 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a00074e0 con 0x7fd9b006d390 2024-03-12T17:11:44.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.161+0000 7fd9a6000640 1 -- 172.21.15.52:0/2913628302 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a0007660 con 0x7fd9b006d390 2024-03-12T17:11:44.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.162+0000 7fd9b4a00640 1 -- 172.21.15.52:0/2913628302 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b013f8a0 con 0x7fd9b006d390 2024-03-12T17:11:44.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.162+0000 7fd9a6000640 1 -- 172.21.15.52:0/2913628302 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0004a20 con 0x7fd9b006d390 2024-03-12T17:11:44.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.162+0000 7fd9b4a00640 1 -- 172.21.15.52:0/2913628302 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b014f830 con 0x7fd9b006d390 2024-03-12T17:11:44.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.163+0000 7fd9b4a00640 1 -- 172.21.15.52:0/2913628302 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b006ac20 con 0x7fd9b006d390 2024-03-12T17:11:44.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.164+0000 7fd9a6000640 1 -- 172.21.15.52:0/2913628302 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd9a0004b80 con 0x7fd9b006d390 2024-03-12T17:11:44.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.165+0000 7fd9a6000640 1 --2- 172.21.15.52:0/2913628302 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd99006ac00 0x7fd99006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:44.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.165+0000 7fd9a6000640 1 -- 172.21.15.52:0/2913628302 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fd9a00886b0 con 0x7fd9b006d390 2024-03-12T17:11:44.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.166+0000 7fd9ad400640 1 --2- 172.21.15.52:0/2913628302 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd99006ac00 0x7fd99006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:44.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.167+0000 7fd9ad400640 1 --2- 172.21.15.52:0/2913628302 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd99006ac00 0x7fd99006d0c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd99c036000 tx=0x7fd99c03d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:44.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.170+0000 7fd9a6000640 1 -- 172.21.15.52:0/2913628302 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fd9a0053a40 con 0x7fd9b006d390 2024-03-12T17:11:44.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.366+0000 7fd9b4a00640 1 -- 172.21.15.52:0/2913628302 --> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9b013f450 con 0x7fd99006ac00 2024-03-12T17:11:44.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.368+0000 7fd9a6000640 1 -- 172.21.15.52:0/2913628302 <== mgr.4109 v2:172.21.15.102:6832/36622 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd9b013f450 con 0x7fd99006ac00 2024-03-12T17:11:44.361 INFO:teuthology.orchestra.run.smithi052.stderr:instructed osd(s) 3 to deep-scrub 2024-03-12T17:11:44.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.370+0000 7fd9b4a00640 1 -- 172.21.15.52:0/2913628302 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd99006ac00 msgr2=0x7fd99006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:44.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.370+0000 7fd9b4a00640 1 --2- 172.21.15.52:0/2913628302 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd99006ac00 0x7fd99006d0c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd99c036000 tx=0x7fd99c03d040 comp rx=0 tx=0).stop 2024-03-12T17:11:44.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.370+0000 7fd9b4a00640 1 -- 172.21.15.52:0/2913628302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9b006d390 msgr2=0x7fd9b00acc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:44.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.370+0000 7fd9b4a00640 1 --2- 172.21.15.52:0/2913628302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9b006d390 0x7fd9b00acc30 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd9a000c4d0 tx=0x7fd9a000c990 comp rx=0 tx=0).stop 2024-03-12T17:11:44.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.370+0000 7fd9b4a00640 1 -- 172.21.15.52:0/2913628302 shutdown_connections 2024-03-12T17:11:44.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.370+0000 7fd9b4a00640 1 --2- 172.21.15.52:0/2913628302 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd99006ac00 0x7fd99006d0c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:44.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.370+0000 7fd9b4a00640 1 --2- 172.21.15.52:0/2913628302 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9b00adbc0 0x7fd9b00ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:44.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.370+0000 7fd9b4a00640 1 --2- 172.21.15.52:0/2913628302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9b00cbab0 0x7fd9b01ef380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:44.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.370+0000 7fd9b4a00640 1 --2- 172.21.15.52:0/2913628302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9b006d390 0x7fd9b00acc30 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:44.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.370+0000 7fd9b4a00640 1 -- 172.21.15.52:0/2913628302 >> 172.21.15.52:0/2913628302 conn(0x7fd9b006a3d0 msgr2=0x7fd9b00b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:44.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.370+0000 7fd9b4a00640 1 -- 172.21.15.52:0/2913628302 shutdown_connections 2024-03-12T17:11:44.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.370+0000 7fd9b4a00640 1 -- 172.21.15.52:0/2913628302 wait complete. 2024-03-12T17:11:44.384 INFO:tasks.ceph:Scrubbing osd.4 2024-03-12T17:11:44.384 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-03-12T17:11:44.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.617+0000 7f1b67c00640 1 Processor -- start 2024-03-12T17:11:44.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.617+0000 7f1b67c00640 1 -- start start 2024-03-12T17:11:44.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.619+0000 7f1b67c00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1b60099150 0x7f1b6006ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:44.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.619+0000 7f1b67c00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b600cbb90 con 0x7f1b600995e0 2024-03-12T17:11:44.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.619+0000 7f1b67c00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b600cb2a0 con 0x7f1b60160170 2024-03-12T17:11:44.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.619+0000 7f1b67c00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b600cf4a0 con 0x7f1b60099150 2024-03-12T17:11:44.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.619+0000 7f1b5fe00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f1b600995e0 0x7f1b60098a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51972/0 (socket says 172.21.15.52:51972) 2024-03-12T17:11:44.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.619+0000 7f1b5fe00640 1 -- 172.21.15.52:0/376509456 learned_addr learned my addr 172.21.15.52:0/376509456 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:11:44.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.619+0000 7f1b65c00640 1 --2- 172.21.15.52:0/376509456 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1b60099150 0x7f1b6006ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:44.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.620+0000 7f1b5f400640 1 -- 172.21.15.52:0/376509456 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1086559282 0 0) 0x7f1b600cbb90 con 0x7f1b600995e0 2024-03-12T17:11:44.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.620+0000 7f1b5f400640 1 -- 172.21.15.52:0/376509456 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b44003520 con 0x7f1b600995e0 2024-03-12T17:11:44.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.620+0000 7f1b5f400640 1 -- 172.21.15.52:0/376509456 <== mon.1 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1291892602 0 0) 0x7f1b600cb2a0 con 0x7f1b60160170 2024-03-12T17:11:44.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.620+0000 7f1b5f400640 1 -- 172.21.15.52:0/376509456 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b600cbb90 con 0x7f1b60160170 2024-03-12T17:11:44.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.620+0000 7f1b5f400640 1 -- 172.21.15.52:0/376509456 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2504932516 0 0) 0x7f1b44003520 con 0x7f1b600995e0 2024-03-12T17:11:44.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.620+0000 7f1b65c00640 1 -- 172.21.15.52:0/376509456 >> v1:172.21.15.102:6789/0 conn(0x7f1b60160170 legacy=0x7f1b600cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:44.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.620+0000 7f1b65c00640 1 -- 172.21.15.52:0/376509456 >> v1:172.21.15.52:6789/0 conn(0x7f1b600995e0 legacy=0x7f1b60098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:44.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.620+0000 7f1b65c00640 1 -- 172.21.15.52:0/376509456 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b600cb540 con 0x7f1b60099150 2024-03-12T17:11:44.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.622+0000 7f1b65c00640 1 --2- 172.21.15.52:0/376509456 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1b60099150 0x7f1b6006ada0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1b58009dc0 tx=0x7f1b5802bcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5602da625294e6cc server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:44.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.622+0000 7f1b5f400640 1 -- 172.21.15.52:0/376509456 <== mon.2 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b48001670 con 0x7f1b60099150 2024-03-12T17:11:44.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.622+0000 7f1b5f400640 1 -- 172.21.15.52:0/376509456 <== mon.2 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b5802c7f0 con 0x7f1b60099150 2024-03-12T17:11:44.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.622+0000 7f1b5f400640 1 -- 172.21.15.52:0/376509456 <== mon.2 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5802cb80 con 0x7f1b60099150 2024-03-12T17:11:44.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.623+0000 7f1b67c00640 1 -- 172.21.15.52:0/376509456 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1b60099150 msgr2=0x7f1b6006ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:44.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.623+0000 7f1b67c00640 1 --2- 172.21.15.52:0/376509456 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1b60099150 0x7f1b6006ada0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1b58009dc0 tx=0x7f1b5802bcc0 comp rx=0 tx=0).stop 2024-03-12T17:11:44.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.623+0000 7f1b67c00640 1 -- 172.21.15.52:0/376509456 shutdown_connections 2024-03-12T17:11:44.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.623+0000 7f1b67c00640 1 --2- 172.21.15.52:0/376509456 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1b60099150 0x7f1b6006ada0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:44.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.623+0000 7f1b67c00640 1 -- 172.21.15.52:0/376509456 >> 172.21.15.52:0/376509456 conn(0x7f1b600b2180 msgr2=0x7f1b600b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:44.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.623+0000 7f1b67c00640 1 -- 172.21.15.52:0/376509456 shutdown_connections 2024-03-12T17:11:44.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.623+0000 7f1b67c00640 1 -- 172.21.15.52:0/376509456 wait complete. 2024-03-12T17:11:44.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.625+0000 7f1b67c00640 1 Processor -- start 2024-03-12T17:11:44.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.625+0000 7f1b67c00640 1 -- start start 2024-03-12T17:11:44.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.626+0000 7f1b67c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1b60099150 0x7f1b6015d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:44.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.626+0000 7f1b67c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b600995e0 0x7f1b600c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:44.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.626+0000 7f1b67c00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1b60160170 0x7f1b601f1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:44.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.626+0000 7f1b67c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b600c9840 con 0x7f1b60099150 2024-03-12T17:11:44.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.626+0000 7f1b67c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b600c8c30 con 0x7f1b600995e0 2024-03-12T17:11:44.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.626+0000 7f1b67c00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b600c8f10 con 0x7f1b60160170 2024-03-12T17:11:44.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.626+0000 7f1b65200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1b60099150 0x7f1b6015d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:44.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.626+0000 7f1b65200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1b60099150 0x7f1b6015d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45728/0 (socket says 172.21.15.52:45728) 2024-03-12T17:11:44.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.626+0000 7f1b65200640 1 -- 172.21.15.52:0/2741745544 learned_addr learned my addr 172.21.15.52:0/2741745544 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:44.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.627+0000 7f1b65200640 1 -- 172.21.15.52:0/2741745544 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1b60160170 msgr2=0x7f1b601f1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:44.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.627+0000 7f1b65200640 1 --2- 172.21.15.52:0/2741745544 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1b60160170 0x7f1b601f1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:44.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.627+0000 7f1b65200640 1 -- 172.21.15.52:0/2741745544 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b600995e0 msgr2=0x7f1b600c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:44.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.627+0000 7f1b65200640 1 --2- 172.21.15.52:0/2741745544 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b600995e0 0x7f1b600c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:44.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.627+0000 7f1b65200640 1 -- 172.21.15.52:0/2741745544 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b58009a70 con 0x7f1b60099150 2024-03-12T17:11:44.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.627+0000 7f1b65200640 1 --2- 172.21.15.52:0/2741745544 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1b60099150 0x7f1b6015d230 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000c430 tx=0x7f1b5000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:44.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.627+0000 7f1b5cc00640 1 -- 172.21.15.52:0/2741745544 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b500052d0 con 0x7f1b60099150 2024-03-12T17:11:44.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.627+0000 7f1b5cc00640 1 -- 172.21.15.52:0/2741745544 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b50005430 con 0x7f1b60099150 2024-03-12T17:11:44.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.627+0000 7f1b5cc00640 1 -- 172.21.15.52:0/2741745544 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b50011890 con 0x7f1b60099150 2024-03-12T17:11:44.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.627+0000 7f1b67c00640 1 -- 172.21.15.52:0/2741745544 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b600c91f0 con 0x7f1b60099150 2024-03-12T17:11:44.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.627+0000 7f1b67c00640 1 -- 172.21.15.52:0/2741745544 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b6015f380 con 0x7f1b60099150 2024-03-12T17:11:44.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.628+0000 7f1b67c00640 1 -- 172.21.15.52:0/2741745544 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1b60160550 con 0x7f1b60099150 2024-03-12T17:11:44.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.629+0000 7f1b5cc00640 1 -- 172.21.15.52:0/2741745544 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1b500119f0 con 0x7f1b60099150 2024-03-12T17:11:44.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.630+0000 7f1b5cc00640 1 --2- 172.21.15.52:0/2741745544 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1b3c06ac00 0x7f1b3c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:44.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.631+0000 7f1b5cc00640 1 -- 172.21.15.52:0/2741745544 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f1b50088830 con 0x7f1b60099150 2024-03-12T17:11:44.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.631+0000 7f1b5cc00640 1 --2- 172.21.15.52:0/2741745544 >> [v2:172.21.15.102:6800/37191,v1:172.21.15.102:6801/37191] conn(0x7f1b3c070f00 0x7f1b3c073340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:44.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.631+0000 7f1b5cc00640 1 -- 172.21.15.52:0/2741745544 --> [v2:172.21.15.102:6800/37191,v1:172.21.15.102:6801/37191] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b3c073a10 con 0x7f1b3c070f00 2024-03-12T17:11:44.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.631+0000 7f1b5cc00640 1 -- 172.21.15.52:0/2741745544 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b5008ae10 con 0x7f1b60099150 2024-03-12T17:11:44.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.631+0000 7f1b5fe00640 1 --2- 172.21.15.52:0/2741745544 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1b3c06ac00 0x7f1b3c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:44.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.631+0000 7f1b65c00640 1 --2- 172.21.15.52:0/2741745544 >> [v2:172.21.15.102:6800/37191,v1:172.21.15.102:6801/37191] conn(0x7f1b3c070f00 0x7f1b3c073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:44.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.632+0000 7f1b5fe00640 1 --2- 172.21.15.52:0/2741745544 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1b3c06ac00 0x7f1b3c06d0c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1b600c7710 tx=0x7f1b480010a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:44.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.633+0000 7f1b65c00640 1 --2- 172.21.15.52:0/2741745544 >> [v2:172.21.15.102:6800/37191,v1:172.21.15.102:6801/37191] conn(0x7f1b3c070f00 0x7f1b3c073340 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:44.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.635+0000 7f1b5cc00640 1 -- 172.21.15.52:0/2741745544 <== osd.4 v2:172.21.15.102:6800/37191 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f1b3c073a10 con 0x7f1b3c070f00 2024-03-12T17:11:44.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.649+0000 7f1b67c00640 1 -- 172.21.15.52:0/2741745544 --> [v2:172.21.15.102:6800/37191,v1:172.21.15.102:6801/37191] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1b600c9980 con 0x7f1b3c070f00 2024-03-12T17:11:44.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.650+0000 7f1b5cc00640 1 -- 172.21.15.52:0/2741745544 <== osd.4 v2:172.21.15.102:6800/37191 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f1b5802cce0 con 0x7f1b3c070f00 2024-03-12T17:11:44.643 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-03-12T17:11:44.643 INFO:teuthology.orchestra.run.smithi052.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-12T17:11:44.643 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-03-12T17:11:44.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.650+0000 7f1b67c00640 1 -- 172.21.15.52:0/2741745544 >> [v2:172.21.15.102:6800/37191,v1:172.21.15.102:6801/37191] conn(0x7f1b3c070f00 msgr2=0x7f1b3c073340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:44.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.650+0000 7f1b67c00640 1 --2- 172.21.15.52:0/2741745544 >> [v2:172.21.15.102:6800/37191,v1:172.21.15.102:6801/37191] conn(0x7f1b3c070f00 0x7f1b3c073340 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:44.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.650+0000 7f1b67c00640 1 -- 172.21.15.52:0/2741745544 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1b3c06ac00 msgr2=0x7f1b3c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:44.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.650+0000 7f1b67c00640 1 --2- 172.21.15.52:0/2741745544 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1b3c06ac00 0x7f1b3c06d0c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1b600c7710 tx=0x7f1b480010a0 comp rx=0 tx=0).stop 2024-03-12T17:11:44.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.651+0000 7f1b67c00640 1 -- 172.21.15.52:0/2741745544 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1b60099150 msgr2=0x7f1b6015d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:44.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.651+0000 7f1b67c00640 1 --2- 172.21.15.52:0/2741745544 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1b60099150 0x7f1b6015d230 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000c430 tx=0x7f1b5000c8f0 comp rx=0 tx=0).stop 2024-03-12T17:11:44.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.651+0000 7f1b65c00640 1 -- 172.21.15.52:0/2741745544 reap_dead start 2024-03-12T17:11:44.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.651+0000 7f1b67c00640 1 -- 172.21.15.52:0/2741745544 shutdown_connections 2024-03-12T17:11:44.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.651+0000 7f1b67c00640 1 -- 172.21.15.52:0/2741745544 >> 172.21.15.52:0/2741745544 conn(0x7f1b600b2180 msgr2=0x7f1b6006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:44.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.651+0000 7f1b67c00640 1 -- 172.21.15.52:0/2741745544 shutdown_connections 2024-03-12T17:11:44.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.651+0000 7f1b67c00640 1 -- 172.21.15.52:0/2741745544 wait complete. 2024-03-12T17:11:44.666 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-03-12T17:11:44.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.901+0000 7f24aa000640 1 Processor -- start 2024-03-12T17:11:44.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.901+0000 7f24aa000640 1 -- start start 2024-03-12T17:11:44.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.903+0000 7f24aa000640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f24a4160d30 0x7f24a41679b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:44.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.903+0000 7f24aa000640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24a40b2760 con 0x7f24a40abd80 2024-03-12T17:11:44.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.903+0000 7f24aa000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24a40b29b0 con 0x7f24a415c990 2024-03-12T17:11:44.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.903+0000 7f24aa000640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24a40b2c90 con 0x7f24a4160d30 2024-03-12T17:11:44.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.903+0000 7f24a3e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f24a4160d30 0x7f24a41679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:44.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.903+0000 7f24a2a00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f24a40abd80 0x7f24a4160620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:58588/0 (socket says 172.21.15.52:58588) 2024-03-12T17:11:44.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.903+0000 7f24a2a00640 1 -- 172.21.15.52:0/3673443112 learned_addr learned my addr 172.21.15.52:0/3673443112 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:11:44.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.904+0000 7f24a3e00640 1 -- 172.21.15.52:0/3673443112 >> v1:172.21.15.102:6789/0 conn(0x7f24a415c990 legacy=0x7f24a40b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:44.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.904+0000 7f24a3e00640 1 -- 172.21.15.52:0/3673443112 >> v1:172.21.15.52:6789/0 conn(0x7f24a40abd80 legacy=0x7f24a4160620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:44.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.904+0000 7f24a3e00640 1 -- 172.21.15.52:0/3673443112 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a4069c90 con 0x7f24a4160d30 2024-03-12T17:11:44.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.905+0000 7f24a3e00640 1 --2- 172.21.15.52:0/3673443112 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24a4160d30 0x7f24a41679b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f249c009d90 tx=0x7f249c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03fe11042a59450 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:44.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.905+0000 7f24a2000640 1 -- 172.21.15.52:0/3673443112 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249c02cc60 con 0x7f24a4160d30 2024-03-12T17:11:44.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.905+0000 7f24a2000640 1 -- 172.21.15.52:0/3673443112 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f249c02cdc0 con 0x7f24a4160d30 2024-03-12T17:11:44.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.905+0000 7f24a2000640 1 -- 172.21.15.52:0/3673443112 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249c034660 con 0x7f24a4160d30 2024-03-12T17:11:44.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.906+0000 7f24aa000640 1 -- 172.21.15.52:0/3673443112 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24a4160d30 msgr2=0x7f24a41679b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:44.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.906+0000 7f24aa000640 1 --2- 172.21.15.52:0/3673443112 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24a4160d30 0x7f24a41679b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f249c009d90 tx=0x7f249c02beb0 comp rx=0 tx=0).stop 2024-03-12T17:11:44.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.906+0000 7f24aa000640 1 -- 172.21.15.52:0/3673443112 shutdown_connections 2024-03-12T17:11:44.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.906+0000 7f24aa000640 1 --2- 172.21.15.52:0/3673443112 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24a4160d30 0x7f24a41679b0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:44.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.906+0000 7f24aa000640 1 -- 172.21.15.52:0/3673443112 >> 172.21.15.52:0/3673443112 conn(0x7f24a40b1270 msgr2=0x7f24a40ced50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:44.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.906+0000 7f24aa000640 1 -- 172.21.15.52:0/3673443112 shutdown_connections 2024-03-12T17:11:44.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.906+0000 7f24aa000640 1 -- 172.21.15.52:0/3673443112 wait complete. 2024-03-12T17:11:44.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.908+0000 7f24aa000640 1 Processor -- start 2024-03-12T17:11:44.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.908+0000 7f24aa000640 1 -- start start 2024-03-12T17:11:44.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.909+0000 7f24aa000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24a40abd80 0x7f24a4156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:44.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.909+0000 7f24aa000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24a415c990 0x7f24a4150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:44.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.909+0000 7f24aa000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f24a4160d30 0x7f24a4159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:44.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.909+0000 7f24aa000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a41f2a50 con 0x7f24a40abd80 2024-03-12T17:11:44.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.909+0000 7f24aa000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a415f850 con 0x7f24a415c990 2024-03-12T17:11:44.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.909+0000 7f24aa000640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a4156f40 con 0x7f24a4160d30 2024-03-12T17:11:44.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.909+0000 7f24a3400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24a40abd80 0x7f24a4156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:44.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.909+0000 7f24a3400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24a40abd80 0x7f24a4156a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45736/0 (socket says 172.21.15.52:45736) 2024-03-12T17:11:44.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.909+0000 7f24a3400640 1 -- 172.21.15.52:0/1447948585 learned_addr learned my addr 172.21.15.52:0/1447948585 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:44.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.910+0000 7f24a2a00640 1 --2- 172.21.15.52:0/1447948585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24a415c990 0x7f24a4150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:44.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.910+0000 7f24a3e00640 1 --2- 172.21.15.52:0/1447948585 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f24a4160d30 0x7f24a4159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:44.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.910+0000 7f24a3400640 1 -- 172.21.15.52:0/1447948585 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f24a4160d30 msgr2=0x7f24a4159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:44.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.910+0000 7f24a3400640 1 --2- 172.21.15.52:0/1447948585 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f24a4160d30 0x7f24a4159f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:44.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.910+0000 7f24a3400640 1 -- 172.21.15.52:0/1447948585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24a415c990 msgr2=0x7f24a4150200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:44.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.910+0000 7f24a3400640 1 --2- 172.21.15.52:0/1447948585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24a415c990 0x7f24a4150200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:44.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.910+0000 7f24a3400640 1 -- 172.21.15.52:0/1447948585 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f249c009a70 con 0x7f24a40abd80 2024-03-12T17:11:44.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.910+0000 7f24a3e00640 1 --2- 172.21.15.52:0/1447948585 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f24a4160d30 0x7f24a4159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:11:44.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.910+0000 7f24a3400640 1 --2- 172.21.15.52:0/1447948585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24a40abd80 0x7f24a4156a00 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f249400c410 tx=0x7f249400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:44.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.910+0000 7f249b400640 1 -- 172.21.15.52:0/1447948585 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24940035f0 con 0x7f24a40abd80 2024-03-12T17:11:44.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.911+0000 7f249b400640 1 -- 172.21.15.52:0/1447948585 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2494003750 con 0x7f24a40abd80 2024-03-12T17:11:44.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.911+0000 7f24aa000640 1 -- 172.21.15.52:0/1447948585 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a40cd660 con 0x7f24a40abd80 2024-03-12T17:11:44.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.911+0000 7f249b400640 1 -- 172.21.15.52:0/1447948585 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2494003b00 con 0x7f24a40abd80 2024-03-12T17:11:44.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.911+0000 7f24aa000640 1 -- 172.21.15.52:0/1447948585 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a40cdc20 con 0x7f24a40abd80 2024-03-12T17:11:44.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.912+0000 7f24aa000640 1 -- 172.21.15.52:0/1447948585 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24a4073390 con 0x7f24a40abd80 2024-03-12T17:11:44.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.913+0000 7f249b400640 1 -- 172.21.15.52:0/1447948585 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2494003c60 con 0x7f24a40abd80 2024-03-12T17:11:44.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.914+0000 7f249b400640 1 --2- 172.21.15.52:0/1447948585 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f248406ac00 0x7f248406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:44.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.914+0000 7f249b400640 1 -- 172.21.15.52:0/1447948585 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f2494088820 con 0x7f24a40abd80 2024-03-12T17:11:44.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.915+0000 7f24a2a00640 1 --2- 172.21.15.52:0/1447948585 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f248406ac00 0x7f248406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:44.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.916+0000 7f24a2a00640 1 --2- 172.21.15.52:0/1447948585 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f248406ac00 0x7f248406d0c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f24a40cb280 tx=0x7f2490009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:44.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:44.918+0000 7f249b400640 1 -- 172.21.15.52:0/1447948585 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f2494053bb0 con 0x7f24a40abd80 2024-03-12T17:11:45.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.114+0000 7f24aa000640 1 -- 172.21.15.52:0/1447948585 --> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f24a406a250 con 0x7f248406ac00 2024-03-12T17:11:45.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.115+0000 7f249b400640 1 -- 172.21.15.52:0/1447948585 <== mgr.4109 v2:172.21.15.102:6832/36622 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2490001350 con 0x7f248406ac00 2024-03-12T17:11:45.108 INFO:teuthology.orchestra.run.smithi052.stderr:instructed osd(s) 4 to deep-scrub 2024-03-12T17:11:45.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.118+0000 7f24aa000640 1 -- 172.21.15.52:0/1447948585 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f248406ac00 msgr2=0x7f248406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:45.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.118+0000 7f24aa000640 1 --2- 172.21.15.52:0/1447948585 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f248406ac00 0x7f248406d0c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f24a40cb280 tx=0x7f2490009040 comp rx=0 tx=0).stop 2024-03-12T17:11:45.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.118+0000 7f24aa000640 1 -- 172.21.15.52:0/1447948585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24a40abd80 msgr2=0x7f24a4156a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:45.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.118+0000 7f24aa000640 1 --2- 172.21.15.52:0/1447948585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24a40abd80 0x7f24a4156a00 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f249400c410 tx=0x7f249400c8d0 comp rx=0 tx=0).stop 2024-03-12T17:11:45.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.118+0000 7f24aa000640 1 -- 172.21.15.52:0/1447948585 shutdown_connections 2024-03-12T17:11:45.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.118+0000 7f24aa000640 1 --2- 172.21.15.52:0/1447948585 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f248406ac00 0x7f248406d0c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:45.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.118+0000 7f24aa000640 1 --2- 172.21.15.52:0/1447948585 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f24a4160d30 0x7f24a4159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:45.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.118+0000 7f24aa000640 1 --2- 172.21.15.52:0/1447948585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24a415c990 0x7f24a4150200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:45.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.118+0000 7f24aa000640 1 --2- 172.21.15.52:0/1447948585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24a40abd80 0x7f24a4156a00 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:45.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.118+0000 7f24aa000640 1 -- 172.21.15.52:0/1447948585 >> 172.21.15.52:0/1447948585 conn(0x7f24a40b1270 msgr2=0x7f24a40aef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:45.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.118+0000 7f24aa000640 1 -- 172.21.15.52:0/1447948585 shutdown_connections 2024-03-12T17:11:45.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.118+0000 7f24aa000640 1 -- 172.21.15.52:0/1447948585 wait complete. 2024-03-12T17:11:45.130 INFO:tasks.ceph:Scrubbing osd.5 2024-03-12T17:11:45.130 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-03-12T17:11:45.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.366+0000 7fb75d200640 1 Processor -- start 2024-03-12T17:11:45.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.367+0000 7fb75d200640 1 -- start start 2024-03-12T17:11:45.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.368+0000 7fb75d200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb7580cf7c0 0x7fb7580cd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:45.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.368+0000 7fb75d200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb758160e90 0x7fb7580d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:45.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.368+0000 7fb75d200640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7580ccd90 con 0x7fb75815c820 2024-03-12T17:11:45.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.368+0000 7fb75d200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7580cdd70 con 0x7fb7580cf7c0 2024-03-12T17:11:45.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.368+0000 7fb75d200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7580b7fc0 con 0x7fb758160e90 2024-03-12T17:11:45.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.368+0000 7fb757400640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fb75815c820 0x7fb7580cf0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:58594/0 (socket says 172.21.15.52:58594) 2024-03-12T17:11:45.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.369+0000 7fb757400640 1 -- 172.21.15.52:0/4207233966 learned_addr learned my addr 172.21.15.52:0/4207233966 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:11:45.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.369+0000 7fb756000640 1 -- 172.21.15.52:0/4207233966 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1933519370 0 0) 0x7fb7580ccd90 con 0x7fb75815c820 2024-03-12T17:11:45.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.369+0000 7fb757e00640 1 --2- 172.21.15.52:0/4207233966 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb758160e90 0x7fb7580d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:45.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.369+0000 7fb756a00640 1 --2- 172.21.15.52:0/4207233966 >> v2:172.21.15.102:3300/0 conn(0x7fb7580cf7c0 0x7fb7580cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:45.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.369+0000 7fb756000640 1 -- 172.21.15.52:0/4207233966 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb73c003520 con 0x7fb75815c820 2024-03-12T17:11:45.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.369+0000 7fb756000640 1 -- 172.21.15.52:0/4207233966 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1836204493 0 0) 0x7fb73c003520 con 0x7fb75815c820 2024-03-12T17:11:45.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.370+0000 7fb756000640 1 -- 172.21.15.52:0/4207233966 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb758160e90 msgr2=0x7fb7580d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:45.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.370+0000 7fb756000640 1 --2- 172.21.15.52:0/4207233966 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb758160e90 0x7fb7580d0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:45.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.370+0000 7fb756000640 1 -- 172.21.15.52:0/4207233966 >> v2:172.21.15.102:3300/0 conn(0x7fb7580cf7c0 msgr2=0x7fb7580cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:45.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.370+0000 7fb756000640 1 --2- 172.21.15.52:0/4207233966 >> v2:172.21.15.102:3300/0 conn(0x7fb7580cf7c0 0x7fb7580cd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:45.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.370+0000 7fb756000640 1 -- 172.21.15.52:0/4207233966 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7580b82a0 con 0x7fb75815c820 2024-03-12T17:11:45.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.370+0000 7fb756000640 1 -- 172.21.15.52:0/4207233966 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 790246142 0 0) 0x7fb7480019f0 con 0x7fb75815c820 2024-03-12T17:11:45.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.370+0000 7fb756000640 1 -- 172.21.15.52:0/4207233966 >> v1:172.21.15.52:6789/0 conn(0x7fb75815c820 legacy=0x7fb7580cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:45.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.370+0000 7fb756000640 1 --2- 172.21.15.52:0/4207233966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb73c003970 0x7fb73c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:45.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.370+0000 7fb756000640 1 --2- 172.21.15.52:0/4207233966 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb758160e90 0x7fb73c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:45.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.370+0000 7fb756000640 1 --2- 172.21.15.52:0/4207233966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb73c004d00 0x7fb73c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:45.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.370+0000 7fb756000640 1 -- 172.21.15.52:0/4207233966 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7580cdd70 con 0x7fb73c004d00 2024-03-12T17:11:45.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.370+0000 7fb756000640 1 -- 172.21.15.52:0/4207233966 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7580b7fc0 con 0x7fb73c003970 2024-03-12T17:11:45.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.370+0000 7fb756000640 1 -- 172.21.15.52:0/4207233966 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c00b9b0 con 0x7fb758160e90 2024-03-12T17:11:45.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.370+0000 7fb757e00640 1 --2- 172.21.15.52:0/4207233966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb73c004d00 0x7fb73c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:45.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.370+0000 7fb756a00640 1 --2- 172.21.15.52:0/4207233966 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb758160e90 0x7fb73c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:45.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.370+0000 7fb757400640 1 --2- 172.21.15.52:0/4207233966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb73c003970 0x7fb73c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:45.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.370+0000 7fb757e00640 1 -- 172.21.15.52:0/4207233966 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb758160e90 msgr2=0x7fb73c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:45.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.370+0000 7fb757e00640 1 --2- 172.21.15.52:0/4207233966 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb758160e90 0x7fb73c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:45.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.370+0000 7fb757e00640 1 -- 172.21.15.52:0/4207233966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb73c003970 msgr2=0x7fb73c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:45.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.371+0000 7fb757e00640 1 --2- 172.21.15.52:0/4207233966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb73c003970 0x7fb73c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:45.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.371+0000 7fb757e00640 1 -- 172.21.15.52:0/4207233966 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb73c003520 con 0x7fb73c004d00 2024-03-12T17:11:45.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.371+0000 7fb756a00640 1 --2- 172.21.15.52:0/4207233966 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb758160e90 0x7fb73c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:11:45.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.371+0000 7fb757400640 1 --2- 172.21.15.52:0/4207233966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb73c003970 0x7fb73c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:11:45.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.372+0000 7fb757e00640 1 --2- 172.21.15.52:0/4207233966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb73c004d00 0x7fb73c005180 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb750009ac0 tx=0x7fb73c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324ee02d6784cc8e server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:45.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.372+0000 7fb756000640 1 -- 172.21.15.52:0/4207233966 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75002c6b0 con 0x7fb73c004d00 2024-03-12T17:11:45.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.372+0000 7fb756000640 1 -- 172.21.15.52:0/4207233966 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb75002c810 con 0x7fb73c004d00 2024-03-12T17:11:45.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.372+0000 7fb756000640 1 -- 172.21.15.52:0/4207233966 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75002cba0 con 0x7fb73c004d00 2024-03-12T17:11:45.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.372+0000 7fb75d200640 1 -- 172.21.15.52:0/4207233966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb73c004d00 msgr2=0x7fb73c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:45.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.372+0000 7fb75d200640 1 --2- 172.21.15.52:0/4207233966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb73c004d00 0x7fb73c005180 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb750009ac0 tx=0x7fb73c0057f0 comp rx=0 tx=0).stop 2024-03-12T17:11:45.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.373+0000 7fb757e00640 1 -- 172.21.15.52:0/4207233966 reap_dead start 2024-03-12T17:11:45.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.373+0000 7fb75d200640 1 -- 172.21.15.52:0/4207233966 shutdown_connections 2024-03-12T17:11:45.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.373+0000 7fb75d200640 1 -- 172.21.15.52:0/4207233966 >> 172.21.15.52:0/4207233966 conn(0x7fb75806b050 msgr2=0x7fb75806b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:45.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.373+0000 7fb75d200640 1 -- 172.21.15.52:0/4207233966 shutdown_connections 2024-03-12T17:11:45.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.373+0000 7fb75d200640 1 -- 172.21.15.52:0/4207233966 wait complete. 2024-03-12T17:11:45.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.374+0000 7fb75d200640 1 Processor -- start 2024-03-12T17:11:45.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.375+0000 7fb75d200640 1 -- start start 2024-03-12T17:11:45.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.375+0000 7fb75d200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7580cbf70 0x7fb75815fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:45.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.376+0000 7fb75d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb75815cc00 0x7fb7581f2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:45.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.376+0000 7fb75d200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb7580b67e0 0x7fb7580b6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:45.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.376+0000 7fb75d200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75815dff0 con 0x7fb7580cbf70 2024-03-12T17:11:45.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.376+0000 7fb75d200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7580b71a0 con 0x7fb75815cc00 2024-03-12T17:11:45.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.376+0000 7fb75d200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7580cc350 con 0x7fb7580b67e0 2024-03-12T17:11:45.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.376+0000 7fb757400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7580cbf70 0x7fb75815fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:45.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.376+0000 7fb757400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7580cbf70 0x7fb75815fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45744/0 (socket says 172.21.15.52:45744) 2024-03-12T17:11:45.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.376+0000 7fb757400640 1 -- 172.21.15.52:0/729354848 learned_addr learned my addr 172.21.15.52:0/729354848 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:45.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.376+0000 7fb756a00640 1 --2- 172.21.15.52:0/729354848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb75815cc00 0x7fb7581f2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:45.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.376+0000 7fb757e00640 1 --2- 172.21.15.52:0/729354848 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb7580b67e0 0x7fb7580b6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:45.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.376+0000 7fb757400640 1 -- 172.21.15.52:0/729354848 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb7580b67e0 msgr2=0x7fb7580b6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:45.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.376+0000 7fb757400640 1 --2- 172.21.15.52:0/729354848 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb7580b67e0 0x7fb7580b6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:45.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.376+0000 7fb757400640 1 -- 172.21.15.52:0/729354848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb75815cc00 msgr2=0x7fb7581f2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:45.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.376+0000 7fb757400640 1 --2- 172.21.15.52:0/729354848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb75815cc00 0x7fb7581f2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:45.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.376+0000 7fb757400640 1 -- 172.21.15.52:0/729354848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb75815d370 con 0x7fb7580cbf70 2024-03-12T17:11:45.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.377+0000 7fb757400640 1 --2- 172.21.15.52:0/729354848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7580cbf70 0x7fb75815fc80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb7480021f0 tx=0x7fb74800b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:45.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.377+0000 7fb74f400640 1 -- 172.21.15.52:0/729354848 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7480072d0 con 0x7fb7580cbf70 2024-03-12T17:11:45.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.377+0000 7fb74f400640 1 -- 172.21.15.52:0/729354848 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb748007430 con 0x7fb7580cbf70 2024-03-12T17:11:45.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.377+0000 7fb74f400640 1 -- 172.21.15.52:0/729354848 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7480077e0 con 0x7fb7580cbf70 2024-03-12T17:11:45.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.377+0000 7fb75d200640 1 -- 172.21.15.52:0/729354848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb75815d650 con 0x7fb7580cbf70 2024-03-12T17:11:45.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.377+0000 7fb75d200640 1 -- 172.21.15.52:0/729354848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb75815dc10 con 0x7fb7580cbf70 2024-03-12T17:11:45.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.378+0000 7fb75d200640 1 -- 172.21.15.52:0/729354848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb758161320 con 0x7fb7580cbf70 2024-03-12T17:11:45.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.379+0000 7fb74f400640 1 -- 172.21.15.52:0/729354848 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb748007940 con 0x7fb7580cbf70 2024-03-12T17:11:45.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.380+0000 7fb74f400640 1 --2- 172.21.15.52:0/729354848 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb73406ac30 0x7fb73406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:45.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.380+0000 7fb74f400640 1 -- 172.21.15.52:0/729354848 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fb748089740 con 0x7fb7580cbf70 2024-03-12T17:11:45.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.380+0000 7fb74f400640 1 --2- 172.21.15.52:0/729354848 >> [v2:172.21.15.102:6816/37201,v1:172.21.15.102:6817/37201] conn(0x7fb734070f30 0x7fb734073370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:45.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.380+0000 7fb74f400640 1 -- 172.21.15.52:0/729354848 --> [v2:172.21.15.102:6816/37201,v1:172.21.15.102:6817/37201] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb734073a40 con 0x7fb734070f30 2024-03-12T17:11:45.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.380+0000 7fb74f400640 1 -- 172.21.15.52:0/729354848 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb74808bd20 con 0x7fb7580cbf70 2024-03-12T17:11:45.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.381+0000 7fb756a00640 1 --2- 172.21.15.52:0/729354848 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb73406ac30 0x7fb73406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:45.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.381+0000 7fb757e00640 1 --2- 172.21.15.52:0/729354848 >> [v2:172.21.15.102:6816/37201,v1:172.21.15.102:6817/37201] conn(0x7fb734070f30 0x7fb734073370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:45.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.382+0000 7fb756a00640 1 --2- 172.21.15.52:0/729354848 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb73406ac30 0x7fb73406d0f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb7581f5f20 tx=0x7fb740008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:45.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.382+0000 7fb757e00640 1 --2- 172.21.15.52:0/729354848 >> [v2:172.21.15.102:6816/37201,v1:172.21.15.102:6817/37201] conn(0x7fb734070f30 0x7fb734073370 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:45.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.384+0000 7fb74f400640 1 -- 172.21.15.52:0/729354848 <== osd.5 v2:172.21.15.102:6816/37201 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fb734073a40 con 0x7fb734070f30 2024-03-12T17:11:45.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.398+0000 7fb75d200640 1 -- 172.21.15.52:0/729354848 --> [v2:172.21.15.102:6816/37201,v1:172.21.15.102:6817/37201] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb7580cd310 con 0x7fb734070f30 2024-03-12T17:11:45.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.399+0000 7fb74f400640 1 -- 172.21.15.52:0/729354848 <== osd.5 v2:172.21.15.102:6816/37201 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fb75000f080 con 0x7fb734070f30 2024-03-12T17:11:45.393 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-03-12T17:11:45.393 INFO:teuthology.orchestra.run.smithi052.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-12T17:11:45.393 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-03-12T17:11:45.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.399+0000 7fb75d200640 1 -- 172.21.15.52:0/729354848 >> [v2:172.21.15.102:6816/37201,v1:172.21.15.102:6817/37201] conn(0x7fb734070f30 msgr2=0x7fb734073370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:45.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.399+0000 7fb75d200640 1 --2- 172.21.15.52:0/729354848 >> [v2:172.21.15.102:6816/37201,v1:172.21.15.102:6817/37201] conn(0x7fb734070f30 0x7fb734073370 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:45.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.400+0000 7fb75d200640 1 -- 172.21.15.52:0/729354848 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb73406ac30 msgr2=0x7fb73406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:45.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.400+0000 7fb75d200640 1 --2- 172.21.15.52:0/729354848 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb73406ac30 0x7fb73406d0f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb7581f5f20 tx=0x7fb740008040 comp rx=0 tx=0).stop 2024-03-12T17:11:45.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.400+0000 7fb75d200640 1 -- 172.21.15.52:0/729354848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7580cbf70 msgr2=0x7fb75815fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:45.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.400+0000 7fb75d200640 1 --2- 172.21.15.52:0/729354848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7580cbf70 0x7fb75815fc80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb7480021f0 tx=0x7fb74800b560 comp rx=0 tx=0).stop 2024-03-12T17:11:45.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.400+0000 7fb757e00640 1 -- 172.21.15.52:0/729354848 reap_dead start 2024-03-12T17:11:45.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.400+0000 7fb75d200640 1 -- 172.21.15.52:0/729354848 shutdown_connections 2024-03-12T17:11:45.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.400+0000 7fb75d200640 1 -- 172.21.15.52:0/729354848 >> 172.21.15.52:0/729354848 conn(0x7fb75806b050 msgr2=0x7fb75806b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:45.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.400+0000 7fb75d200640 1 -- 172.21.15.52:0/729354848 shutdown_connections 2024-03-12T17:11:45.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.400+0000 7fb75d200640 1 -- 172.21.15.52:0/729354848 wait complete. 2024-03-12T17:11:45.412 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-03-12T17:11:45.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.637+0000 7fb15b200640 1 Processor -- start 2024-03-12T17:11:45.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.637+0000 7fb15b200640 1 -- start start 2024-03-12T17:11:45.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.638+0000 7fb15b200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb15415c890 0x7fb154160480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:45.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.639+0000 7fb15b200640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb1540cdb80 0x7fb1540d10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:45.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.639+0000 7fb15b200640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb154161390 con 0x7fb1540b80a0 2024-03-12T17:11:45.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.639+0000 7fb15b200640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1540d0ca0 con 0x7fb1540cdb80 2024-03-12T17:11:45.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.639+0000 7fb15b200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb154160dd0 con 0x7fb15415c890 2024-03-12T17:11:45.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.639+0000 7fb15a800640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb1540cdb80 0x7fb1540d10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:45.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.639+0000 7fb15a800640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb1540cdb80 0x7fb1540d10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45752/0 (socket says 172.21.15.52:45752) 2024-03-12T17:11:45.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.639+0000 7fb15a800640 1 -- 172.21.15.52:0/1809148387 learned_addr learned my addr 172.21.15.52:0/1809148387 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:45.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.639+0000 7fb159e00640 1 --2- 172.21.15.52:0/1809148387 >> v2:172.21.15.102:3300/0 conn(0x7fb15415c890 0x7fb154160480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:45.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.639+0000 7fb15a800640 1 -- 172.21.15.52:0/1809148387 >> v2:172.21.15.102:3300/0 conn(0x7fb15415c890 msgr2=0x7fb154160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:45.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.639+0000 7fb15a800640 1 --2- 172.21.15.52:0/1809148387 >> v2:172.21.15.102:3300/0 conn(0x7fb15415c890 0x7fb154160480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:45.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.639+0000 7fb15a800640 1 -- 172.21.15.52:0/1809148387 >> v1:172.21.15.52:6789/0 conn(0x7fb1540b80a0 legacy=0x7fb1540cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:45.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.639+0000 7fb15a800640 1 -- 172.21.15.52:0/1809148387 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1541610b0 con 0x7fb1540cdb80 2024-03-12T17:11:45.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.641+0000 7fb15a800640 1 --2- 172.21.15.52:0/1809148387 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1540cdb80 0x7fb1540d10c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb1540d3710 tx=0x7fb15002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b390357cb5d996 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:45.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.641+0000 7fb158a00640 1 -- 172.21.15.52:0/1809148387 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb15002ca80 con 0x7fb1540cdb80 2024-03-12T17:11:45.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.641+0000 7fb158a00640 1 -- 172.21.15.52:0/1809148387 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb15002cbe0 con 0x7fb1540cdb80 2024-03-12T17:11:45.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.641+0000 7fb158a00640 1 -- 172.21.15.52:0/1809148387 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1500343f0 con 0x7fb1540cdb80 2024-03-12T17:11:45.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.641+0000 7fb15b200640 1 -- 172.21.15.52:0/1809148387 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1540cdb80 msgr2=0x7fb1540d10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:45.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.642+0000 7fb15b200640 1 --2- 172.21.15.52:0/1809148387 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1540cdb80 0x7fb1540d10c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb1540d3710 tx=0x7fb15002bee0 comp rx=0 tx=0).stop 2024-03-12T17:11:45.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.642+0000 7fb15b200640 1 -- 172.21.15.52:0/1809148387 shutdown_connections 2024-03-12T17:11:45.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.642+0000 7fb15b200640 1 --2- 172.21.15.52:0/1809148387 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1540cdb80 0x7fb1540d10c0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:45.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.642+0000 7fb15b200640 1 --2- 172.21.15.52:0/1809148387 >> v2:172.21.15.102:3300/0 conn(0x7fb15415c890 0x7fb154160480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:45.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.642+0000 7fb15b200640 1 -- 172.21.15.52:0/1809148387 >> 172.21.15.52:0/1809148387 conn(0x7fb15406ae90 msgr2=0x7fb15406b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:45.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.642+0000 7fb15b200640 1 -- 172.21.15.52:0/1809148387 shutdown_connections 2024-03-12T17:11:45.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.642+0000 7fb15b200640 1 -- 172.21.15.52:0/1809148387 wait complete. 2024-03-12T17:11:45.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.644+0000 7fb15b200640 1 Processor -- start 2024-03-12T17:11:45.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.644+0000 7fb15b200640 1 -- start start 2024-03-12T17:11:45.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.645+0000 7fb15b200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1540b80a0 0x7fb1540b6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:45.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.645+0000 7fb15b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1540cdb80 0x7fb1540b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:45.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.645+0000 7fb15b200640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb15415c890 0x7fb154151790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:45.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.645+0000 7fb15b200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb154153790 con 0x7fb1540b80a0 2024-03-12T17:11:45.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.645+0000 7fb15b200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1540b29e0 con 0x7fb1540cdb80 2024-03-12T17:11:45.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.645+0000 7fb15b200640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1540b2cc0 con 0x7fb15415c890 2024-03-12T17:11:45.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.645+0000 7fb159e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1540b80a0 0x7fb1540b6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:45.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.645+0000 7fb159e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1540b80a0 0x7fb1540b6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45764/0 (socket says 172.21.15.52:45764) 2024-03-12T17:11:45.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.645+0000 7fb159e00640 1 -- 172.21.15.52:0/1269041934 learned_addr learned my addr 172.21.15.52:0/1269041934 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:45.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.645+0000 7fb159e00640 1 -- 172.21.15.52:0/1269041934 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb15415c890 msgr2=0x7fb154151790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:45.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.645+0000 7fb159400640 1 --2- 172.21.15.52:0/1269041934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1540cdb80 0x7fb1540b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:45.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.645+0000 7fb159e00640 1 --2- 172.21.15.52:0/1269041934 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb15415c890 0x7fb154151790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:45.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.645+0000 7fb159e00640 1 -- 172.21.15.52:0/1269041934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1540cdb80 msgr2=0x7fb1540b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:45.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.646+0000 7fb159e00640 1 --2- 172.21.15.52:0/1269041934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1540cdb80 0x7fb1540b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:45.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.646+0000 7fb159e00640 1 -- 172.21.15.52:0/1269041934 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb150009b30 con 0x7fb1540b80a0 2024-03-12T17:11:45.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.646+0000 7fb159400640 1 --2- 172.21.15.52:0/1269041934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1540cdb80 0x7fb1540b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:11:45.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.646+0000 7fb159e00640 1 --2- 172.21.15.52:0/1269041934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1540b80a0 0x7fb1540b6e90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb14800d5f0 tx=0x7fb14800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:45.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.646+0000 7fb14e000640 1 -- 172.21.15.52:0/1269041934 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1480125f0 con 0x7fb1540b80a0 2024-03-12T17:11:45.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.646+0000 7fb14e000640 1 -- 172.21.15.52:0/1269041934 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb148012750 con 0x7fb1540b80a0 2024-03-12T17:11:45.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.646+0000 7fb14e000640 1 -- 172.21.15.52:0/1269041934 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb148012b00 con 0x7fb1540b80a0 2024-03-12T17:11:45.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.646+0000 7fb15b200640 1 -- 172.21.15.52:0/1269041934 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb15415cdf0 con 0x7fb1540b80a0 2024-03-12T17:11:45.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.646+0000 7fb15b200640 1 -- 172.21.15.52:0/1269041934 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1540b3280 con 0x7fb1540b80a0 2024-03-12T17:11:45.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.648+0000 7fb15b200640 1 -- 172.21.15.52:0/1269041934 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1540b3410 con 0x7fb1540b80a0 2024-03-12T17:11:45.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.649+0000 7fb14e000640 1 -- 172.21.15.52:0/1269041934 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb148012c60 con 0x7fb1540b80a0 2024-03-12T17:11:45.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.649+0000 7fb14e000640 1 --2- 172.21.15.52:0/1269041934 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb13406ac00 0x7fb13406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:45.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.650+0000 7fb14e000640 1 -- 172.21.15.52:0/1269041934 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fb148089680 con 0x7fb1540b80a0 2024-03-12T17:11:45.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.650+0000 7fb159400640 1 --2- 172.21.15.52:0/1269041934 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb13406ac00 0x7fb13406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:45.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.651+0000 7fb159400640 1 --2- 172.21.15.52:0/1269041934 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb13406ac00 0x7fb13406d0c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb15415bda0 tx=0x7fb140009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:45.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.655+0000 7fb14e000640 1 -- 172.21.15.52:0/1269041934 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fb1540b3410 con 0x7fb1540b80a0 2024-03-12T17:11:45.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.854+0000 7fb15b200640 1 -- 172.21.15.52:0/1269041934 --> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fb15415cc70 con 0x7fb13406ac00 2024-03-12T17:11:45.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.855+0000 7fb14e000640 1 -- 172.21.15.52:0/1269041934 <== mgr.4109 v2:172.21.15.102:6832/36622 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb1400013e0 con 0x7fb13406ac00 2024-03-12T17:11:45.848 INFO:teuthology.orchestra.run.smithi052.stderr:instructed osd(s) 5 to deep-scrub 2024-03-12T17:11:45.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.857+0000 7fb15b200640 1 -- 172.21.15.52:0/1269041934 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb13406ac00 msgr2=0x7fb13406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:45.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.857+0000 7fb15b200640 1 --2- 172.21.15.52:0/1269041934 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb13406ac00 0x7fb13406d0c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb15415bda0 tx=0x7fb140009040 comp rx=0 tx=0).stop 2024-03-12T17:11:45.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.857+0000 7fb15b200640 1 -- 172.21.15.52:0/1269041934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1540b80a0 msgr2=0x7fb1540b6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:45.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.857+0000 7fb15b200640 1 --2- 172.21.15.52:0/1269041934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1540b80a0 0x7fb1540b6e90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb14800d5f0 tx=0x7fb14800dab0 comp rx=0 tx=0).stop 2024-03-12T17:11:45.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.858+0000 7fb15b200640 1 -- 172.21.15.52:0/1269041934 shutdown_connections 2024-03-12T17:11:45.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.858+0000 7fb15b200640 1 --2- 172.21.15.52:0/1269041934 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fb13406ac00 0x7fb13406d0c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:45.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.858+0000 7fb15b200640 1 --2- 172.21.15.52:0/1269041934 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb15415c890 0x7fb154151790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:45.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.858+0000 7fb15b200640 1 --2- 172.21.15.52:0/1269041934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1540cdb80 0x7fb1540b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:45.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.858+0000 7fb15b200640 1 --2- 172.21.15.52:0/1269041934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1540b80a0 0x7fb1540b6e90 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:45.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.858+0000 7fb15b200640 1 -- 172.21.15.52:0/1269041934 >> 172.21.15.52:0/1269041934 conn(0x7fb15406ae90 msgr2=0x7fb15406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:45.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.858+0000 7fb15b200640 1 -- 172.21.15.52:0/1269041934 shutdown_connections 2024-03-12T17:11:45.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:45.858+0000 7fb15b200640 1 -- 172.21.15.52:0/1269041934 wait complete. 2024-03-12T17:11:45.872 INFO:tasks.ceph:Scrubbing osd.6 2024-03-12T17:11:45.872 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-03-12T17:11:46.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.107+0000 7f278c400640 1 Processor -- start 2024-03-12T17:11:46.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.107+0000 7f278c400640 1 -- start start 2024-03-12T17:11:46.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.108+0000 7f278c400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f27840cd740 0x7f27840b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:46.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.108+0000 7f278c400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f27840b5aa0 0x7f27840b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:46.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.109+0000 7f278c400640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27840d20c0 con 0x7f27840d0d40 2024-03-12T17:11:46.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.109+0000 7f278c400640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27840d14b0 con 0x7f27840cd740 2024-03-12T17:11:46.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.109+0000 7f278c400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27840d1790 con 0x7f27840b5aa0 2024-03-12T17:11:46.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.109+0000 7f2789a00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f27840cd740 0x7f27840b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:46.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.109+0000 7f2789a00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f27840cd740 0x7f27840b84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45768/0 (socket says 172.21.15.52:45768) 2024-03-12T17:11:46.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.109+0000 7f2789a00640 1 -- 172.21.15.52:0/3825027885 learned_addr learned my addr 172.21.15.52:0/3825027885 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:46.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.109+0000 7f2789000640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f27840d0d40 0x7f27840cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:58618/0 (socket says 172.21.15.52:58618) 2024-03-12T17:11:46.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.109+0000 7f278a400640 1 --2- 172.21.15.52:0/3825027885 >> v2:172.21.15.102:3300/0 conn(0x7f27840b5aa0 0x7f27840b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:46.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.109+0000 7f2789a00640 1 -- 172.21.15.52:0/3825027885 >> v2:172.21.15.102:3300/0 conn(0x7f27840b5aa0 msgr2=0x7f27840b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:46.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.109+0000 7f2789a00640 1 --2- 172.21.15.52:0/3825027885 >> v2:172.21.15.102:3300/0 conn(0x7f27840b5aa0 0x7f27840b5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:46.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.109+0000 7f2789a00640 1 -- 172.21.15.52:0/3825027885 >> v1:172.21.15.52:6789/0 conn(0x7f27840d0d40 legacy=0x7f27840cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:46.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.109+0000 7f2789a00640 1 -- 172.21.15.52:0/3825027885 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27840d1a70 con 0x7f27840cd740 2024-03-12T17:11:46.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.109+0000 7f278a400640 1 --2- 172.21.15.52:0/3825027885 >> v2:172.21.15.102:3300/0 conn(0x7f27840b5aa0 0x7f27840b5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:11:46.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.111+0000 7f2789a00640 1 --2- 172.21.15.52:0/3825027885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27840cd740 0x7f27840b84b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f27840c99c0 tx=0x7f277802b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e1d2075fc2407f server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:46.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.111+0000 7f2783e00640 1 -- 172.21.15.52:0/3825027885 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2778030880 con 0x7f27840cd740 2024-03-12T17:11:46.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.111+0000 7f2783e00640 1 -- 172.21.15.52:0/3825027885 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27780309e0 con 0x7f27840cd740 2024-03-12T17:11:46.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.111+0000 7f2783e00640 1 -- 172.21.15.52:0/3825027885 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2778030d70 con 0x7f27840cd740 2024-03-12T17:11:46.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.112+0000 7f278c400640 1 -- 172.21.15.52:0/3825027885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27840cd740 msgr2=0x7f27840b84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:46.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.112+0000 7f278c400640 1 --2- 172.21.15.52:0/3825027885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27840cd740 0x7f27840b84b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f27840c99c0 tx=0x7f277802b9c0 comp rx=0 tx=0).stop 2024-03-12T17:11:46.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.112+0000 7f278c400640 1 -- 172.21.15.52:0/3825027885 shutdown_connections 2024-03-12T17:11:46.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.112+0000 7f278c400640 1 --2- 172.21.15.52:0/3825027885 >> v2:172.21.15.102:3300/0 conn(0x7f27840b5aa0 0x7f27840b5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:46.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.112+0000 7f278c400640 1 --2- 172.21.15.52:0/3825027885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27840cd740 0x7f27840b84b0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:46.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.112+0000 7f278c400640 1 -- 172.21.15.52:0/3825027885 >> 172.21.15.52:0/3825027885 conn(0x7f27840cf5d0 msgr2=0x7f27840cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:46.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.112+0000 7f278c400640 1 -- 172.21.15.52:0/3825027885 shutdown_connections 2024-03-12T17:11:46.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.112+0000 7f278c400640 1 -- 172.21.15.52:0/3825027885 wait complete. 2024-03-12T17:11:46.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.114+0000 7f278c400640 1 Processor -- start 2024-03-12T17:11:46.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.114+0000 7f278c400640 1 -- start start 2024-03-12T17:11:46.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.115+0000 7f278c400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27840b5aa0 0x7f27840a9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:46.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.115+0000 7f278c400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27840cd740 0x7f278414b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:46.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.115+0000 7f278c400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f27840d0d40 0x7f27840abcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:46.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.115+0000 7f278c400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278414cb00 con 0x7f27840cd740 2024-03-12T17:11:46.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.115+0000 7f278c400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278414bef0 con 0x7f27840b5aa0 2024-03-12T17:11:46.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.115+0000 7f278c400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f278414c1d0 con 0x7f27840d0d40 2024-03-12T17:11:46.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.115+0000 7f2789000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27840cd740 0x7f278414b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:46.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.115+0000 7f2789000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27840cd740 0x7f278414b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45782/0 (socket says 172.21.15.52:45782) 2024-03-12T17:11:46.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.115+0000 7f2789000640 1 -- 172.21.15.52:0/2922137590 learned_addr learned my addr 172.21.15.52:0/2922137590 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:46.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.115+0000 7f2789a00640 1 --2- 172.21.15.52:0/2922137590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27840b5aa0 0x7f27840a9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:46.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.116+0000 7f278a400640 1 --2- 172.21.15.52:0/2922137590 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f27840d0d40 0x7f27840abcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:46.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.116+0000 7f2789000640 1 -- 172.21.15.52:0/2922137590 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f27840d0d40 msgr2=0x7f27840abcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:46.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.116+0000 7f2789000640 1 --2- 172.21.15.52:0/2922137590 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f27840d0d40 0x7f27840abcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:46.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.116+0000 7f2789000640 1 -- 172.21.15.52:0/2922137590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27840b5aa0 msgr2=0x7f27840a9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:46.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.116+0000 7f2789000640 1 --2- 172.21.15.52:0/2922137590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27840b5aa0 0x7f27840a9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:46.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.116+0000 7f2789000640 1 -- 172.21.15.52:0/2922137590 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27780094d0 con 0x7f27840cd740 2024-03-12T17:11:46.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.116+0000 7f2789000640 1 --2- 172.21.15.52:0/2922137590 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27840cd740 0x7f278414b720 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f277000c3e0 tx=0x7f277000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:46.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.116+0000 7f2781600640 1 -- 172.21.15.52:0/2922137590 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27700074e0 con 0x7f27840cd740 2024-03-12T17:11:46.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.116+0000 7f2781600640 1 -- 172.21.15.52:0/2922137590 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2770007640 con 0x7f27840cd740 2024-03-12T17:11:46.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.116+0000 7f2781600640 1 -- 172.21.15.52:0/2922137590 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27700049e0 con 0x7f27840cd740 2024-03-12T17:11:46.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.116+0000 7f278c400640 1 -- 172.21.15.52:0/2922137590 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f278414c4b0 con 0x7f27840cd740 2024-03-12T17:11:46.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.117+0000 7f278c400640 1 -- 172.21.15.52:0/2922137590 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27840ac210 con 0x7f27840cd740 2024-03-12T17:11:46.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.118+0000 7f278c400640 1 -- 172.21.15.52:0/2922137590 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2784160300 con 0x7f27840cd740 2024-03-12T17:11:46.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.119+0000 7f2781600640 1 -- 172.21.15.52:0/2922137590 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2770004b40 con 0x7f27840cd740 2024-03-12T17:11:46.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.119+0000 7f2781600640 1 --2- 172.21.15.52:0/2922137590 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f276406ac00 0x7f276406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:46.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.120+0000 7f2781600640 1 -- 172.21.15.52:0/2922137590 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f2770088b80 con 0x7f27840cd740 2024-03-12T17:11:46.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.120+0000 7f2781600640 1 --2- 172.21.15.52:0/2922137590 >> [v2:172.21.15.102:6808/37192,v1:172.21.15.102:6809/37192] conn(0x7f2764070f00 0x7f2764073340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:46.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.120+0000 7f2781600640 1 -- 172.21.15.52:0/2922137590 --> [v2:172.21.15.102:6808/37192,v1:172.21.15.102:6809/37192] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2764073a10 con 0x7f2764070f00 2024-03-12T17:11:46.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.120+0000 7f2781600640 1 -- 172.21.15.52:0/2922137590 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f277008d260 con 0x7f27840cd740 2024-03-12T17:11:46.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.120+0000 7f2789a00640 1 --2- 172.21.15.52:0/2922137590 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f276406ac00 0x7f276406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:46.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.120+0000 7f278a400640 1 --2- 172.21.15.52:0/2922137590 >> [v2:172.21.15.102:6808/37192,v1:172.21.15.102:6809/37192] conn(0x7f2764070f00 0x7f2764073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:46.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.121+0000 7f278a400640 1 --2- 172.21.15.52:0/2922137590 >> [v2:172.21.15.102:6808/37192,v1:172.21.15.102:6809/37192] conn(0x7f2764070f00 0x7f2764073340 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:46.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.121+0000 7f2789a00640 1 --2- 172.21.15.52:0/2922137590 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f276406ac00 0x7f276406d0c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2778036000 tx=0x7f2778008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:46.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.123+0000 7f2781600640 1 -- 172.21.15.52:0/2922137590 <== osd.6 v2:172.21.15.102:6808/37192 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f2764073a10 con 0x7f2764070f00 2024-03-12T17:11:46.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.138+0000 7f278c400640 1 -- 172.21.15.52:0/2922137590 --> [v2:172.21.15.102:6808/37192,v1:172.21.15.102:6809/37192] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f27840d11d0 con 0x7f2764070f00 2024-03-12T17:11:46.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.139+0000 7f2781600640 1 -- 172.21.15.52:0/2922137590 <== osd.6 v2:172.21.15.102:6808/37192 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f277c00e5a0 con 0x7f2764070f00 2024-03-12T17:11:46.132 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-03-12T17:11:46.132 INFO:teuthology.orchestra.run.smithi052.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-12T17:11:46.132 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-03-12T17:11:46.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.139+0000 7f278c400640 1 -- 172.21.15.52:0/2922137590 >> [v2:172.21.15.102:6808/37192,v1:172.21.15.102:6809/37192] conn(0x7f2764070f00 msgr2=0x7f2764073340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:46.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.139+0000 7f278c400640 1 --2- 172.21.15.52:0/2922137590 >> [v2:172.21.15.102:6808/37192,v1:172.21.15.102:6809/37192] conn(0x7f2764070f00 0x7f2764073340 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:46.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.139+0000 7f278c400640 1 -- 172.21.15.52:0/2922137590 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f276406ac00 msgr2=0x7f276406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:46.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.139+0000 7f278c400640 1 --2- 172.21.15.52:0/2922137590 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f276406ac00 0x7f276406d0c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2778036000 tx=0x7f2778008030 comp rx=0 tx=0).stop 2024-03-12T17:11:46.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.139+0000 7f278c400640 1 -- 172.21.15.52:0/2922137590 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27840cd740 msgr2=0x7f278414b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:46.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.139+0000 7f278c400640 1 --2- 172.21.15.52:0/2922137590 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27840cd740 0x7f278414b720 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f277000c3e0 tx=0x7f277000c8a0 comp rx=0 tx=0).stop 2024-03-12T17:11:46.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.139+0000 7f278a400640 1 -- 172.21.15.52:0/2922137590 reap_dead start 2024-03-12T17:11:46.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.139+0000 7f278c400640 1 -- 172.21.15.52:0/2922137590 shutdown_connections 2024-03-12T17:11:46.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.139+0000 7f278c400640 1 -- 172.21.15.52:0/2922137590 >> 172.21.15.52:0/2922137590 conn(0x7f27840cf5d0 msgr2=0x7f278415ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:46.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.139+0000 7f278c400640 1 -- 172.21.15.52:0/2922137590 shutdown_connections 2024-03-12T17:11:46.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.140+0000 7f278c400640 1 -- 172.21.15.52:0/2922137590 wait complete. 2024-03-12T17:11:46.150 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-03-12T17:11:46.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.368+0000 7f69ebe00640 1 Processor -- start 2024-03-12T17:11:46.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.369+0000 7f69ebe00640 1 -- start start 2024-03-12T17:11:46.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.370+0000 7f69ebe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f69ec15b630 0x7f69ec0700d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:46.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.370+0000 7f69ebe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f69ec070610 0x7f69ec06c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:46.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.370+0000 7f69ebe00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69ec15b1a0 con 0x7f69ec0ccd40 2024-03-12T17:11:46.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.370+0000 7f69ebe00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69ec070aa0 con 0x7f69ec15b630 2024-03-12T17:11:46.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.370+0000 7f69ebe00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69ec06d940 con 0x7f69ec070610 2024-03-12T17:11:46.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.371+0000 7f69eaa00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f69ec0ccd40 0x7f69ec15e030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:58632/0 (socket says 172.21.15.52:58632) 2024-03-12T17:11:46.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.371+0000 7f69e3e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f69ec15b630 0x7f69ec0700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:46.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.371+0000 7f69eaa00640 1 -- 172.21.15.52:0/2884203637 learned_addr learned my addr 172.21.15.52:0/2884203637 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:11:46.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.371+0000 7f69eb400640 1 --2- 172.21.15.52:0/2884203637 >> v2:172.21.15.102:3300/0 conn(0x7f69ec070610 0x7f69ec06c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:46.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.372+0000 7f69ea000640 1 -- 172.21.15.52:0/2884203637 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2714810988 0 0) 0x7f69ec15b1a0 con 0x7f69ec0ccd40 2024-03-12T17:11:46.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.372+0000 7f69ea000640 1 -- 172.21.15.52:0/2884203637 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69d0003520 con 0x7f69ec0ccd40 2024-03-12T17:11:46.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.372+0000 7f69eb400640 1 -- 172.21.15.52:0/2884203637 >> v2:172.21.15.52:3300/0 conn(0x7f69ec15b630 msgr2=0x7f69ec0700d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:46.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.372+0000 7f69eb400640 1 --2- 172.21.15.52:0/2884203637 >> v2:172.21.15.52:3300/0 conn(0x7f69ec15b630 0x7f69ec0700d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:46.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.372+0000 7f69eb400640 1 -- 172.21.15.52:0/2884203637 >> v1:172.21.15.52:6789/0 conn(0x7f69ec0ccd40 legacy=0x7f69ec15e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:46.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.372+0000 7f69eb400640 1 -- 172.21.15.52:0/2884203637 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69ec15eba0 con 0x7f69ec070610 2024-03-12T17:11:46.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.373+0000 7f69eb400640 1 --2- 172.21.15.52:0/2884203637 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69ec070610 0x7f69ec06c130 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f69ec0b15e0 tx=0x7f69e402be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe8a8727f73860cb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:46.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.374+0000 7f69ea000640 1 -- 172.21.15.52:0/2884203637 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e402cbf0 con 0x7f69ec070610 2024-03-12T17:11:46.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.374+0000 7f69ea000640 1 -- 172.21.15.52:0/2884203637 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e402cd50 con 0x7f69ec070610 2024-03-12T17:11:46.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.374+0000 7f69ea000640 1 -- 172.21.15.52:0/2884203637 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e4034660 con 0x7f69ec070610 2024-03-12T17:11:46.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.374+0000 7f69ebe00640 1 -- 172.21.15.52:0/2884203637 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69ec070610 msgr2=0x7f69ec06c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:46.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.374+0000 7f69ebe00640 1 --2- 172.21.15.52:0/2884203637 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69ec070610 0x7f69ec06c130 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f69ec0b15e0 tx=0x7f69e402be40 comp rx=0 tx=0).stop 2024-03-12T17:11:46.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.375+0000 7f69ebe00640 1 -- 172.21.15.52:0/2884203637 shutdown_connections 2024-03-12T17:11:46.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.375+0000 7f69ebe00640 1 --2- 172.21.15.52:0/2884203637 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69ec070610 0x7f69ec06c130 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:46.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.375+0000 7f69ebe00640 1 --2- 172.21.15.52:0/2884203637 >> v2:172.21.15.52:3300/0 conn(0x7f69ec15b630 0x7f69ec0700d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:46.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.375+0000 7f69ebe00640 1 -- 172.21.15.52:0/2884203637 >> 172.21.15.52:0/2884203637 conn(0x7f69ec065df0 msgr2=0x7f69ec069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:46.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.375+0000 7f69ebe00640 1 -- 172.21.15.52:0/2884203637 shutdown_connections 2024-03-12T17:11:46.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.375+0000 7f69ebe00640 1 -- 172.21.15.52:0/2884203637 wait complete. 2024-03-12T17:11:46.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.376+0000 7f69ebe00640 1 Processor -- start 2024-03-12T17:11:46.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.377+0000 7f69ebe00640 1 -- start start 2024-03-12T17:11:46.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.378+0000 7f69ebe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69ec070610 0x7f69ec14ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:46.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.378+0000 7f69ebe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69ec0ccd40 0x7f69ec0ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:46.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.378+0000 7f69ebe00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f69ec15b630 0x7f69ec0ad380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:46.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.378+0000 7f69ebe00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec0acb30 con 0x7f69ec070610 2024-03-12T17:11:46.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.378+0000 7f69ebe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec0ac570 con 0x7f69ec0ccd40 2024-03-12T17:11:46.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.378+0000 7f69ebe00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec0ac850 con 0x7f69ec15b630 2024-03-12T17:11:46.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.378+0000 7f69eaa00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69ec070610 0x7f69ec14ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:46.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.378+0000 7f69eaa00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69ec070610 0x7f69ec14ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45804/0 (socket says 172.21.15.52:45804) 2024-03-12T17:11:46.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.378+0000 7f69eaa00640 1 -- 172.21.15.52:0/4032630109 learned_addr learned my addr 172.21.15.52:0/4032630109 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:46.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.378+0000 7f69eaa00640 1 -- 172.21.15.52:0/4032630109 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f69ec15b630 msgr2=0x7f69ec0ad380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:46.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.378+0000 7f69eaa00640 1 --2- 172.21.15.52:0/4032630109 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f69ec15b630 0x7f69ec0ad380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:46.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.378+0000 7f69eaa00640 1 -- 172.21.15.52:0/4032630109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69ec0ccd40 msgr2=0x7f69ec0ace40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:46.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.378+0000 7f69eaa00640 1 --2- 172.21.15.52:0/4032630109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69ec0ccd40 0x7f69ec0ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:46.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.378+0000 7f69eaa00640 1 -- 172.21.15.52:0/4032630109 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e4009a70 con 0x7f69ec070610 2024-03-12T17:11:46.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.379+0000 7f69eaa00640 1 --2- 172.21.15.52:0/4032630109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69ec070610 0x7f69ec14ea90 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f69dc00c500 tx=0x7f69dc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:46.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.379+0000 7f69e3400640 1 -- 172.21.15.52:0/4032630109 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69dc005ce0 con 0x7f69ec070610 2024-03-12T17:11:46.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.379+0000 7f69e3400640 1 -- 172.21.15.52:0/4032630109 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69dc005e40 con 0x7f69ec070610 2024-03-12T17:11:46.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.379+0000 7f69e3400640 1 -- 172.21.15.52:0/4032630109 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69dc007330 con 0x7f69ec070610 2024-03-12T17:11:46.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.379+0000 7f69ebe00640 1 -- 172.21.15.52:0/4032630109 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69ec0b1780 con 0x7f69ec070610 2024-03-12T17:11:46.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.379+0000 7f69ebe00640 1 -- 172.21.15.52:0/4032630109 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69ec0ab8b0 con 0x7f69ec070610 2024-03-12T17:11:46.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.380+0000 7f69ebe00640 1 -- 172.21.15.52:0/4032630109 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69ec0b1ca0 con 0x7f69ec070610 2024-03-12T17:11:46.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.381+0000 7f69e3400640 1 -- 172.21.15.52:0/4032630109 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f69dc007550 con 0x7f69ec070610 2024-03-12T17:11:46.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.382+0000 7f69e3400640 1 --2- 172.21.15.52:0/4032630109 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f69c806ac00 0x7f69c806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:46.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.382+0000 7f69e3400640 1 -- 172.21.15.52:0/4032630109 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f69dc088880 con 0x7f69ec070610 2024-03-12T17:11:46.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.383+0000 7f69e3e00640 1 --2- 172.21.15.52:0/4032630109 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f69c806ac00 0x7f69c806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:46.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.384+0000 7f69e3e00640 1 --2- 172.21.15.52:0/4032630109 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f69c806ac00 0x7f69c806d0c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f69d4006a90 tx=0x7f69d4006a20 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:46.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.387+0000 7f69e3400640 1 -- 172.21.15.52:0/4032630109 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f69ec0b1ca0 con 0x7f69ec070610 2024-03-12T17:11:46.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.582+0000 7f69ebe00640 1 -- 172.21.15.52:0/4032630109 --> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f69ec15ee80 con 0x7f69c806ac00 2024-03-12T17:11:46.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.583+0000 7f69e3400640 1 -- 172.21.15.52:0/4032630109 <== mgr.4109 v2:172.21.15.102:6832/36622 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f69d400e7a0 con 0x7f69c806ac00 2024-03-12T17:11:46.577 INFO:teuthology.orchestra.run.smithi052.stderr:instructed osd(s) 6 to deep-scrub 2024-03-12T17:11:46.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.586+0000 7f69ebe00640 1 -- 172.21.15.52:0/4032630109 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f69c806ac00 msgr2=0x7f69c806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:46.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.586+0000 7f69ebe00640 1 --2- 172.21.15.52:0/4032630109 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f69c806ac00 0x7f69c806d0c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f69d4006a90 tx=0x7f69d4006a20 comp rx=0 tx=0).stop 2024-03-12T17:11:46.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.586+0000 7f69ebe00640 1 -- 172.21.15.52:0/4032630109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69ec070610 msgr2=0x7f69ec14ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:46.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.586+0000 7f69ebe00640 1 --2- 172.21.15.52:0/4032630109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69ec070610 0x7f69ec14ea90 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f69dc00c500 tx=0x7f69dc00c9c0 comp rx=0 tx=0).stop 2024-03-12T17:11:46.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.586+0000 7f69ebe00640 1 -- 172.21.15.52:0/4032630109 shutdown_connections 2024-03-12T17:11:46.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.586+0000 7f69ebe00640 1 --2- 172.21.15.52:0/4032630109 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f69c806ac00 0x7f69c806d0c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:46.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.586+0000 7f69ebe00640 1 --2- 172.21.15.52:0/4032630109 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f69ec15b630 0x7f69ec0ad380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:46.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.586+0000 7f69ebe00640 1 --2- 172.21.15.52:0/4032630109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69ec0ccd40 0x7f69ec0ace40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:46.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.586+0000 7f69ebe00640 1 --2- 172.21.15.52:0/4032630109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69ec070610 0x7f69ec14ea90 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:46.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.586+0000 7f69ebe00640 1 -- 172.21.15.52:0/4032630109 >> 172.21.15.52:0/4032630109 conn(0x7f69ec065df0 msgr2=0x7f69ec1588e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:46.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.586+0000 7f69ebe00640 1 -- 172.21.15.52:0/4032630109 shutdown_connections 2024-03-12T17:11:46.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.586+0000 7f69ebe00640 1 -- 172.21.15.52:0/4032630109 wait complete. 2024-03-12T17:11:46.600 INFO:tasks.ceph:Scrubbing osd.7 2024-03-12T17:11:46.601 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-03-12T17:11:46.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.828+0000 7f2f55400640 1 Processor -- start 2024-03-12T17:11:46.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.828+0000 7f2f55400640 1 -- start start 2024-03-12T17:11:46.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.830+0000 7f2f55400640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2f50160fc0 0x7f2f5006b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:46.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.830+0000 7f2f55400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2f50099400 0x7f2f500c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:46.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.830+0000 7f2f55400640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f500c3ff0 con 0x7f2f500c3ba0 2024-03-12T17:11:46.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.830+0000 7f2f55400640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f50099bc0 con 0x7f2f50160fc0 2024-03-12T17:11:46.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.830+0000 7f2f55400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f500ce400 con 0x7f2f50099400 2024-03-12T17:11:46.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.830+0000 7f2f4e800640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2f50160fc0 0x7f2f5006b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:46.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.830+0000 7f2f4f200640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f2f500c3ba0 0x7f2f50164a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:58640/0 (socket says 172.21.15.52:58640) 2024-03-12T17:11:46.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.830+0000 7f2f4e800640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2f50160fc0 0x7f2f5006b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45820/0 (socket says 172.21.15.52:45820) 2024-03-12T17:11:46.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.830+0000 7f2f4f200640 1 -- 172.21.15.52:0/2990830943 learned_addr learned my addr 172.21.15.52:0/2990830943 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:11:46.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.830+0000 7f2f4de00640 1 --2- 172.21.15.52:0/2990830943 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2f50099400 0x7f2f500c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:46.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.831+0000 7f2f4e800640 1 -- 172.21.15.52:0/2990830943 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2f50099400 msgr2=0x7f2f500c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:46.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.831+0000 7f2f4e800640 1 --2- 172.21.15.52:0/2990830943 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2f50099400 0x7f2f500c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:46.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.831+0000 7f2f4e800640 1 -- 172.21.15.52:0/2990830943 >> v1:172.21.15.52:6789/0 conn(0x7f2f500c3ba0 legacy=0x7f2f50164a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:46.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.831+0000 7f2f4e800640 1 -- 172.21.15.52:0/2990830943 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f500ce6e0 con 0x7f2f50160fc0 2024-03-12T17:11:46.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.831+0000 7f2f4de00640 1 --2- 172.21.15.52:0/2990830943 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2f50099400 0x7f2f500c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-12T17:11:46.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.831+0000 7f2f4d400640 1 -- 172.21.15.52:0/2990830943 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1296518300 0 0) 0x7f2f500c3ff0 con 0x7f2f500c3ba0 2024-03-12T17:11:46.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.832+0000 7f2f4e800640 1 --2- 172.21.15.52:0/2990830943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f50160fc0 0x7f2f5006b260 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2f40009d90 tx=0x7f2f4002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e115b9022df73998 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:46.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.832+0000 7f2f4d400640 1 -- 172.21.15.52:0/2990830943 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4002cc50 con 0x7f2f50160fc0 2024-03-12T17:11:46.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.832+0000 7f2f4d400640 1 -- 172.21.15.52:0/2990830943 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f4002cdb0 con 0x7f2f50160fc0 2024-03-12T17:11:46.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.832+0000 7f2f4d400640 1 -- 172.21.15.52:0/2990830943 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f40034660 con 0x7f2f50160fc0 2024-03-12T17:11:46.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.833+0000 7f2f55400640 1 -- 172.21.15.52:0/2990830943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f50160fc0 msgr2=0x7f2f5006b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:46.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.833+0000 7f2f55400640 1 --2- 172.21.15.52:0/2990830943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f50160fc0 0x7f2f5006b260 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2f40009d90 tx=0x7f2f4002bea0 comp rx=0 tx=0).stop 2024-03-12T17:11:46.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.833+0000 7f2f55400640 1 -- 172.21.15.52:0/2990830943 shutdown_connections 2024-03-12T17:11:46.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.833+0000 7f2f55400640 1 --2- 172.21.15.52:0/2990830943 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2f50099400 0x7f2f500c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:46.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.833+0000 7f2f55400640 1 --2- 172.21.15.52:0/2990830943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f50160fc0 0x7f2f5006b260 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:46.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.833+0000 7f2f55400640 1 -- 172.21.15.52:0/2990830943 >> 172.21.15.52:0/2990830943 conn(0x7f2f500b2180 msgr2=0x7f2f500b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:46.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.833+0000 7f2f55400640 1 -- 172.21.15.52:0/2990830943 shutdown_connections 2024-03-12T17:11:46.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.833+0000 7f2f55400640 1 -- 172.21.15.52:0/2990830943 wait complete. 2024-03-12T17:11:46.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.835+0000 7f2f55400640 1 Processor -- start 2024-03-12T17:11:46.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.835+0000 7f2f55400640 1 -- start start 2024-03-12T17:11:46.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.836+0000 7f2f55400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f50099400 0x7f2f501f7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:46.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.836+0000 7f2f55400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f500c3ba0 0x7f2f500c9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:46.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.836+0000 7f2f55400640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2f50160fc0 0x7f2f500cd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:46.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.836+0000 7f2f55400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f500c87f0 con 0x7f2f500c3ba0 2024-03-12T17:11:46.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.836+0000 7f2f55400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f500b3390 con 0x7f2f50099400 2024-03-12T17:11:46.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.836+0000 7f2f55400640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f501f7cd0 con 0x7f2f50160fc0 2024-03-12T17:11:46.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.837+0000 7f2f4de00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f500c3ba0 0x7f2f500c9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:46.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.837+0000 7f2f4de00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f500c3ba0 0x7f2f500c9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45824/0 (socket says 172.21.15.52:45824) 2024-03-12T17:11:46.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.837+0000 7f2f4de00640 1 -- 172.21.15.52:0/534994429 learned_addr learned my addr 172.21.15.52:0/534994429 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:46.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.837+0000 7f2f4e800640 1 --2- 172.21.15.52:0/534994429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f50099400 0x7f2f501f7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:46.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.837+0000 7f2f4de00640 1 -- 172.21.15.52:0/534994429 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2f50160fc0 msgr2=0x7f2f500cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:46.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.837+0000 7f2f4de00640 1 --2- 172.21.15.52:0/534994429 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2f50160fc0 0x7f2f500cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:46.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.837+0000 7f2f4de00640 1 -- 172.21.15.52:0/534994429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f50099400 msgr2=0x7f2f501f7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:46.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.837+0000 7f2f4de00640 1 --2- 172.21.15.52:0/534994429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f50099400 0x7f2f501f7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:46.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.837+0000 7f2f4de00640 1 -- 172.21.15.52:0/534994429 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f40009a70 con 0x7f2f500c3ba0 2024-03-12T17:11:46.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.837+0000 7f2f4de00640 1 --2- 172.21.15.52:0/534994429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f500c3ba0 0x7f2f500c9170 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c00d7d0 tx=0x7f2f3c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:46.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.838+0000 7f2f46a00640 1 -- 172.21.15.52:0/534994429 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c004030 con 0x7f2f500c3ba0 2024-03-12T17:11:46.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.838+0000 7f2f46a00640 1 -- 172.21.15.52:0/534994429 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3c004190 con 0x7f2f500c3ba0 2024-03-12T17:11:46.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.838+0000 7f2f46a00640 1 -- 172.21.15.52:0/534994429 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c004520 con 0x7f2f500c3ba0 2024-03-12T17:11:46.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.838+0000 7f2f55400640 1 -- 172.21.15.52:0/534994429 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f500c7cd0 con 0x7f2f500c3ba0 2024-03-12T17:11:46.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.838+0000 7f2f55400640 1 -- 172.21.15.52:0/534994429 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f500c8510 con 0x7f2f500c3ba0 2024-03-12T17:11:46.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.839+0000 7f2f55400640 1 -- 172.21.15.52:0/534994429 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2f50099860 con 0x7f2f500c3ba0 2024-03-12T17:11:46.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.840+0000 7f2f46a00640 1 -- 172.21.15.52:0/534994429 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2f3c00b6c0 con 0x7f2f500c3ba0 2024-03-12T17:11:46.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.841+0000 7f2f46a00640 1 --2- 172.21.15.52:0/534994429 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2f3006ac00 0x7f2f3006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:46.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.841+0000 7f2f46a00640 1 -- 172.21.15.52:0/534994429 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f2f3c0893c0 con 0x7f2f500c3ba0 2024-03-12T17:11:46.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.841+0000 7f2f46a00640 1 --2- 172.21.15.52:0/534994429 >> [v2:172.21.15.102:6824/37214,v1:172.21.15.102:6825/37214] conn(0x7f2f30070f00 0x7f2f30073340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:46.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.841+0000 7f2f46a00640 1 -- 172.21.15.52:0/534994429 --> [v2:172.21.15.102:6824/37214,v1:172.21.15.102:6825/37214] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2f30073a10 con 0x7f2f30070f00 2024-03-12T17:11:46.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.841+0000 7f2f46a00640 1 -- 172.21.15.52:0/534994429 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f3c08ba80 con 0x7f2f500c3ba0 2024-03-12T17:11:46.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.842+0000 7f2f4e800640 1 --2- 172.21.15.52:0/534994429 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2f3006ac00 0x7f2f3006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:46.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.842+0000 7f2f4f200640 1 --2- 172.21.15.52:0/534994429 >> [v2:172.21.15.102:6824/37214,v1:172.21.15.102:6825/37214] conn(0x7f2f30070f00 0x7f2f30073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:46.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.843+0000 7f2f4e800640 1 --2- 172.21.15.52:0/534994429 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2f3006ac00 0x7f2f3006d0c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2f40036000 tx=0x7f2f40030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:46.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.843+0000 7f2f4f200640 1 --2- 172.21.15.52:0/534994429 >> [v2:172.21.15.102:6824/37214,v1:172.21.15.102:6825/37214] conn(0x7f2f30070f00 0x7f2f30073340 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:46.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.845+0000 7f2f46a00640 1 -- 172.21.15.52:0/534994429 <== osd.7 v2:172.21.15.102:6824/37214 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f2f30073a10 con 0x7f2f30070f00 2024-03-12T17:11:46.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.859+0000 7f2f55400640 1 -- 172.21.15.52:0/534994429 --> [v2:172.21.15.102:6824/37214,v1:172.21.15.102:6825/37214] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2f500c8070 con 0x7f2f30070f00 2024-03-12T17:11:46.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.860+0000 7f2f46a00640 1 -- 172.21.15.52:0/534994429 <== osd.7 v2:172.21.15.102:6824/37214 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f2f500c8070 con 0x7f2f30070f00 2024-03-12T17:11:46.854 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-03-12T17:11:46.854 INFO:teuthology.orchestra.run.smithi052.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-12T17:11:46.854 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-03-12T17:11:46.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.861+0000 7f2f55400640 1 -- 172.21.15.52:0/534994429 >> [v2:172.21.15.102:6824/37214,v1:172.21.15.102:6825/37214] conn(0x7f2f30070f00 msgr2=0x7f2f30073340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:46.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.861+0000 7f2f55400640 1 --2- 172.21.15.52:0/534994429 >> [v2:172.21.15.102:6824/37214,v1:172.21.15.102:6825/37214] conn(0x7f2f30070f00 0x7f2f30073340 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:46.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.861+0000 7f2f55400640 1 -- 172.21.15.52:0/534994429 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2f3006ac00 msgr2=0x7f2f3006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:46.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.861+0000 7f2f55400640 1 --2- 172.21.15.52:0/534994429 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f2f3006ac00 0x7f2f3006d0c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2f40036000 tx=0x7f2f40030000 comp rx=0 tx=0).stop 2024-03-12T17:11:46.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.861+0000 7f2f55400640 1 -- 172.21.15.52:0/534994429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f500c3ba0 msgr2=0x7f2f500c9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:46.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.861+0000 7f2f55400640 1 --2- 172.21.15.52:0/534994429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f500c3ba0 0x7f2f500c9170 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c00d7d0 tx=0x7f2f3c00dc90 comp rx=0 tx=0).stop 2024-03-12T17:11:46.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.861+0000 7f2f4f200640 1 -- 172.21.15.52:0/534994429 reap_dead start 2024-03-12T17:11:46.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.861+0000 7f2f55400640 1 -- 172.21.15.52:0/534994429 shutdown_connections 2024-03-12T17:11:46.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.861+0000 7f2f55400640 1 -- 172.21.15.52:0/534994429 >> 172.21.15.52:0/534994429 conn(0x7f2f500b2180 msgr2=0x7f2f5006f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:46.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.861+0000 7f2f55400640 1 -- 172.21.15.52:0/534994429 shutdown_connections 2024-03-12T17:11:46.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:46.861+0000 7f2f55400640 1 -- 172.21.15.52:0/534994429 wait complete. 2024-03-12T17:11:46.871 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-03-12T17:11:47.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.097+0000 7f939ea00640 1 Processor -- start 2024-03-12T17:11:47.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.097+0000 7f939ea00640 1 -- start start 2024-03-12T17:11:47.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.098+0000 7f939ea00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f939813f770 0x7f939806d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:47.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.098+0000 7f939ea00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f939806d780 0x7f93980cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:47.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.098+0000 7f939ea00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f939813f2e0 con 0x7f939813c3d0 2024-03-12T17:11:47.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.098+0000 7f939ea00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93980ccac0 con 0x7f939813f770 2024-03-12T17:11:47.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.098+0000 7f939ea00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f939806dbf0 con 0x7f939806d780 2024-03-12T17:11:47.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.099+0000 7f939e000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f939806d780 0x7f93980cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:47.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.099+0000 7f939cc00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f939813f770 0x7f939806d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:47.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.099+0000 7f939e000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f939806d780 0x7f93980cd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.52:43258/0 (socket says 172.21.15.52:43258) 2024-03-12T17:11:47.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.099+0000 7f939e000640 1 -- 172.21.15.52:0/1614665433 learned_addr learned my addr 172.21.15.52:0/1614665433 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:47.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.100+0000 7f939e000640 1 -- 172.21.15.52:0/1614665433 >> v2:172.21.15.102:3300/0 conn(0x7f939813f770 msgr2=0x7f939806d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:47.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.100+0000 7f939e000640 1 --2- 172.21.15.52:0/1614665433 >> v2:172.21.15.102:3300/0 conn(0x7f939813f770 0x7f939806d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:47.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.100+0000 7f939e000640 1 -- 172.21.15.52:0/1614665433 >> v1:172.21.15.102:6789/0 conn(0x7f939813c3d0 legacy=0x7f9398159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:47.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.100+0000 7f939e000640 1 -- 172.21.15.52:0/1614665433 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939815a5f0 con 0x7f939806d780 2024-03-12T17:11:47.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.102+0000 7f939e000640 1 --2- 172.21.15.52:0/1614665433 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f939806d780 0x7f93980cd260 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f939000d150 tx=0x7f939002e160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1be275187161c0d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:47.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.102+0000 7f9397e00640 1 -- 172.21.15.52:0/1614665433 <== mon.2 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939000b0d0 con 0x7f939806d780 2024-03-12T17:11:47.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.102+0000 7f9397e00640 1 -- 172.21.15.52:0/1614665433 <== mon.2 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939000b230 con 0x7f939806d780 2024-03-12T17:11:47.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.102+0000 7f9397e00640 1 -- 172.21.15.52:0/1614665433 <== mon.2 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9390009600 con 0x7f939806d780 2024-03-12T17:11:47.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.103+0000 7f939ea00640 1 -- 172.21.15.52:0/1614665433 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f939806d780 msgr2=0x7f93980cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:47.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.103+0000 7f939ea00640 1 --2- 172.21.15.52:0/1614665433 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f939806d780 0x7f93980cd260 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f939000d150 tx=0x7f939002e160 comp rx=0 tx=0).stop 2024-03-12T17:11:47.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.103+0000 7f939ea00640 1 -- 172.21.15.52:0/1614665433 shutdown_connections 2024-03-12T17:11:47.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.103+0000 7f939ea00640 1 --2- 172.21.15.52:0/1614665433 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f939806d780 0x7f93980cd260 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:47.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.103+0000 7f939ea00640 1 --2- 172.21.15.52:0/1614665433 >> v2:172.21.15.102:3300/0 conn(0x7f939813f770 0x7f939806d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:47.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.103+0000 7f939ea00640 1 -- 172.21.15.52:0/1614665433 >> 172.21.15.52:0/1614665433 conn(0x7f9398065df0 msgr2=0x7f9398069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:47.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.103+0000 7f939ea00640 1 -- 172.21.15.52:0/1614665433 shutdown_connections 2024-03-12T17:11:47.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.104+0000 7f939ea00640 1 -- 172.21.15.52:0/1614665433 wait complete. 2024-03-12T17:11:47.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.105+0000 7f939ea00640 1 Processor -- start 2024-03-12T17:11:47.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.105+0000 7f939ea00640 1 -- start start 2024-03-12T17:11:47.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.106+0000 7f939ea00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f939806d780 0x7f93981f1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:47.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.106+0000 7f939ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f939813c3d0 0x7f93980ae030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:47.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.106+0000 7f939ea00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f939813f770 0x7f9398131700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:47.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.106+0000 7f939ea00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93980af8a0 con 0x7f939813f770 2024-03-12T17:11:47.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.106+0000 7f939ea00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93980ab950 con 0x7f939813c3d0 2024-03-12T17:11:47.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.107+0000 7f939ea00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93980abc30 con 0x7f939806d780 2024-03-12T17:11:47.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.107+0000 7f939e000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f939813f770 0x7f9398131700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:47.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.107+0000 7f939e000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f939813f770 0x7f9398131700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45828/0 (socket says 172.21.15.52:45828) 2024-03-12T17:11:47.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.107+0000 7f939e000640 1 -- 172.21.15.52:0/345043344 learned_addr learned my addr 172.21.15.52:0/345043344 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:47.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.107+0000 7f939cc00640 1 --2- 172.21.15.52:0/345043344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f939813c3d0 0x7f93980ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:47.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.107+0000 7f939d600640 1 --2- 172.21.15.52:0/345043344 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f939806d780 0x7f93981f1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:47.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.107+0000 7f939e000640 1 -- 172.21.15.52:0/345043344 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f939806d780 msgr2=0x7f93981f1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:47.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.107+0000 7f939e000640 1 --2- 172.21.15.52:0/345043344 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f939806d780 0x7f93981f1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:47.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.107+0000 7f939e000640 1 -- 172.21.15.52:0/345043344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f939813c3d0 msgr2=0x7f93980ae030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:47.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.107+0000 7f939e000640 1 --2- 172.21.15.52:0/345043344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f939813c3d0 0x7f93980ae030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:47.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.107+0000 7f939e000640 1 -- 172.21.15.52:0/345043344 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939000bd40 con 0x7f939813f770 2024-03-12T17:11:47.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.108+0000 7f939e000640 1 --2- 172.21.15.52:0/345043344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f939813f770 0x7f9398131700 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9390038040 tx=0x7f9390004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:47.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.108+0000 7f9395600640 1 -- 172.21.15.52:0/345043344 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9390036800 con 0x7f939813f770 2024-03-12T17:11:47.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.108+0000 7f9395600640 1 -- 172.21.15.52:0/345043344 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9390036980 con 0x7f939813f770 2024-03-12T17:11:47.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.108+0000 7f939ea00640 1 -- 172.21.15.52:0/345043344 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93980abf10 con 0x7f939813f770 2024-03-12T17:11:47.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.108+0000 7f9395600640 1 -- 172.21.15.52:0/345043344 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9390036d30 con 0x7f939813f770 2024-03-12T17:11:47.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.108+0000 7f939ea00640 1 -- 172.21.15.52:0/345043344 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93980ac4d0 con 0x7f939813f770 2024-03-12T17:11:47.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.110+0000 7f939ea00640 1 -- 172.21.15.52:0/345043344 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9398071b10 con 0x7f939813f770 2024-03-12T17:11:47.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.110+0000 7f9395600640 1 -- 172.21.15.52:0/345043344 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f939004a460 con 0x7f939813f770 2024-03-12T17:11:47.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.111+0000 7f9395600640 1 --2- 172.21.15.52:0/345043344 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f937806ac00 0x7f937806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:47.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.111+0000 7f9395600640 1 -- 172.21.15.52:0/345043344 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f93900ac530 con 0x7f939813f770 2024-03-12T17:11:47.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.112+0000 7f939d600640 1 --2- 172.21.15.52:0/345043344 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f937806ac00 0x7f937806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:47.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.113+0000 7f939d600640 1 --2- 172.21.15.52:0/345043344 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f937806ac00 0x7f937806d0c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f938c000f60 tx=0x7f938c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:47.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.116+0000 7f9395600640 1 -- 172.21.15.52:0/345043344 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f9390077830 con 0x7f939813f770 2024-03-12T17:11:47.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.315+0000 7f939ea00640 1 -- 172.21.15.52:0/345043344 --> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f93980ade20 con 0x7f937806ac00 2024-03-12T17:11:47.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.316+0000 7f9395600640 1 -- 172.21.15.52:0/345043344 <== mgr.4109 v2:172.21.15.102:6832/36622 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f938c001c70 con 0x7f937806ac00 2024-03-12T17:11:47.309 INFO:teuthology.orchestra.run.smithi052.stderr:instructed osd(s) 7 to deep-scrub 2024-03-12T17:11:47.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.319+0000 7f939ea00640 1 -- 172.21.15.52:0/345043344 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f937806ac00 msgr2=0x7f937806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:47.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.319+0000 7f939ea00640 1 --2- 172.21.15.52:0/345043344 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f937806ac00 0x7f937806d0c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f938c000f60 tx=0x7f938c009040 comp rx=0 tx=0).stop 2024-03-12T17:11:47.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.319+0000 7f939ea00640 1 -- 172.21.15.52:0/345043344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f939813f770 msgr2=0x7f9398131700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:47.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.319+0000 7f939ea00640 1 --2- 172.21.15.52:0/345043344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f939813f770 0x7f9398131700 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9390038040 tx=0x7f9390004a40 comp rx=0 tx=0).stop 2024-03-12T17:11:47.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.319+0000 7f939ea00640 1 -- 172.21.15.52:0/345043344 shutdown_connections 2024-03-12T17:11:47.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.319+0000 7f939ea00640 1 --2- 172.21.15.52:0/345043344 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f937806ac00 0x7f937806d0c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:47.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.319+0000 7f939ea00640 1 --2- 172.21.15.52:0/345043344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f939813f770 0x7f9398131700 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:47.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.319+0000 7f939ea00640 1 --2- 172.21.15.52:0/345043344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f939813c3d0 0x7f93980ae030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:47.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.319+0000 7f939ea00640 1 --2- 172.21.15.52:0/345043344 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f939806d780 0x7f93981f1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:47.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.319+0000 7f939ea00640 1 -- 172.21.15.52:0/345043344 >> 172.21.15.52:0/345043344 conn(0x7f9398065df0 msgr2=0x7f939815ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:47.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.319+0000 7f939ea00640 1 -- 172.21.15.52:0/345043344 shutdown_connections 2024-03-12T17:11:47.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.320+0000 7f939ea00640 1 -- 172.21.15.52:0/345043344 wait complete. 2024-03-12T17:11:47.333 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-12T17:11:47.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.559+0000 7f0beea00640 1 Processor -- start 2024-03-12T17:11:47.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.560+0000 7f0beea00640 1 -- start start 2024-03-12T17:11:47.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.561+0000 7f0beea00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0be8160ca0 0x7f0be8161ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:47.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.561+0000 7f0beea00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0be806d110 0x7f0be80b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:47.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.561+0000 7f0beea00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0be80cda90 con 0x7f0be806c080 2024-03-12T17:11:47.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.561+0000 7f0beea00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0be80ce280 con 0x7f0be8160ca0 2024-03-12T17:11:47.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.561+0000 7f0beea00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be8162220 con 0x7f0be806d110 2024-03-12T17:11:47.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.562+0000 7f0bed600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0be8160ca0 0x7f0be8161ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:47.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.562+0000 7f0bee000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0be806d110 0x7f0be80b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:47.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.562+0000 7f0bed600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0be8160ca0 0x7f0be8161ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:47860/0 (socket says 172.21.15.52:47860) 2024-03-12T17:11:47.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.562+0000 7f0bed600640 1 -- 172.21.15.52:0/4136969603 learned_addr learned my addr 172.21.15.52:0/4136969603 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:47.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.562+0000 7f0bee000640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0be806d110 0x7f0be80b1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.52:43276/0 (socket says 172.21.15.52:43276) 2024-03-12T17:11:47.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.563+0000 7f0bed600640 1 -- 172.21.15.52:0/4136969603 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0be806d110 msgr2=0x7f0be80b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:11:47.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.563+0000 7f0bed600640 1 --2- 172.21.15.52:0/4136969603 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0be806d110 0x7f0be80b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:47.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.563+0000 7f0bed600640 1 -- 172.21.15.52:0/4136969603 >> v1:172.21.15.102:6789/0 conn(0x7f0be806c080 legacy=0x7f0be806ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:47.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.563+0000 7f0bed600640 1 -- 172.21.15.52:0/4136969603 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be80cc9f0 con 0x7f0be8160ca0 2024-03-12T17:11:47.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.564+0000 7f0bed600640 1 --2- 172.21.15.52:0/4136969603 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0be8160ca0 0x7f0be8161ce0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc009870 tx=0x7f0bdc02b900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53eb4059a3efc30c server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:47.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.565+0000 7f0be7e00640 1 -- 172.21.15.52:0/4136969603 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc031860 con 0x7f0be8160ca0 2024-03-12T17:11:47.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.565+0000 7f0be7e00640 1 -- 172.21.15.52:0/4136969603 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bdc0319c0 con 0x7f0be8160ca0 2024-03-12T17:11:47.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.565+0000 7f0be7e00640 1 -- 172.21.15.52:0/4136969603 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc031d50 con 0x7f0be8160ca0 2024-03-12T17:11:47.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.565+0000 7f0beea00640 1 -- 172.21.15.52:0/4136969603 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0be8160ca0 msgr2=0x7f0be8161ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:47.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.565+0000 7f0beea00640 1 --2- 172.21.15.52:0/4136969603 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0be8160ca0 0x7f0be8161ce0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc009870 tx=0x7f0bdc02b900 comp rx=0 tx=0).stop 2024-03-12T17:11:47.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.566+0000 7f0beea00640 1 -- 172.21.15.52:0/4136969603 shutdown_connections 2024-03-12T17:11:47.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.566+0000 7f0beea00640 1 --2- 172.21.15.52:0/4136969603 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0be806d110 0x7f0be80b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:47.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.566+0000 7f0beea00640 1 --2- 172.21.15.52:0/4136969603 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0be8160ca0 0x7f0be8161ce0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:47.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.566+0000 7f0beea00640 1 -- 172.21.15.52:0/4136969603 >> 172.21.15.52:0/4136969603 conn(0x7f0be8065df0 msgr2=0x7f0be8069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:47.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.566+0000 7f0beea00640 1 -- 172.21.15.52:0/4136969603 shutdown_connections 2024-03-12T17:11:47.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.566+0000 7f0beea00640 1 -- 172.21.15.52:0/4136969603 wait complete. 2024-03-12T17:11:47.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.568+0000 7f0beea00640 1 Processor -- start 2024-03-12T17:11:47.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.568+0000 7f0beea00640 1 -- start start 2024-03-12T17:11:47.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.569+0000 7f0beea00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0be806c080 0x7f0be814cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:47.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.569+0000 7f0beea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0be806d110 0x7f0be814d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:47.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.569+0000 7f0beea00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0be8160ca0 0x7f0be8163570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:47.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.569+0000 7f0beea00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be814e490 con 0x7f0be806c080 2024-03-12T17:11:47.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.569+0000 7f0beea00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be8153040 con 0x7f0be806d110 2024-03-12T17:11:47.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.569+0000 7f0beea00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be814d810 con 0x7f0be8160ca0 2024-03-12T17:11:47.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.569+0000 7f0bed600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0be806c080 0x7f0be814cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:47.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.569+0000 7f0bed600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0be806c080 0x7f0be814cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45840/0 (socket says 172.21.15.52:45840) 2024-03-12T17:11:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.569+0000 7f0bed600640 1 -- 172.21.15.52:0/3117703595 learned_addr learned my addr 172.21.15.52:0/3117703595 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:11:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.569+0000 7f0becc00640 1 --2- 172.21.15.52:0/3117703595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0be806d110 0x7f0be814d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.570+0000 7f0bed600640 1 -- 172.21.15.52:0/3117703595 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0be8160ca0 msgr2=0x7f0be8163570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.570+0000 7f0bee000640 1 --2- 172.21.15.52:0/3117703595 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0be8160ca0 0x7f0be8163570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.570+0000 7f0bed600640 1 --2- 172.21.15.52:0/3117703595 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0be8160ca0 0x7f0be8163570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.570+0000 7f0bed600640 1 -- 172.21.15.52:0/3117703595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0be806d110 msgr2=0x7f0be814d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.570+0000 7f0bed600640 1 --2- 172.21.15.52:0/3117703595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0be806d110 0x7f0be814d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.570+0000 7f0bed600640 1 -- 172.21.15.52:0/3117703595 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bdc0094d0 con 0x7f0be806c080 2024-03-12T17:11:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.570+0000 7f0bee000640 1 --2- 172.21.15.52:0/3117703595 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0be8160ca0 0x7f0be8163570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:11:47.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.570+0000 7f0bed600640 1 --2- 172.21.15.52:0/3117703595 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0be806c080 0x7f0be814cc50 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc000c00 tx=0x7f0bdc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:47.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.570+0000 7f0be5600640 1 -- 172.21.15.52:0/3117703595 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc03ac60 con 0x7f0be806c080 2024-03-12T17:11:47.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.570+0000 7f0be5600640 1 -- 172.21.15.52:0/3117703595 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bdc03adc0 con 0x7f0be806c080 2024-03-12T17:11:47.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.570+0000 7f0be5600640 1 -- 172.21.15.52:0/3117703595 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc044630 con 0x7f0be806c080 2024-03-12T17:11:47.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.571+0000 7f0beea00640 1 -- 172.21.15.52:0/3117703595 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be814daf0 con 0x7f0be806c080 2024-03-12T17:11:47.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.571+0000 7f0beea00640 1 -- 172.21.15.52:0/3117703595 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be814e0b0 con 0x7f0be806c080 2024-03-12T17:11:47.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.572+0000 7f0beea00640 1 -- 172.21.15.52:0/3117703595 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0be804e030 con 0x7f0be806c080 2024-03-12T17:11:47.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.573+0000 7f0be5600640 1 -- 172.21.15.52:0/3117703595 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0bdc04d450 con 0x7f0be806c080 2024-03-12T17:11:47.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.574+0000 7f0be5600640 1 --2- 172.21.15.52:0/3117703595 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f0bcc06ac00 0x7f0bcc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:11:47.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.574+0000 7f0be5600640 1 -- 172.21.15.52:0/3117703595 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f0bdc0ab690 con 0x7f0be806c080 2024-03-12T17:11:47.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.575+0000 7f0becc00640 1 --2- 172.21.15.52:0/3117703595 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f0bcc06ac00 0x7f0bcc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:11:47.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.576+0000 7f0becc00640 1 --2- 172.21.15.52:0/3117703595 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f0bcc06ac00 0x7f0bcc06d0c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0be8152ca0 tx=0x7f0bd8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:11:47.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.579+0000 7f0be5600640 1 -- 172.21.15.52:0/3117703595 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f0bdc0769a0 con 0x7f0be806c080 2024-03-12T17:11:47.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.773+0000 7f0beea00640 1 -- 172.21.15.52:0/3117703595 --> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0be80cd290 con 0x7f0bcc06ac00 2024-03-12T17:11:47.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.779+0000 7f0be5600640 1 -- 172.21.15.52:0/3117703595 <== mgr.4109 v2:172.21.15.102:6832/36622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48673 (secure 0 0 0) 0x7f0bd8001c70 con 0x7f0bcc06ac00 2024-03-12T17:11:47.773 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:11:47.774 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-03-12T17:11:47.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.782+0000 7f0beea00640 1 -- 172.21.15.52:0/3117703595 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f0bcc06ac00 msgr2=0x7f0bcc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:47.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.782+0000 7f0beea00640 1 --2- 172.21.15.52:0/3117703595 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f0bcc06ac00 0x7f0bcc06d0c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0be8152ca0 tx=0x7f0bd8009040 comp rx=0 tx=0).stop 2024-03-12T17:11:47.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.782+0000 7f0beea00640 1 -- 172.21.15.52:0/3117703595 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0be806c080 msgr2=0x7f0be814cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:11:47.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.782+0000 7f0beea00640 1 --2- 172.21.15.52:0/3117703595 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0be806c080 0x7f0be814cc50 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc000c00 tx=0x7f0bdc0046c0 comp rx=0 tx=0).stop 2024-03-12T17:11:47.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.782+0000 7f0beea00640 1 -- 172.21.15.52:0/3117703595 shutdown_connections 2024-03-12T17:11:47.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.782+0000 7f0beea00640 1 --2- 172.21.15.52:0/3117703595 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f0bcc06ac00 0x7f0bcc06d0c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:47.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.783+0000 7f0beea00640 1 --2- 172.21.15.52:0/3117703595 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0be8160ca0 0x7f0be8163570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:47.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.783+0000 7f0beea00640 1 --2- 172.21.15.52:0/3117703595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0be806d110 0x7f0be814d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:47.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.783+0000 7f0beea00640 1 --2- 172.21.15.52:0/3117703595 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0be806c080 0x7f0be814cc50 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:11:47.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.783+0000 7f0beea00640 1 -- 172.21.15.52:0/3117703595 >> 172.21.15.52:0/3117703595 conn(0x7f0be8065df0 msgr2=0x7f0be80b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:11:47.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.783+0000 7f0beea00640 1 -- 172.21.15.52:0/3117703595 shutdown_connections 2024-03-12T17:11:47.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:11:47.783+0000 7f0beea00640 1 -- 172.21.15.52:0/3117703595 wait complete. 2024-03-12T17:11:47.792 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2024-03-12T17:11:47.268677+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":237876,"kb_used_data":5320,"kb_used_omap":13,"kb_used_meta":232434,"kb_avail":754736844,"statfs":{"total":773094113280,"available":772850528256,"internally_reserved":0,"allocated":5447680,"data_stored":3020856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":238012504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003212"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":186,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-03-12T17:11:34.219639+0000","last_change":"2024-03-12T17:06:21.714813+0000","last_active":"2024-03-12T17:11:34.219639+0000","last_peered":"2024-03-12T17:11:34.219639+0000","last_clean":"2024-03-12T17:11:34.219639+0000","last_became_active":"2024-03-12T17:06:21.712488+0000","last_became_peered":"2024-03-12T17:06:21.712488+0000","last_unstale":"2024-03-12T17:11:34.219639+0000","last_undegraded":"2024-03-12T17:11:34.219639+0000","last_fullsized":"2024-03-12T17:11:34.219639+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T17:06:19.815688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T17:06:19.815688+0000","last_clean_scrub_stamp":"2024-03-12T17:06:19.815688+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T23:38:54.300793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574916,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28124,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343716,"statfs":{"total":96636764160,"available":96607965184,"internally_reserved":0,"allocated":1126400,"data_stored":820383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.1200000000000001,"15min":1.0389999999999999},"min":{"1min":0.68500000000000005,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.8080000000000001,"5min":13.563000000000001,"15min":13.563000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.004,"5min":1.1040000000000001,"15min":1.0740000000000001},"min":{"1min":0.68100000000000005,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.776,"5min":6.6779999999999999,"15min":6.6779999999999999},"last":1.089}]},{"osd":1,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.107,"15min":1.1519999999999999},"min":{"1min":0.79800000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.488,"5min":2.9990000000000001,"15min":2.9990000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.0429999999999999},"min":{"1min":0.748,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.5860000000000001,"5min":5.181,"15min":5.181},"last":0.57699999999999996}]},{"osd":2,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.167,"15min":1.1359999999999999},"min":{"1min":0.754,"5min":0.748,"15min":0.748},"max":{"1min":1.5189999999999999,"5min":5.0819999999999999,"15min":5.0819999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.2250000000000001,"15min":1.21},"min":{"1min":0.76300000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.425,"5min":6.5289999999999999,"15min":6.5289999999999999},"last":0.88200000000000001}]},{"osd":3,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.1339999999999999,"15min":1.1879999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.5149999999999999,"5min":3.9849999999999999,"15min":3.9849999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":1.264,"5min":1.258,"15min":1.26},"min":{"1min":0.98099999999999998,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.657,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":0.98799999999999999}]},{"osd":4,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.234,"15min":1.268},"min":{"1min":0.98299999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.8740000000000001,"5min":3.8580000000000001,"15min":3.8580000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.4099999999999999,"15min":1.407},"min":{"1min":1.093,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.5900000000000001,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":1.1779999999999999}]},{"osd":5,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.246,"15min":1.264},"min":{"1min":0.97599999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.5469999999999999,"5min":3.3799999999999999,"15min":3.3799999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":1.337,"5min":1.341,"15min":1.3600000000000001},"min":{"1min":0.98399999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.677,"5min":4.0750000000000002,"15min":4.0750000000000002},"last":1.0609999999999999}]},{"osd":6,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.375,"15min":1.3919999999999999},"min":{"1min":1.01,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.599,"5min":3.7599999999999998,"15min":3.7599999999999998},"last":1.337},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4690000000000001,"15min":1.53},"min":{"1min":0.90200000000000002,"5min":0.626,"15min":0.626},"max":{"1min":1.927,"5min":4.7560000000000002,"15min":4.7560000000000002},"last":1.02}]}]},{"osd":6,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":230015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.0409999999999999,"15min":1.0680000000000001},"min":{"1min":0.61799999999999999,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5600000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.073,"5min":1.0449999999999999,"15min":0.97199999999999998},"min":{"1min":0.70499999999999996,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.492,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":0.83899999999999997}]},{"osd":1,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":1.026,"15min":0.999},"min":{"1min":0.66700000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.2909999999999999,"5min":1.74,"15min":1.74},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.1259999999999999,"15min":1.1419999999999999},"min":{"1min":0.68300000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.486,"5min":1.73,"15min":1.73},"last":1.224}]},{"osd":2,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.222,"15min":1.1899999999999999},"min":{"1min":0.79300000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.355,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.125,"5min":1.143,"15min":1.165},"min":{"1min":0.74399999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5920000000000001,"5min":1.764,"15min":1.764},"last":1.153}]},{"osd":3,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.25,"15min":1.268},"min":{"1min":0.83999999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":1.274},{"interface":"front","average":{"1min":1.163,"5min":1.236,"15min":1.2470000000000001},"min":{"1min":0.89800000000000002,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.556,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.5249999999999999}]},{"osd":4,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.3440000000000001,"15min":1.3100000000000001},"min":{"1min":0.89700000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.7390000000000001,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":1.492},{"interface":"front","average":{"1min":1.319,"5min":1.3400000000000001,"15min":1.363},"min":{"1min":0.93500000000000005,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.554,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.3680000000000001}]},{"osd":5,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.3140000000000001,"15min":1.3260000000000001},"min":{"1min":0.89100000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.7,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.7190000000000001},{"interface":"front","average":{"1min":1.377,"5min":1.4590000000000001,"15min":1.4410000000000001},"min":{"1min":1.1279999999999999,"5min":0.5,"15min":0.5},"max":{"1min":1.6419999999999999,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":1.5600000000000001}]},{"osd":7,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.5469999999999999,"15min":1.458},"min":{"1min":0.96499999999999997,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.766,"5min":9.9610000000000003,"15min":9.9610000000000003},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.4610000000000001,"15min":1.464},"min":{"1min":0.98599999999999999,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.79,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.9079999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":230015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":1.0269999999999999,"15min":1.1160000000000001},"min":{"1min":0.71599999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2,"5min":3.6709999999999998,"15min":3.6709999999999998},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.995,"5min":1.069,"15min":1.1619999999999999},"min":{"1min":0.625,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.359,"5min":3.706,"15min":3.706},"last":1.353}]},{"osd":1,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.0600000000000001,"15min":1.077},"min":{"1min":0.73399999999999999,"5min":0.442,"15min":0.442},"max":{"1min":1.3089999999999999,"5min":3.0390000000000001,"15min":3.0390000000000001},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.165,"5min":1.161,"15min":1.1699999999999999},"min":{"1min":0.81299999999999994,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.407,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.399}]},{"osd":2,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.175,"15min":1.1919999999999999},"min":{"1min":0.82599999999999996,"5min":0.498,"15min":0.498},"max":{"1min":1.6499999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.1990000000000001,"15min":1.242},"min":{"1min":0.879,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4610000000000001,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.4750000000000001}]},{"osd":3,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2709999999999999,"15min":1.349},"min":{"1min":0.82499999999999996,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.478,"5min":3.7440000000000002,"15min":3.7440000000000002},"last":1.514},{"interface":"front","average":{"1min":1.262,"5min":1.2829999999999999,"15min":1.29},"min":{"1min":0.91600000000000004,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.5920000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":1.4419999999999999}]},{"osd":4,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2569999999999999,"15min":1.2410000000000001},"min":{"1min":0.63200000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.5580000000000001,"5min":1.873,"15min":1.873},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3280000000000001,"15min":1.294},"min":{"1min":0.67200000000000004,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.6040000000000001,"5min":2,"15min":2},"last":1.629}]},{"osd":6,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.4319999999999999,"15min":1.599},"min":{"1min":0.89300000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.7390000000000001,"5min":8.2080000000000002,"15min":8.2080000000000002},"last":1.25},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4419999999999999,"15min":1.4490000000000001},"min":{"1min":0.70299999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.5880000000000001}]},{"osd":7,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.4850000000000001,"5min":1.4630000000000001,"15min":1.417},"min":{"1min":0.79800000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.7130000000000001,"5min":2.173,"15min":2.173},"last":1.7010000000000001},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.575,"15min":1.764},"min":{"1min":1.0209999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":2.1339999999999999,"5min":8.0139999999999993,"15min":8.0139999999999993},"last":1.5549999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":230015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.1020000000000001,"15min":1.1739999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.476,"5min":1.655,"15min":1.655},"last":1.087},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.119,"15min":1.1879999999999999},"min":{"1min":0.73399999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.446,"5min":1.724,"15min":1.724},"last":1.0589999999999999}]},{"osd":1,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.0760000000000001,"15min":1.0149999999999999},"min":{"1min":0.81000000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.504,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.1479999999999999,"15min":1.1919999999999999},"min":{"1min":0.78100000000000003,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.47,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.3859999999999999}]},{"osd":2,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.1639999999999999,"15min":1.204},"min":{"1min":0.80900000000000005,"5min":0.752,"15min":0.752},"max":{"1min":1.899,"5min":1.982,"15min":1.982},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.224,"5min":1.1950000000000001,"15min":1.256},"min":{"1min":0.88600000000000001,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.546,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.4910000000000001}]},{"osd":3,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.161,"15min":1.171},"min":{"1min":0.748,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.6000000000000001,"5min":1.661,"15min":1.661},"last":1.25},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.3260000000000001,"15min":1.3979999999999999},"min":{"1min":0.90500000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.821,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.208}]},{"osd":5,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.3280000000000001,"15min":1.3520000000000001},"min":{"1min":0.94799999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.5389999999999999,"5min":2.2589999999999999,"15min":2.2589999999999999},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.393,"5min":1.3220000000000001,"15min":1.321},"min":{"1min":0.83399999999999996,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.6499999999999999,"5min":1.95,"15min":1.95},"last":1.359}]},{"osd":6,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.3979999999999999,"15min":1.389},"min":{"1min":0.94299999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.9470000000000001,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.431,"5min":1.462,"15min":1.512},"min":{"1min":1.093,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.675,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":1.274}]},{"osd":7,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.488,"5min":1.4279999999999999,"15min":1.4259999999999999},"min":{"1min":1.01,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.752,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.52,"5min":1.5469999999999999,"15min":1.615},"min":{"1min":1.1919999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":2.0779999999999998,"5min":3.5720000000000001,"15min":3.5720000000000001},"last":1.546}]}]},{"osd":3,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31640,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":532480,"data_stored":230015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:11:33 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.92600000000000005,"15min":1.1859999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.369,"5min":5.0039999999999996,"15min":5.0039999999999996},"last":0.435},{"interface":"front","average":{"1min":0.97599999999999998,"5min":1.0389999999999999,"15min":1.238},"min":{"1min":0.69699999999999995,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.375,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":0.70999999999999996}]},{"osd":1,"last update":"Tue Mar 12 17:11:33 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.96899999999999997,"15min":1.2390000000000001},"min":{"1min":0.53000000000000003,"5min":0.377,"15min":0.377},"max":{"1min":1.2350000000000001,"5min":4.9080000000000004,"15min":4.9080000000000004},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.98199999999999998,"15min":1.1739999999999999},"min":{"1min":0.64500000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.444,"5min":3.2410000000000001,"15min":3.2410000000000001},"last":0.66700000000000004}]},{"osd":2,"last update":"Tue Mar 12 17:11:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0700000000000001,"15min":1.3129999999999999},"min":{"1min":0.71799999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.4590000000000001,"5min":3.4990000000000001,"15min":3.4990000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.107,"5min":1.163,"15min":1.4550000000000001},"min":{"1min":0.67100000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.6639999999999999,"5min":5.1870000000000003,"15min":5.1870000000000003},"last":0.73699999999999999}]},{"osd":4,"last update":"Tue Mar 12 17:11:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.2030000000000001,"15min":1.5289999999999999},"min":{"1min":0.78000000000000003,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.5009999999999999,"5min":9.9299999999999997,"15min":9.9299999999999997},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.345,"15min":1.573},"min":{"1min":0.91300000000000003,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.8400000000000001,"5min":5.2450000000000001,"15min":5.2450000000000001},"last":1.131}]},{"osd":5,"last update":"Tue Mar 12 17:11:33 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.1850000000000001,"15min":1.3069999999999999},"min":{"1min":0.88100000000000001,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.5700000000000001,"5min":2.9550000000000001,"15min":2.9550000000000001},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.224,"5min":1.319,"15min":1.4990000000000001},"min":{"1min":0.875,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.9239999999999999,"5min":3.089,"15min":3.089},"last":1.1040000000000001}]},{"osd":6,"last update":"Tue Mar 12 17:11:33 2024","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3660000000000001,"15min":1.5580000000000001},"min":{"1min":0.96199999999999997,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":2.0049999999999999,"5min":3.3170000000000002,"15min":3.3170000000000002},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2569999999999999,"15min":1.401},"min":{"1min":0.85399999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.7509999999999999,"5min":2.722,"15min":2.722},"last":0.83599999999999997}]},{"osd":7,"last update":"Tue Mar 12 17:11:33 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.3,"15min":1.474},"min":{"1min":0.90400000000000003,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.8200000000000001,"5min":2.7559999999999998,"15min":2.7559999999999998},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.395,"15min":1.5489999999999999},"min":{"1min":0.96699999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2.032,"5min":3.1389999999999998,"15min":3.1389999999999998},"last":0.89200000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31636,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340204,"statfs":{"total":96636764160,"available":96604368896,"internally_reserved":0,"allocated":532480,"data_stored":230015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.78100000000000003,"15min":0.81499999999999995},"min":{"1min":0.498,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0780000000000001,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.88900000000000001,"15min":1.0149999999999999},"min":{"1min":0.56200000000000006,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.234,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":0.53300000000000003}]},{"osd":1,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.84399999999999997,"15min":0.91700000000000004},"min":{"1min":0.49099999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.2370000000000001,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.92200000000000004,"15min":0.94699999999999995},"min":{"1min":0.61199999999999999,"5min":0.439,"15min":0.439},"max":{"1min":1.573,"5min":1.732,"15min":1.732},"last":0.47799999999999998}]},{"osd":3,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.01,"15min":1.0229999999999999},"min":{"1min":0.56799999999999995,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.611,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.747},{"interface":"front","average":{"1min":0.998,"5min":0.95899999999999996,"15min":1.0249999999999999},"min":{"1min":0.57899999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.859,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.67300000000000004}]},{"osd":4,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.23,"15min":1.25},"min":{"1min":0.84799999999999998,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.9730000000000001,"5min":2.9569999999999999,"15min":2.9569999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3029999999999999,"15min":1.407},"min":{"1min":0.91700000000000004,"5min":0.751,"15min":0.751},"max":{"1min":1.7390000000000001,"5min":4.4640000000000004,"15min":4.4640000000000004},"last":1.016}]},{"osd":5,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.2889999999999999,"15min":1.359},"min":{"1min":0.999,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5900000000000001,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3049999999999999,"15min":1.353},"min":{"1min":0.90600000000000003,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.077,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":0.86499999999999999}]},{"osd":6,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.3109999999999999,"15min":1.4379999999999999},"min":{"1min":0.89200000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.663,"5min":3.5179999999999998,"15min":3.5179999999999998},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.341,"5min":1.3560000000000001,"15min":1.446},"min":{"1min":1.028,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.7869999999999999,"5min":2.464,"15min":2.464},"last":0.94399999999999995}]},{"osd":7,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.391,"15min":1.526},"min":{"1min":0.98499999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.7350000000000001,"5min":3.6110000000000002,"15min":3.6110000000000002},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.421,"15min":1.5209999999999999},"min":{"1min":0.872,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.004,"5min":2.8730000000000002,"15min":2.8730000000000002},"last":1.1319999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31636,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340204,"statfs":{"total":96636764160,"available":96604368896,"internally_reserved":0,"allocated":532480,"data_stored":230015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.85999999999999999,"15min":0.90300000000000002},"min":{"1min":0.54800000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.222,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.90200000000000002,"15min":1.034},"min":{"1min":0.56100000000000005,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.2729999999999999,"5min":3.6419999999999999,"15min":3.6419999999999999},"last":0.97499999999999998}]},{"osd":2,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.94999999999999996,"15min":1.052},"min":{"1min":0.57399999999999995,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4610000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.97099999999999997,"15min":1.1399999999999999},"min":{"1min":0.63300000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1879999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.1279999999999999}]},{"osd":3,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.0960000000000001,"15min":1.3200000000000001},"min":{"1min":0.69699999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.331,"5min":3.9580000000000002,"15min":3.9580000000000002},"last":1.1950000000000001},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.93000000000000005,"15min":1.04},"min":{"1min":0.52100000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.159,"5min":1.8,"15min":1.8},"last":0.50800000000000001}]},{"osd":4,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.2110000000000001,"15min":1.331},"min":{"1min":0.85199999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.5649999999999999,"5min":2.694,"15min":2.694},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.238,"5min":1.2789999999999999,"15min":1.403},"min":{"1min":0.89100000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.8999999999999999,"5min":3.5150000000000001,"15min":3.5150000000000001},"last":1.2549999999999999}]},{"osd":5,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.1819999999999999,"15min":1.25},"min":{"1min":0.78300000000000003,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.7470000000000001,"5min":2.109,"15min":2.109},"last":1.095},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.337,"15min":1.4690000000000001},"min":{"1min":0.95699999999999996,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.677,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":1.0589999999999999}]},{"osd":6,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3680000000000001,"15min":1.514},"min":{"1min":1.034,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.8240000000000001,"5min":4.1269999999999998,"15min":4.1269999999999998},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.244,"5min":1.272,"15min":1.417},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.575,"5min":3.7850000000000001,"15min":3.7850000000000001},"last":1.0269999999999999}]},{"osd":7,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3680000000000001,"15min":1.5049999999999999},"min":{"1min":0.91100000000000003,"5min":0.873,"15min":0.873},"max":{"1min":1.7689999999999999,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.377,"5min":1.4139999999999999,"15min":1.5529999999999999},"min":{"1min":1.075,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.74,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.173}]}]},{"osd":0,"up_from":13,"seq":55834574915,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1126400,"data_stored":820383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.79800000000000004,"15min":0.91600000000000004},"min":{"1min":0.40400000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.877,"15min":0.91400000000000003},"min":{"1min":0.438,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.3839999999999999,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.67400000000000004}]},{"osd":2,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":0.95199999999999996,"15min":1.052},"min":{"1min":0.58499999999999996,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.9410000000000001,"5min":1.952,"15min":1.952},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.98199999999999998,"15min":1.1060000000000001},"min":{"1min":0.55300000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.571,"5min":1.915,"15min":1.915},"last":0.60799999999999998}]},{"osd":3,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.94699999999999995,"15min":1.0029999999999999},"min":{"1min":0.53200000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":2.2189999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.071,"5min":1.0940000000000001,"15min":1.242},"min":{"1min":0.63,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.1000000000000001,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.59399999999999997}]},{"osd":4,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.216,"15min":1.3400000000000001},"min":{"1min":0.84799999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.4209999999999998,"5min":3.0499999999999998,"15min":3.0499999999999998},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.298,"5min":1.337,"15min":1.474},"min":{"1min":0.88500000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":2.3730000000000002,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":1.538}]},{"osd":5,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.347,"15min":1.4650000000000001},"min":{"1min":0.874,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.859,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":4.5780000000000003},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.3140000000000001,"15min":1.3939999999999999},"min":{"1min":0.96099999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.9990000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":4.6319999999999997}]},{"osd":6,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.361,"15min":1.5329999999999999},"min":{"1min":0.83099999999999996,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.2719999999999998,"5min":2.597,"15min":2.597},"last":1.407},{"interface":"front","average":{"1min":1.234,"5min":1.3420000000000001,"15min":1.4990000000000001},"min":{"1min":0.92200000000000004,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.6850000000000001,"5min":2.452,"15min":2.452},"last":0.86199999999999999}]},{"osd":7,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.3959999999999999,"15min":1.5529999999999999},"min":{"1min":0.86799999999999999,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.3180000000000001,"5min":2.7240000000000002,"15min":2.7240000000000002},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.321,"15min":1.518},"min":{"1min":0.86099999999999999,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.4670000000000001,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":2.839}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-12T17:11:47.797 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-12T17:06:19.815688+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=12, tm_hour=17, tm_min=6, tm_sec=19, tm_wday=1, tm_yday=72, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=12, tm_hour=17, tm_min=11, tm_sec=40, tm_wday=1, tm_yday=72, tm_isdst=0) 2024-03-12T17:11:47.797 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-12T17:11:54.640 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-12T17:11:54.649 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-12T17:12:07.798 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-12T17:12:08.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.035+0000 7f1c77c00640 1 Processor -- start 2024-03-12T17:12:08.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.036+0000 7f1c77c00640 1 -- start start 2024-03-12T17:12:08.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.037+0000 7f1c77c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1c7015cda0 0x7f1c700cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:12:08.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.037+0000 7f1c77c00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c700b5a30 0x7f1c700b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:12:08.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.037+0000 7f1c77c00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f1c700b4350 0x7f1c700b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:12:08.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.037+0000 7f1c77c00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c700b4d20 con 0x7f1c700b4350 2024-03-12T17:12:08.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.037+0000 7f1c77c00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c700d1830 con 0x7f1c7015cda0 2024-03-12T17:12:08.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.037+0000 7f1c77c00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c700b1f20 con 0x7f1c700b5a30 2024-03-12T17:12:08.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.038+0000 7f1c75c00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f1c700b4350 0x7f1c700b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:12:08.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.038+0000 7f1c75c00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f1c700b4350 0x7f1c700b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33322/0 (socket says 172.21.15.52:33322) 2024-03-12T17:12:08.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.038+0000 7f1c75c00640 1 -- 172.21.15.52:0/1798008417 learned_addr learned my addr 172.21.15.52:0/1798008417 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:12:08.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.038+0000 7f1c6fe00640 1 --2- 172.21.15.52:0/1798008417 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c700b5a30 0x7f1c700b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:12:08.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.038+0000 7f1c75200640 1 --2- 172.21.15.52:0/1798008417 >> v2:172.21.15.102:3300/0 conn(0x7f1c7015cda0 0x7f1c700cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:12:08.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.038+0000 7f1c75c00640 1 -- 172.21.15.52:0/1798008417 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c700b5a30 msgr2=0x7f1c700b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:12:08.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.038+0000 7f1c75c00640 1 --2- 172.21.15.52:0/1798008417 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c700b5a30 0x7f1c700b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:12:08.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.038+0000 7f1c75c00640 1 -- 172.21.15.52:0/1798008417 >> v2:172.21.15.102:3300/0 conn(0x7f1c7015cda0 msgr2=0x7f1c700cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:12:08.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.038+0000 7f1c75c00640 1 --2- 172.21.15.52:0/1798008417 >> v2:172.21.15.102:3300/0 conn(0x7f1c7015cda0 0x7f1c700cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:12:08.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.038+0000 7f1c75c00640 1 -- 172.21.15.52:0/1798008417 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c700d0c20 con 0x7f1c700b4350 2024-03-12T17:12:08.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.040+0000 7f1c75c00640 1 --2- 172.21.15.52:0/1798008417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c700b4350 0x7f1c700b47e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1c7006ad20 tx=0x7f1c6802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fae1eb8a20ea329 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:12:08.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.040+0000 7f1c6f400640 1 -- 172.21.15.52:0/1798008417 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c68031870 con 0x7f1c700b4350 2024-03-12T17:12:08.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.040+0000 7f1c6f400640 1 -- 172.21.15.52:0/1798008417 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c680319d0 con 0x7f1c700b4350 2024-03-12T17:12:08.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.040+0000 7f1c6f400640 1 -- 172.21.15.52:0/1798008417 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c68031d60 con 0x7f1c700b4350 2024-03-12T17:12:08.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.041+0000 7f1c77c00640 1 -- 172.21.15.52:0/1798008417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c700b4350 msgr2=0x7f1c700b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:12:08.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.041+0000 7f1c77c00640 1 --2- 172.21.15.52:0/1798008417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c700b4350 0x7f1c700b47e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1c7006ad20 tx=0x7f1c6802b910 comp rx=0 tx=0).stop 2024-03-12T17:12:08.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.041+0000 7f1c77c00640 1 -- 172.21.15.52:0/1798008417 shutdown_connections 2024-03-12T17:12:08.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.041+0000 7f1c77c00640 1 --2- 172.21.15.52:0/1798008417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c700b4350 0x7f1c700b47e0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:12:08.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.041+0000 7f1c77c00640 1 --2- 172.21.15.52:0/1798008417 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c700b5a30 0x7f1c700b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:12:08.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.041+0000 7f1c77c00640 1 --2- 172.21.15.52:0/1798008417 >> v2:172.21.15.102:3300/0 conn(0x7f1c7015cda0 0x7f1c700cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:12:08.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.041+0000 7f1c77c00640 1 -- 172.21.15.52:0/1798008417 >> 172.21.15.52:0/1798008417 conn(0x7f1c700cf350 msgr2=0x7f1c700cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:12:08.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.041+0000 7f1c77c00640 1 -- 172.21.15.52:0/1798008417 shutdown_connections 2024-03-12T17:12:08.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.041+0000 7f1c77c00640 1 -- 172.21.15.52:0/1798008417 wait complete. 2024-03-12T17:12:08.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.043+0000 7f1c77c00640 1 Processor -- start 2024-03-12T17:12:08.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.043+0000 7f1c77c00640 1 -- start start 2024-03-12T17:12:08.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.044+0000 7f1c77c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c700b4350 0x7f1c700d2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:12:08.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.044+0000 7f1c77c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c700b5a30 0x7f1c700c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:12:08.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.044+0000 7f1c77c00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c7015cda0 0x7f1c700c7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:12:08.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.044+0000 7f1c77c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c700c81e0 con 0x7f1c700b4350 2024-03-12T17:12:08.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.044+0000 7f1c77c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7014d380 con 0x7f1c700b5a30 2024-03-12T17:12:08.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.044+0000 7f1c77c00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c700c7940 con 0x7f1c7015cda0 2024-03-12T17:12:08.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.044+0000 7f1c75200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c700b4350 0x7f1c700d2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:12:08.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.044+0000 7f1c75200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c700b4350 0x7f1c700d2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33330/0 (socket says 172.21.15.52:33330) 2024-03-12T17:12:08.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.044+0000 7f1c75200640 1 -- 172.21.15.52:0/4230222162 learned_addr learned my addr 172.21.15.52:0/4230222162 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:12:08.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.045+0000 7f1c6fe00640 1 --2- 172.21.15.52:0/4230222162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c700b5a30 0x7f1c700c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:12:08.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.045+0000 7f1c75c00640 1 --2- 172.21.15.52:0/4230222162 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c7015cda0 0x7f1c700c7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:12:08.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.045+0000 7f1c75200640 1 -- 172.21.15.52:0/4230222162 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c7015cda0 msgr2=0x7f1c700c7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:12:08.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.045+0000 7f1c75200640 1 --2- 172.21.15.52:0/4230222162 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c7015cda0 0x7f1c700c7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:12:08.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.045+0000 7f1c75200640 1 -- 172.21.15.52:0/4230222162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c700b5a30 msgr2=0x7f1c700c6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:12:08.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.045+0000 7f1c75200640 1 --2- 172.21.15.52:0/4230222162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c700b5a30 0x7f1c700c6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:12:08.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.045+0000 7f1c75200640 1 -- 172.21.15.52:0/4230222162 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c680094d0 con 0x7f1c700b4350 2024-03-12T17:12:08.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.045+0000 7f1c75200640 1 --2- 172.21.15.52:0/4230222162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c700b4350 0x7f1c700d2e80 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f1c60007ec0 tx=0x7f1c6000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:12:08.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.045+0000 7f1c6cc00640 1 -- 172.21.15.52:0/4230222162 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c60014620 con 0x7f1c700b4350 2024-03-12T17:12:08.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.046+0000 7f1c6cc00640 1 -- 172.21.15.52:0/4230222162 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c600147a0 con 0x7f1c700b4350 2024-03-12T17:12:08.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.046+0000 7f1c6cc00640 1 -- 172.21.15.52:0/4230222162 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c60014b50 con 0x7f1c700b4350 2024-03-12T17:12:08.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.046+0000 7f1c77c00640 1 -- 172.21.15.52:0/4230222162 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c700c7c20 con 0x7f1c700b4350 2024-03-12T17:12:08.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.046+0000 7f1c77c00640 1 -- 172.21.15.52:0/4230222162 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c700c9ad0 con 0x7f1c700b4350 2024-03-12T17:12:08.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.047+0000 7f1c77c00640 1 -- 172.21.15.52:0/4230222162 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c70073390 con 0x7f1c700b4350 2024-03-12T17:12:08.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.048+0000 7f1c6cc00640 1 -- 172.21.15.52:0/4230222162 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1c60014cb0 con 0x7f1c700b4350 2024-03-12T17:12:08.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.049+0000 7f1c6cc00640 1 --2- 172.21.15.52:0/4230222162 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1c4c06ac00 0x7f1c4c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:12:08.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.049+0000 7f1c6cc00640 1 -- 172.21.15.52:0/4230222162 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f1c60089b70 con 0x7f1c700b4350 2024-03-12T17:12:08.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.049+0000 7f1c6fe00640 1 --2- 172.21.15.52:0/4230222162 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1c4c06ac00 0x7f1c4c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:12:08.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.051+0000 7f1c6fe00640 1 --2- 172.21.15.52:0/4230222162 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1c4c06ac00 0x7f1c4c06d0c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1c7015c1b0 tx=0x7f1c58008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:12:08.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.054+0000 7f1c6cc00640 1 -- 172.21.15.52:0/4230222162 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f1c60054f00 con 0x7f1c700b4350 2024-03-12T17:12:08.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.249+0000 7f1c77c00640 1 -- 172.21.15.52:0/4230222162 --> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1c700d14c0 con 0x7f1c4c06ac00 2024-03-12T17:12:08.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.256+0000 7f1c6cc00640 1 -- 172.21.15.52:0/4230222162 <== mgr.4109 v2:172.21.15.102:6832/36622 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48379 (secure 0 0 0) 0x7f1c5800b6f0 con 0x7f1c4c06ac00 2024-03-12T17:12:08.251 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:12:08.251 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-03-12T17:12:08.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.259+0000 7f1c77c00640 1 -- 172.21.15.52:0/4230222162 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1c4c06ac00 msgr2=0x7f1c4c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:12:08.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.259+0000 7f1c77c00640 1 --2- 172.21.15.52:0/4230222162 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1c4c06ac00 0x7f1c4c06d0c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1c7015c1b0 tx=0x7f1c58008040 comp rx=0 tx=0).stop 2024-03-12T17:12:08.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.259+0000 7f1c77c00640 1 -- 172.21.15.52:0/4230222162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c700b4350 msgr2=0x7f1c700d2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:12:08.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.259+0000 7f1c77c00640 1 --2- 172.21.15.52:0/4230222162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c700b4350 0x7f1c700d2e80 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f1c60007ec0 tx=0x7f1c6000e4b0 comp rx=0 tx=0).stop 2024-03-12T17:12:08.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.259+0000 7f1c77c00640 1 -- 172.21.15.52:0/4230222162 shutdown_connections 2024-03-12T17:12:08.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.259+0000 7f1c77c00640 1 --2- 172.21.15.52:0/4230222162 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f1c4c06ac00 0x7f1c4c06d0c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:12:08.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.259+0000 7f1c77c00640 1 --2- 172.21.15.52:0/4230222162 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c7015cda0 0x7f1c700c7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:12:08.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.259+0000 7f1c77c00640 1 --2- 172.21.15.52:0/4230222162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c700b5a30 0x7f1c700c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:12:08.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.259+0000 7f1c77c00640 1 --2- 172.21.15.52:0/4230222162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c700b4350 0x7f1c700d2e80 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:12:08.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.259+0000 7f1c77c00640 1 -- 172.21.15.52:0/4230222162 >> 172.21.15.52:0/4230222162 conn(0x7f1c700cf350 msgr2=0x7f1c700b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:12:08.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.259+0000 7f1c77c00640 1 -- 172.21.15.52:0/4230222162 shutdown_connections 2024-03-12T17:12:08.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.259+0000 7f1c77c00640 1 -- 172.21.15.52:0/4230222162 wait complete. 2024-03-12T17:12:08.274 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2024-03-12T17:12:07.274258+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":237860,"kb_used_data":5320,"kb_used_omap":13,"kb_used_meta":232434,"kb_avail":754736860,"statfs":{"total":773094113280,"available":772850544640,"internally_reserved":0,"allocated":5447680,"data_stored":3020856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":238012504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003321"},"pg_stats":[{"pgid":"1.0","version":"45'119","reported_seq":200,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-12T17:11:47.781215+0000","last_change":"2024-03-12T17:11:47.781427+0000","last_active":"2024-03-12T17:11:47.781215+0000","last_peered":"2024-03-12T17:11:47.781215+0000","last_clean":"2024-03-12T17:11:47.781215+0000","last_became_active":"2024-03-12T17:06:21.712488+0000","last_became_peered":"2024-03-12T17:06:21.712488+0000","last_unstale":"2024-03-12T17:11:47.781215+0000","last_undegraded":"2024-03-12T17:11:47.781215+0000","last_fullsized":"2024-03-12T17:11:47.781215+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'119","last_scrub_stamp":"2024-03-12T17:11:47.781137+0000","last_deep_scrub":"45'119","last_deep_scrub_stamp":"2024-03-12T17:11:47.781137+0000","last_clean_scrub_stamp":"2024-03-12T17:11:47.781137+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T00:49:50.962537+0000","scrub_duration":0.0092373569999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574920,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1126400,"data_stored":820383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.1200000000000001,"15min":1.0389999999999999},"min":{"1min":0.68500000000000005,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.8080000000000001,"5min":13.563000000000001,"15min":13.563000000000001},"last":1.044},{"interface":"front","average":{"1min":1.004,"5min":1.1040000000000001,"15min":1.0740000000000001},"min":{"1min":0.68100000000000005,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.776,"5min":6.6779999999999999,"15min":6.6779999999999999},"last":1.171}]},{"osd":1,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.107,"15min":1.1519999999999999},"min":{"1min":0.79800000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.488,"5min":2.9990000000000001,"15min":2.9990000000000001},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.0429999999999999},"min":{"1min":0.748,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.5860000000000001,"5min":5.181,"15min":5.181},"last":1.089}]},{"osd":2,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.167,"15min":1.1359999999999999},"min":{"1min":0.754,"5min":0.748,"15min":0.748},"max":{"1min":1.5189999999999999,"5min":5.0819999999999999,"15min":5.0819999999999999},"last":1.409},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.2250000000000001,"15min":1.21},"min":{"1min":0.76300000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.425,"5min":6.5289999999999999,"15min":6.5289999999999999},"last":1.0740000000000001}]},{"osd":3,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.1339999999999999,"15min":1.1879999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.5149999999999999,"5min":3.9849999999999999,"15min":3.9849999999999999},"last":1.204},{"interface":"front","average":{"1min":1.264,"5min":1.258,"15min":1.26},"min":{"1min":0.98099999999999998,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.657,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":1.2589999999999999}]},{"osd":4,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.234,"15min":1.268},"min":{"1min":0.98299999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.8740000000000001,"5min":3.8580000000000001,"15min":3.8580000000000001},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.4099999999999999,"15min":1.407},"min":{"1min":1.093,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.5900000000000001,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":1.375}]},{"osd":5,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.246,"15min":1.264},"min":{"1min":0.97599999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.5469999999999999,"5min":3.3799999999999999,"15min":3.3799999999999999},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.337,"5min":1.341,"15min":1.3600000000000001},"min":{"1min":0.98399999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.677,"5min":4.0750000000000002,"15min":4.0750000000000002},"last":1.29}]},{"osd":6,"last update":"Tue Mar 12 17:11:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.375,"15min":1.3919999999999999},"min":{"1min":1.01,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.599,"5min":3.7599999999999998,"15min":3.7599999999999998},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4690000000000001,"15min":1.53},"min":{"1min":0.90200000000000002,"5min":0.626,"15min":0.626},"max":{"1min":1.927,"5min":4.7560000000000002,"15min":4.7560000000000002},"last":1.129}]}]},{"osd":6,"up_from":13,"seq":55834574920,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":230015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.0409999999999999,"15min":1.0680000000000001},"min":{"1min":0.61799999999999999,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5600000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.073,"5min":1.0449999999999999,"15min":0.97199999999999998},"min":{"1min":0.70499999999999996,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.492,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":1.216}]},{"osd":1,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":1.026,"15min":0.999},"min":{"1min":0.66700000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.2909999999999999,"5min":1.74,"15min":1.74},"last":1.071},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.1259999999999999,"15min":1.1419999999999999},"min":{"1min":0.68300000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.486,"5min":1.73,"15min":1.73},"last":1.288}]},{"osd":2,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.222,"15min":1.1899999999999999},"min":{"1min":0.79300000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.355,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.125,"5min":1.143,"15min":1.165},"min":{"1min":0.74399999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5920000000000001,"5min":1.764,"15min":1.764},"last":1.3839999999999999}]},{"osd":3,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.25,"15min":1.268},"min":{"1min":0.83999999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":1.321},{"interface":"front","average":{"1min":1.163,"5min":1.236,"15min":1.2470000000000001},"min":{"1min":0.89800000000000002,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.556,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.161}]},{"osd":4,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.3440000000000001,"15min":1.3100000000000001},"min":{"1min":0.89700000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.7390000000000001,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.319,"5min":1.3400000000000001,"15min":1.363},"min":{"1min":0.93500000000000005,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.554,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.022}]},{"osd":5,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.3140000000000001,"15min":1.3260000000000001},"min":{"1min":0.89100000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.7,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.377,"5min":1.4590000000000001,"15min":1.4410000000000001},"min":{"1min":1.1279999999999999,"5min":0.5,"15min":0.5},"max":{"1min":1.6419999999999999,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":0.98199999999999998}]},{"osd":7,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.5469999999999999,"15min":1.458},"min":{"1min":0.96499999999999997,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.766,"5min":9.9610000000000003,"15min":9.9610000000000003},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.4610000000000001,"15min":1.464},"min":{"1min":0.98599999999999999,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.79,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.2470000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574919,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":230015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":1.0269999999999999,"15min":1.1160000000000001},"min":{"1min":0.71599999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2,"5min":3.6709999999999998,"15min":3.6709999999999998},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.995,"5min":1.069,"15min":1.1619999999999999},"min":{"1min":0.625,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.359,"5min":3.706,"15min":3.706},"last":1.4059999999999999}]},{"osd":1,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.0600000000000001,"15min":1.077},"min":{"1min":0.73399999999999999,"5min":0.442,"15min":0.442},"max":{"1min":1.3089999999999999,"5min":3.0390000000000001,"15min":3.0390000000000001},"last":1.29},{"interface":"front","average":{"1min":1.165,"5min":1.161,"15min":1.1699999999999999},"min":{"1min":0.81299999999999994,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.407,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.3620000000000001}]},{"osd":2,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.175,"15min":1.1919999999999999},"min":{"1min":0.82599999999999996,"5min":0.498,"15min":0.498},"max":{"1min":1.6499999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.012},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.1990000000000001,"15min":1.242},"min":{"1min":0.879,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4610000000000001,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.4530000000000001}]},{"osd":3,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2709999999999999,"15min":1.349},"min":{"1min":0.82499999999999996,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.478,"5min":3.7440000000000002,"15min":3.7440000000000002},"last":1.256},{"interface":"front","average":{"1min":1.262,"5min":1.2829999999999999,"15min":1.29},"min":{"1min":0.91600000000000004,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.5920000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":1.3149999999999999}]},{"osd":4,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2569999999999999,"15min":1.2410000000000001},"min":{"1min":0.63200000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.5580000000000001,"5min":1.873,"15min":1.873},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3280000000000001,"15min":1.294},"min":{"1min":0.67200000000000004,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.6040000000000001,"5min":2,"15min":2},"last":1.1319999999999999}]},{"osd":6,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.4319999999999999,"15min":1.599},"min":{"1min":0.89300000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.7390000000000001,"5min":8.2080000000000002,"15min":8.2080000000000002},"last":1.484},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4419999999999999,"15min":1.4490000000000001},"min":{"1min":0.70299999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.069}]},{"osd":7,"last update":"Tue Mar 12 17:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.4850000000000001,"5min":1.4630000000000001,"15min":1.417},"min":{"1min":0.79800000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.7130000000000001,"5min":2.173,"15min":2.173},"last":1.504},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.575,"15min":1.764},"min":{"1min":1.0209999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":2.1339999999999999,"5min":8.0139999999999993,"15min":8.0139999999999993},"last":1.23}]}]},{"osd":4,"up_from":13,"seq":55834574922,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":230015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.1020000000000001,"15min":1.1739999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.476,"5min":1.655,"15min":1.655},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.119,"15min":1.1879999999999999},"min":{"1min":0.73399999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.446,"5min":1.724,"15min":1.724},"last":1.3280000000000001}]},{"osd":1,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.0760000000000001,"15min":1.0149999999999999},"min":{"1min":0.81000000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.504,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.238},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.1479999999999999,"15min":1.1919999999999999},"min":{"1min":0.78100000000000003,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.47,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.123}]},{"osd":2,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.1639999999999999,"15min":1.204},"min":{"1min":0.80900000000000005,"5min":0.752,"15min":0.752},"max":{"1min":1.899,"5min":1.982,"15min":1.982},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.224,"5min":1.1950000000000001,"15min":1.256},"min":{"1min":0.88600000000000001,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.546,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.2869999999999999}]},{"osd":3,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.161,"15min":1.171},"min":{"1min":0.748,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.6000000000000001,"5min":1.661,"15min":1.661},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.3260000000000001,"15min":1.3979999999999999},"min":{"1min":0.90500000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.821,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.5149999999999999}]},{"osd":5,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.3280000000000001,"15min":1.3520000000000001},"min":{"1min":0.94799999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.5389999999999999,"5min":2.2589999999999999,"15min":2.2589999999999999},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.393,"5min":1.3220000000000001,"15min":1.321},"min":{"1min":0.83399999999999996,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.6499999999999999,"5min":1.95,"15min":1.95},"last":1.5880000000000001}]},{"osd":6,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.3979999999999999,"15min":1.389},"min":{"1min":0.94299999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.9470000000000001,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.431,"5min":1.462,"15min":1.512},"min":{"1min":1.093,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.675,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":1.5329999999999999}]},{"osd":7,"last update":"Tue Mar 12 17:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.488,"5min":1.4279999999999999,"15min":1.4259999999999999},"min":{"1min":1.01,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.752,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":1.825},{"interface":"front","average":{"1min":1.52,"5min":1.5469999999999999,"15min":1.615},"min":{"1min":1.1919999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":2.0779999999999998,"5min":3.5720000000000001,"15min":3.5720000000000001},"last":1.5640000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574919,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31640,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":532480,"data_stored":230015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:11:33 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.92600000000000005,"15min":1.1859999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.369,"5min":5.0039999999999996,"15min":5.0039999999999996},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.97599999999999998,"5min":1.0389999999999999,"15min":1.238},"min":{"1min":0.69699999999999995,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.375,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":1.0680000000000001}]},{"osd":1,"last update":"Tue Mar 12 17:11:33 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.96899999999999997,"15min":1.2390000000000001},"min":{"1min":0.53000000000000003,"5min":0.377,"15min":0.377},"max":{"1min":1.2350000000000001,"5min":4.9080000000000004,"15min":4.9080000000000004},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.98199999999999998,"15min":1.1739999999999999},"min":{"1min":0.64500000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.444,"5min":3.2410000000000001,"15min":3.2410000000000001},"last":1.196}]},{"osd":2,"last update":"Tue Mar 12 17:11:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0700000000000001,"15min":1.3129999999999999},"min":{"1min":0.71799999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.4590000000000001,"5min":3.4990000000000001,"15min":3.4990000000000001},"last":1.131},{"interface":"front","average":{"1min":1.107,"5min":1.163,"15min":1.4550000000000001},"min":{"1min":0.67100000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.6639999999999999,"5min":5.1870000000000003,"15min":5.1870000000000003},"last":1.5680000000000001}]},{"osd":4,"last update":"Tue Mar 12 17:11:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.2030000000000001,"15min":1.5289999999999999},"min":{"1min":0.78000000000000003,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.5009999999999999,"5min":9.9299999999999997,"15min":9.9299999999999997},"last":1.371},{"interface":"front","average":{"1min":1.321,"5min":1.345,"15min":1.573},"min":{"1min":0.91300000000000003,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.8400000000000001,"5min":5.2450000000000001,"15min":5.2450000000000001},"last":1.2669999999999999}]},{"osd":5,"last update":"Tue Mar 12 17:11:33 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.1850000000000001,"15min":1.3069999999999999},"min":{"1min":0.88100000000000001,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.5700000000000001,"5min":2.9550000000000001,"15min":2.9550000000000001},"last":1.8089999999999999},{"interface":"front","average":{"1min":1.224,"5min":1.319,"15min":1.4990000000000001},"min":{"1min":0.875,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.9239999999999999,"5min":3.089,"15min":3.089},"last":1.321}]},{"osd":6,"last update":"Tue Mar 12 17:11:33 2024","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3660000000000001,"15min":1.5580000000000001},"min":{"1min":0.96199999999999997,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":2.0049999999999999,"5min":3.3170000000000002,"15min":3.3170000000000002},"last":1.6359999999999999},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2569999999999999,"15min":1.401},"min":{"1min":0.85399999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.7509999999999999,"5min":2.722,"15min":2.722},"last":1.603}]},{"osd":7,"last update":"Tue Mar 12 17:11:33 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.3,"15min":1.474},"min":{"1min":0.90400000000000003,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.8200000000000001,"5min":2.7559999999999998,"15min":2.7559999999999998},"last":1.7090000000000001},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.395,"15min":1.5489999999999999},"min":{"1min":0.96699999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2.032,"5min":3.1389999999999998,"15min":3.1389999999999998},"last":1.5309999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574919,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31636,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340204,"statfs":{"total":96636764160,"available":96604368896,"internally_reserved":0,"allocated":532480,"data_stored":230015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.78100000000000003,"15min":0.81499999999999995},"min":{"1min":0.498,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0780000000000001,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.751},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.88900000000000001,"15min":1.0149999999999999},"min":{"1min":0.56200000000000006,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.234,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":1.2629999999999999}]},{"osd":1,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.84399999999999997,"15min":0.91700000000000004},"min":{"1min":0.49099999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.2370000000000001,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.92200000000000004,"15min":0.94699999999999995},"min":{"1min":0.61199999999999999,"5min":0.439,"15min":0.439},"max":{"1min":1.573,"5min":1.732,"15min":1.732},"last":0.92900000000000005}]},{"osd":3,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.01,"15min":1.0229999999999999},"min":{"1min":0.56799999999999995,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.611,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.111},{"interface":"front","average":{"1min":0.998,"5min":0.95899999999999996,"15min":1.0249999999999999},"min":{"1min":0.57899999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.859,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.321}]},{"osd":4,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.23,"15min":1.25},"min":{"1min":0.84799999999999998,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.9730000000000001,"5min":2.9569999999999999,"15min":2.9569999999999999},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3029999999999999,"15min":1.407},"min":{"1min":0.91700000000000004,"5min":0.751,"15min":0.751},"max":{"1min":1.7390000000000001,"5min":4.4640000000000004,"15min":4.4640000000000004},"last":1.609}]},{"osd":5,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.2889999999999999,"15min":1.359},"min":{"1min":0.999,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5900000000000001,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.7150000000000001},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3049999999999999,"15min":1.353},"min":{"1min":0.90600000000000003,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.077,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":1.677}]},{"osd":6,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.3109999999999999,"15min":1.4379999999999999},"min":{"1min":0.89200000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.663,"5min":3.5179999999999998,"15min":3.5179999999999998},"last":1.768},{"interface":"front","average":{"1min":1.341,"5min":1.3560000000000001,"15min":1.446},"min":{"1min":1.028,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.7869999999999999,"5min":2.464,"15min":2.464},"last":1.4330000000000001}]},{"osd":7,"last update":"Tue Mar 12 17:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.391,"15min":1.526},"min":{"1min":0.98499999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.7350000000000001,"5min":3.6110000000000002,"15min":3.6110000000000002},"last":1.859},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.421,"15min":1.5209999999999999},"min":{"1min":0.872,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.004,"5min":2.8730000000000002,"15min":2.8730000000000002},"last":1.8260000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31636,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340204,"statfs":{"total":96636764160,"available":96604368896,"internally_reserved":0,"allocated":532480,"data_stored":230015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.85999999999999999,"15min":0.90300000000000002},"min":{"1min":0.54800000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.222,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.90200000000000002,"15min":1.034},"min":{"1min":0.56100000000000005,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.2729999999999999,"5min":3.6419999999999999,"15min":3.6419999999999999},"last":1.1699999999999999}]},{"osd":2,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.94999999999999996,"15min":1.052},"min":{"1min":0.57399999999999995,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4610000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.97099999999999997,"15min":1.1399999999999999},"min":{"1min":0.63300000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1879999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.996}]},{"osd":3,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.0960000000000001,"15min":1.3200000000000001},"min":{"1min":0.69699999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.331,"5min":3.9580000000000002,"15min":3.9580000000000002},"last":1.256},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.93000000000000005,"15min":1.04},"min":{"1min":0.52100000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.159,"5min":1.8,"15min":1.8},"last":1.1259999999999999}]},{"osd":4,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.2110000000000001,"15min":1.331},"min":{"1min":0.85199999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.5649999999999999,"5min":2.694,"15min":2.694},"last":1.306},{"interface":"front","average":{"1min":1.238,"5min":1.2789999999999999,"15min":1.403},"min":{"1min":0.89100000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.8999999999999999,"5min":3.5150000000000001,"15min":3.5150000000000001},"last":1.349}]},{"osd":5,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.1819999999999999,"15min":1.25},"min":{"1min":0.78300000000000003,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.7470000000000001,"5min":2.109,"15min":2.109},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.337,"15min":1.4690000000000001},"min":{"1min":0.95699999999999996,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.677,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":1.4059999999999999}]},{"osd":6,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3680000000000001,"15min":1.514},"min":{"1min":1.034,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.8240000000000001,"5min":4.1269999999999998,"15min":4.1269999999999998},"last":1.492},{"interface":"front","average":{"1min":1.244,"5min":1.272,"15min":1.417},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.575,"5min":3.7850000000000001,"15min":3.7850000000000001},"last":1.2829999999999999}]},{"osd":7,"last update":"Tue Mar 12 17:11:25 2024","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3680000000000001,"15min":1.5049999999999999},"min":{"1min":0.91100000000000003,"5min":0.873,"15min":0.873},"max":{"1min":1.7689999999999999,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.458},{"interface":"front","average":{"1min":1.377,"5min":1.4139999999999999,"15min":1.5529999999999999},"min":{"1min":1.075,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.74,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.518}]}]},{"osd":0,"up_from":13,"seq":55834574919,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1126400,"data_stored":820383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.79800000000000004,"15min":0.91600000000000004},"min":{"1min":0.40400000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.1819999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.877,"15min":0.91400000000000003},"min":{"1min":0.438,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.3839999999999999,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.373}]},{"osd":2,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":0.95199999999999996,"15min":1.052},"min":{"1min":0.58499999999999996,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.9410000000000001,"5min":1.952,"15min":1.952},"last":1.429},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.98199999999999998,"15min":1.1060000000000001},"min":{"1min":0.55300000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.571,"5min":1.915,"15min":1.915},"last":1.0589999999999999}]},{"osd":3,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.94699999999999995,"15min":1.0029999999999999},"min":{"1min":0.53200000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":2.2189999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.071,"5min":1.0940000000000001,"15min":1.242},"min":{"1min":0.63,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.1000000000000001,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.3160000000000001}]},{"osd":4,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.216,"15min":1.3400000000000001},"min":{"1min":0.84799999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.4209999999999998,"5min":3.0499999999999998,"15min":3.0499999999999998},"last":1.141},{"interface":"front","average":{"1min":1.298,"5min":1.337,"15min":1.474},"min":{"1min":0.88500000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":2.3730000000000002,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":1.5169999999999999}]},{"osd":5,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.347,"15min":1.4650000000000001},"min":{"1min":0.874,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.859,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.3140000000000001,"15min":1.3939999999999999},"min":{"1min":0.96099999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.9990000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.276}]},{"osd":6,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.361,"15min":1.5329999999999999},"min":{"1min":0.83099999999999996,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.2719999999999998,"5min":2.597,"15min":2.597},"last":1.665},{"interface":"front","average":{"1min":1.234,"5min":1.3420000000000001,"15min":1.4990000000000001},"min":{"1min":0.92200000000000004,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.6850000000000001,"5min":2.452,"15min":2.452},"last":1.238}]},{"osd":7,"last update":"Tue Mar 12 17:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.3959999999999999,"15min":1.5529999999999999},"min":{"1min":0.86799999999999999,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.3180000000000001,"5min":2.7240000000000002,"15min":2.7240000000000002},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.321,"15min":1.518},"min":{"1min":0.86099999999999999,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.4670000000000001,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":1.2210000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-12T17:12:08.276 DEBUG:teuthology.orchestra.run.smithi052:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-12T17:12:08.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.487+0000 7f96c7a00640 1 Processor -- start 2024-03-12T17:12:08.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.488+0000 7f96c7a00640 1 -- start start 2024-03-12T17:12:08.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.490+0000 7f96c7a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f96c01456a0 0x7f96c00ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:12:08.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.490+0000 7f96c7a00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f96c0070cd0 0x7f96c00cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:12:08.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.490+0000 7f96c7a00640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96c006d390 con 0x7f96c0145c20 2024-03-12T17:12:08.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.490+0000 7f96c7a00640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96c00ad890 con 0x7f96c0070cd0 2024-03-12T17:12:08.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.490+0000 7f96c7a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96c0145a80 con 0x7f96c01456a0 2024-03-12T17:12:08.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.490+0000 7f96bfe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f96c0070cd0 0x7f96c00cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:12:08.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.490+0000 7f96c5a00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f96c0145c20 0x7f96c00ab8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:35008/0 (socket says 172.21.15.52:35008) 2024-03-12T17:12:08.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.490+0000 7f96bfe00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f96c0070cd0 0x7f96c00cee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33338/0 (socket says 172.21.15.52:33338) 2024-03-12T17:12:08.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.490+0000 7f96c5a00640 1 -- 172.21.15.52:0/4067383982 learned_addr learned my addr 172.21.15.52:0/4067383982 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:12:08.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.491+0000 7f96c5000640 1 --2- 172.21.15.52:0/4067383982 >> v2:172.21.15.102:3300/0 conn(0x7f96c01456a0 0x7f96c00ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:12:08.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.491+0000 7f96bfe00640 1 -- 172.21.15.52:0/4067383982 >> v2:172.21.15.102:3300/0 conn(0x7f96c01456a0 msgr2=0x7f96c00ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:12:08.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.491+0000 7f96bfe00640 1 --2- 172.21.15.52:0/4067383982 >> v2:172.21.15.102:3300/0 conn(0x7f96c01456a0 0x7f96c00ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:12:08.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.491+0000 7f96bfe00640 1 -- 172.21.15.52:0/4067383982 >> v1:172.21.15.52:6789/0 conn(0x7f96c0145c20 legacy=0x7f96c00ab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:12:08.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.491+0000 7f96bfe00640 1 -- 172.21.15.52:0/4067383982 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c006d670 con 0x7f96c0070cd0 2024-03-12T17:12:08.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.491+0000 7f96bf400640 1 -- 172.21.15.52:0/4067383982 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 749845057 0 0) 0x7f96c006d390 con 0x7f96c0145c20 2024-03-12T17:12:08.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.492+0000 7f96bfe00640 1 --2- 172.21.15.52:0/4067383982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96c0070cd0 0x7f96c00cee70 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f96c00cf4f0 tx=0x7f96a802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8237739b04320ceb server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:12:08.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.492+0000 7f96bf400640 1 -- 172.21.15.52:0/4067383982 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a8031830 con 0x7f96c0070cd0 2024-03-12T17:12:08.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.492+0000 7f96bf400640 1 -- 172.21.15.52:0/4067383982 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a8031990 con 0x7f96c0070cd0 2024-03-12T17:12:08.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.492+0000 7f96bf400640 1 -- 172.21.15.52:0/4067383982 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a8031d20 con 0x7f96c0070cd0 2024-03-12T17:12:08.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.493+0000 7f96c7a00640 1 -- 172.21.15.52:0/4067383982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96c0070cd0 msgr2=0x7f96c00cee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:12:08.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.493+0000 7f96c7a00640 1 --2- 172.21.15.52:0/4067383982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96c0070cd0 0x7f96c00cee70 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f96c00cf4f0 tx=0x7f96a802b8d0 comp rx=0 tx=0).stop 2024-03-12T17:12:08.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.493+0000 7f96c7a00640 1 -- 172.21.15.52:0/4067383982 shutdown_connections 2024-03-12T17:12:08.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.493+0000 7f96c7a00640 1 --2- 172.21.15.52:0/4067383982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96c0070cd0 0x7f96c00cee70 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:12:08.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.493+0000 7f96c7a00640 1 --2- 172.21.15.52:0/4067383982 >> v2:172.21.15.102:3300/0 conn(0x7f96c01456a0 0x7f96c00ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:12:08.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.493+0000 7f96c7a00640 1 -- 172.21.15.52:0/4067383982 >> 172.21.15.52:0/4067383982 conn(0x7f96c006a0b0 msgr2=0x7f96c006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:12:08.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.493+0000 7f96c7a00640 1 -- 172.21.15.52:0/4067383982 shutdown_connections 2024-03-12T17:12:08.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.493+0000 7f96c7a00640 1 -- 172.21.15.52:0/4067383982 wait complete. 2024-03-12T17:12:08.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.495+0000 7f96c7a00640 1 Processor -- start 2024-03-12T17:12:08.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.495+0000 7f96c7a00640 1 -- start start 2024-03-12T17:12:08.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.496+0000 7f96c7a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96c0070cd0 0x7f96c00aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:12:08.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.496+0000 7f96c7a00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96c01456a0 0x7f96c013d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:12:08.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.496+0000 7f96c7a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96c0145c20 0x7f96c015e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:12:08.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.496+0000 7f96c7a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c015a320 con 0x7f96c0145c20 2024-03-12T17:12:08.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.496+0000 7f96c7a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c00af210 con 0x7f96c0070cd0 2024-03-12T17:12:08.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.496+0000 7f96c7a00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c013d610 con 0x7f96c01456a0 2024-03-12T17:12:08.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.496+0000 7f96c5a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96c0145c20 0x7f96c015e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:12:08.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.496+0000 7f96c5a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96c0145c20 0x7f96c015e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33352/0 (socket says 172.21.15.52:33352) 2024-03-12T17:12:08.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.496+0000 7f96c5a00640 1 -- 172.21.15.52:0/3984178230 learned_addr learned my addr 172.21.15.52:0/3984178230 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:12:08.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.497+0000 7f96bfe00640 1 --2- 172.21.15.52:0/3984178230 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96c01456a0 0x7f96c013d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:12:08.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.497+0000 7f96c5000640 1 --2- 172.21.15.52:0/3984178230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96c0070cd0 0x7f96c00aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:12:08.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.497+0000 7f96c5a00640 1 -- 172.21.15.52:0/3984178230 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96c01456a0 msgr2=0x7f96c013d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:12:08.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.497+0000 7f96c5a00640 1 --2- 172.21.15.52:0/3984178230 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96c01456a0 0x7f96c013d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:12:08.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.497+0000 7f96c5a00640 1 -- 172.21.15.52:0/3984178230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96c0070cd0 msgr2=0x7f96c00aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:12:08.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.497+0000 7f96c5a00640 1 --2- 172.21.15.52:0/3984178230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96c0070cd0 0x7f96c00aecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:12:08.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.497+0000 7f96c5a00640 1 -- 172.21.15.52:0/3984178230 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a8009400 con 0x7f96c0145c20 2024-03-12T17:12:08.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.497+0000 7f96bfe00640 1 --2- 172.21.15.52:0/3984178230 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96c01456a0 0x7f96c013d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:12:08.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.497+0000 7f96c5a00640 1 --2- 172.21.15.52:0/3984178230 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96c0145c20 0x7f96c015e050 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f96b8002230 tx=0x7f96b800cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:12:08.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.497+0000 7f96bcc00640 1 -- 172.21.15.52:0/3984178230 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b800ac30 con 0x7f96c0145c20 2024-03-12T17:12:08.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.497+0000 7f96bcc00640 1 -- 172.21.15.52:0/3984178230 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b800ad90 con 0x7f96c0145c20 2024-03-12T17:12:08.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.497+0000 7f96bcc00640 1 -- 172.21.15.52:0/3984178230 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b8012620 con 0x7f96c0145c20 2024-03-12T17:12:08.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.498+0000 7f96c7a00640 1 -- 172.21.15.52:0/3984178230 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c00afb80 con 0x7f96c0145c20 2024-03-12T17:12:08.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.498+0000 7f96c7a00640 1 -- 172.21.15.52:0/3984178230 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c00b0140 con 0x7f96c0145c20 2024-03-12T17:12:08.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.499+0000 7f96c7a00640 1 -- 172.21.15.52:0/3984178230 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96c015a460 con 0x7f96c0145c20 2024-03-12T17:12:08.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.500+0000 7f96bcc00640 1 -- 172.21.15.52:0/3984178230 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f96b8007070 con 0x7f96c0145c20 2024-03-12T17:12:08.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.501+0000 7f96bcc00640 1 --2- 172.21.15.52:0/3984178230 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f969c06ac00 0x7f969c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:12:08.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.501+0000 7f96bcc00640 1 -- 172.21.15.52:0/3984178230 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f96b8089110 con 0x7f96c0145c20 2024-03-12T17:12:08.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.501+0000 7f96c5000640 1 --2- 172.21.15.52:0/3984178230 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f969c06ac00 0x7f969c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:12:08.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.503+0000 7f96c5000640 1 --2- 172.21.15.52:0/3984178230 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f969c06ac00 0x7f969c06d0c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f96b0006fd0 tx=0x7f96b0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:12:08.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.505+0000 7f96bcc00640 1 -- 172.21.15.52:0/3984178230 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f96b80544a0 con 0x7f96c0145c20 2024-03-12T17:12:08.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.707+0000 7f96c7a00640 1 -- 172.21.15.52:0/3984178230 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f96c0073390 con 0x7f96c0145c20 2024-03-12T17:12:08.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.715+0000 7f96bcc00640 1 -- 172.21.15.52:0/3984178230 <== mon.0 v2:172.21.15.52:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f96b8001b50 con 0x7f96c0145c20 2024-03-12T17:12:08.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.715+0000 7f96bcc00640 1 -- 172.21.15.52:0/3984178230 <== mon.0 v2:172.21.15.52:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) v1 ==== 123+0+0 (secure 0 0 0) 0x7f96b8057ac0 con 0x7f96c0145c20 2024-03-12T17:12:08.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.719+0000 7f96c7a00640 1 -- 172.21.15.52:0/3984178230 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f969c06ac00 msgr2=0x7f969c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:12:08.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.719+0000 7f96c7a00640 1 --2- 172.21.15.52:0/3984178230 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f969c06ac00 0x7f969c06d0c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f96b0006fd0 tx=0x7f96b0008040 comp rx=0 tx=0).stop 2024-03-12T17:12:08.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.719+0000 7f96c7a00640 1 -- 172.21.15.52:0/3984178230 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96c0145c20 msgr2=0x7f96c015e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:12:08.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.719+0000 7f96c7a00640 1 --2- 172.21.15.52:0/3984178230 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96c0145c20 0x7f96c015e050 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f96b8002230 tx=0x7f96b800cf50 comp rx=0 tx=0).stop 2024-03-12T17:12:08.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.719+0000 7f96c7a00640 1 -- 172.21.15.52:0/3984178230 shutdown_connections 2024-03-12T17:12:08.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.719+0000 7f96c7a00640 1 --2- 172.21.15.52:0/3984178230 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7f969c06ac00 0x7f969c06d0c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:12:08.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.719+0000 7f96c7a00640 1 --2- 172.21.15.52:0/3984178230 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96c01456a0 0x7f96c013d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:12:08.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.719+0000 7f96c7a00640 1 --2- 172.21.15.52:0/3984178230 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96c0145c20 0x7f96c015e050 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:12:08.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.719+0000 7f96c7a00640 1 --2- 172.21.15.52:0/3984178230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96c0070cd0 0x7f96c00aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:12:08.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.719+0000 7f96c7a00640 1 -- 172.21.15.52:0/3984178230 >> 172.21.15.52:0/3984178230 conn(0x7f96c006a0b0 msgr2=0x7f96c00ce900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:12:08.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.720+0000 7f96c7a00640 1 -- 172.21.15.52:0/3984178230 shutdown_connections 2024-03-12T17:12:08.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.720+0000 7f96c7a00640 1 -- 172.21.15.52:0/3984178230 wait complete. 2024-03-12T17:12:08.733 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-12T17:12:08.734 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-12T17:12:08.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.958+0000 7fd31e800640 1 Processor -- start 2024-03-12T17:12:08.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.958+0000 7fd31e800640 1 -- start start 2024-03-12T17:12:08.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.959+0000 7fd31e800640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd318143460 0x7fd31815aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:12:08.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.959+0000 7fd31e800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd3180cde20 0x7fd318070020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:12:08.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.959+0000 7fd31e800640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3180cc6b0 con 0x7fd318143930 2024-03-12T17:12:08.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.959+0000 7fd31e800640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3180ce270 con 0x7fd318143460 2024-03-12T17:12:08.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.959+0000 7fd31e800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3180cc950 con 0x7fd3180cde20 2024-03-12T17:12:08.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.959+0000 7fd31ca00640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fd318143930 0x7fd3180cd710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:35010/0 (socket says 172.21.15.52:35010) 2024-03-12T17:12:08.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.959+0000 7fd31ca00640 1 -- 172.21.15.52:0/1752703018 learned_addr learned my addr 172.21.15.52:0/1752703018 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-03-12T17:12:08.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.959+0000 7fd31d400640 1 --2- 172.21.15.52:0/1752703018 >> v2:172.21.15.52:3300/0 conn(0x7fd318143460 0x7fd31815aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:12:08.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.960+0000 7fd31de00640 1 --2- 172.21.15.52:0/1752703018 >> v2:172.21.15.102:3300/0 conn(0x7fd3180cde20 0x7fd318070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:12:08.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.960+0000 7fd31d400640 1 -- 172.21.15.52:0/1752703018 >> v2:172.21.15.102:3300/0 conn(0x7fd3180cde20 msgr2=0x7fd318070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-12T17:12:08.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.960+0000 7fd31d400640 1 --2- 172.21.15.52:0/1752703018 >> v2:172.21.15.102:3300/0 conn(0x7fd3180cde20 0x7fd318070020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:12:08.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.960+0000 7fd31d400640 1 -- 172.21.15.52:0/1752703018 >> v1:172.21.15.52:6789/0 conn(0x7fd318143930 legacy=0x7fd3180cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:12:08.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.960+0000 7fd317e00640 1 -- 172.21.15.52:0/1752703018 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 328495671 0 0) 0x7fd3180cc6b0 con 0x7fd318143930 2024-03-12T17:12:08.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.960+0000 7fd31d400640 1 -- 172.21.15.52:0/1752703018 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3180ccc30 con 0x7fd318143460 2024-03-12T17:12:08.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.960+0000 7fd31de00640 1 --2- 172.21.15.52:0/1752703018 >> v2:172.21.15.102:3300/0 conn(0x7fd3180cde20 0x7fd318070020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-12T17:12:08.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.961+0000 7fd31d400640 1 --2- 172.21.15.52:0/1752703018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd318143460 0x7fd31815aba0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd3180b1a50 tx=0x7fd30802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a12470dc3a5c4fa server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:12:08.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.962+0000 7fd317e00640 1 -- 172.21.15.52:0/1752703018 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd308030610 con 0x7fd318143460 2024-03-12T17:12:08.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.962+0000 7fd317e00640 1 -- 172.21.15.52:0/1752703018 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fd308030770 con 0x7fd318143460 2024-03-12T17:12:08.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.962+0000 7fd317e00640 1 -- 172.21.15.52:0/1752703018 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd308030b20 con 0x7fd318143460 2024-03-12T17:12:08.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.962+0000 7fd31e800640 1 -- 172.21.15.52:0/1752703018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd318143460 msgr2=0x7fd31815aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:12:08.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.962+0000 7fd31e800640 1 --2- 172.21.15.52:0/1752703018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd318143460 0x7fd31815aba0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd3180b1a50 tx=0x7fd30802b960 comp rx=0 tx=0).stop 2024-03-12T17:12:08.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.962+0000 7fd31e800640 1 -- 172.21.15.52:0/1752703018 shutdown_connections 2024-03-12T17:12:08.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.962+0000 7fd31e800640 1 --2- 172.21.15.52:0/1752703018 >> v2:172.21.15.102:3300/0 conn(0x7fd3180cde20 0x7fd318070020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:12:08.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.962+0000 7fd31e800640 1 --2- 172.21.15.52:0/1752703018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd318143460 0x7fd31815aba0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:12:08.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.962+0000 7fd31e800640 1 -- 172.21.15.52:0/1752703018 >> 172.21.15.52:0/1752703018 conn(0x7fd318065df0 msgr2=0x7fd318069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:12:08.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.962+0000 7fd31e800640 1 -- 172.21.15.52:0/1752703018 shutdown_connections 2024-03-12T17:12:08.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.962+0000 7fd31e800640 1 -- 172.21.15.52:0/1752703018 wait complete. 2024-03-12T17:12:08.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.964+0000 7fd31e800640 1 Processor -- start 2024-03-12T17:12:08.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.964+0000 7fd31e800640 1 -- start start 2024-03-12T17:12:08.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.965+0000 7fd31e800640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3180cde20 0x7fd3180ab920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:12:08.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.965+0000 7fd31e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd318143460 0x7fd318137b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:12:08.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.965+0000 7fd31e800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd318143930 0x7fd31813a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:12:08.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.965+0000 7fd31e800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3180abfb0 con 0x7fd318143930 2024-03-12T17:12:08.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.965+0000 7fd31e800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3180ac290 con 0x7fd318143460 2024-03-12T17:12:08.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.965+0000 7fd31e800640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3180ac570 con 0x7fd3180cde20 2024-03-12T17:12:08.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.966+0000 7fd31de00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd318143930 0x7fd31813a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:12:08.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.966+0000 7fd31de00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd318143930 0x7fd31813a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33372/0 (socket says 172.21.15.52:33372) 2024-03-12T17:12:08.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.966+0000 7fd31de00640 1 -- 172.21.15.52:0/2350346101 learned_addr learned my addr 172.21.15.52:0/2350346101 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-12T17:12:08.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.966+0000 7fd31d400640 1 --2- 172.21.15.52:0/2350346101 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3180cde20 0x7fd3180ab920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:12:08.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.966+0000 7fd31ca00640 1 --2- 172.21.15.52:0/2350346101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd318143460 0x7fd318137b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:12:08.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.966+0000 7fd31de00640 1 -- 172.21.15.52:0/2350346101 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3180cde20 msgr2=0x7fd3180ab920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:12:08.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.966+0000 7fd31de00640 1 --2- 172.21.15.52:0/2350346101 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3180cde20 0x7fd3180ab920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:12:08.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.966+0000 7fd31de00640 1 -- 172.21.15.52:0/2350346101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd318143460 msgr2=0x7fd318137b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:12:08.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.966+0000 7fd31de00640 1 --2- 172.21.15.52:0/2350346101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd318143460 0x7fd318137b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:12:08.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.966+0000 7fd31de00640 1 -- 172.21.15.52:0/2350346101 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd308009400 con 0x7fd318143930 2024-03-12T17:12:08.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.966+0000 7fd31de00640 1 --2- 172.21.15.52:0/2350346101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd318143930 0x7fd31813a900 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd31000c7f0 tx=0x7fd31000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:12:08.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.967+0000 7fd315600640 1 -- 172.21.15.52:0/2350346101 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd310007820 con 0x7fd318143930 2024-03-12T17:12:08.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.967+0000 7fd315600640 1 -- 172.21.15.52:0/2350346101 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fd310007980 con 0x7fd318143930 2024-03-12T17:12:08.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.967+0000 7fd315600640 1 -- 172.21.15.52:0/2350346101 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd310007d90 con 0x7fd318143930 2024-03-12T17:12:08.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.967+0000 7fd31e800640 1 -- 172.21.15.52:0/2350346101 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3180ac850 con 0x7fd318143930 2024-03-12T17:12:08.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.967+0000 7fd31e800640 1 -- 172.21.15.52:0/2350346101 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd31815a500 con 0x7fd318143930 2024-03-12T17:12:08.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.968+0000 7fd31e800640 1 -- 172.21.15.52:0/2350346101 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3180acd70 con 0x7fd318143930 2024-03-12T17:12:08.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.969+0000 7fd315600640 1 -- 172.21.15.52:0/2350346101 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd3100113f0 con 0x7fd318143930 2024-03-12T17:12:08.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.970+0000 7fd315600640 1 --2- 172.21.15.52:0/2350346101 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd2fc06ac00 0x7fd2fc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-12T17:12:08.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.970+0000 7fd315600640 1 -- 172.21.15.52:0/2350346101 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fd3100896c0 con 0x7fd318143930 2024-03-12T17:12:08.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.971+0000 7fd31d400640 1 --2- 172.21.15.52:0/2350346101 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd2fc06ac00 0x7fd2fc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-12T17:12:08.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.972+0000 7fd31d400640 1 --2- 172.21.15.52:0/2350346101 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd2fc06ac00 0x7fd2fc06d0c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd308036000 tx=0x7fd308008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-12T17:12:08.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:08.974+0000 7fd315600640 1 -- 172.21.15.52:0/2350346101 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fd3180acd70 con 0x7fd318143930 2024-03-12T17:12:09.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:09.213+0000 7fd31e800640 1 -- 172.21.15.52:0/2350346101 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3180acd70 con 0x7fd318143930 2024-03-12T17:12:09.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:09.214+0000 7fd315600640 1 -- 172.21.15.52:0/2350346101 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3529 (secure 0 0 0) 0x7fd3180acd70 con 0x7fd318143930 2024-03-12T17:12:09.209 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-12T17:12:09.209 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6836/285561331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":285561331},{"type":"v1","addr":"172.21.15.102:6836","nonce":285561331}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4909,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1312095655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6835","nonce":1312095655},{"type":"v1","addr":"172.21.15.102:6837","nonce":1312095655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5337,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2451989690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2451989690},{"type":"v1","addr":"172.21.15.52:6833","nonce":2451989690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5658,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/890410147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":890410147},{"type":"v1","addr":"172.21.15.52:6835","nonce":890410147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[]} 2024-03-12T17:12:09.210 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 19 2024-03-12T17:12:09.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:09.217+0000 7fd31e800640 1 -- 172.21.15.52:0/2350346101 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd2fc06ac00 msgr2=0x7fd2fc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:12:09.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:09.217+0000 7fd31e800640 1 --2- 172.21.15.52:0/2350346101 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd2fc06ac00 0x7fd2fc06d0c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd308036000 tx=0x7fd308008030 comp rx=0 tx=0).stop 2024-03-12T17:12:09.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:09.217+0000 7fd31e800640 1 -- 172.21.15.52:0/2350346101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd318143930 msgr2=0x7fd31813a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-12T17:12:09.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:09.217+0000 7fd31e800640 1 --2- 172.21.15.52:0/2350346101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd318143930 0x7fd31813a900 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd31000c7f0 tx=0x7fd31000ccb0 comp rx=0 tx=0).stop 2024-03-12T17:12:09.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:09.217+0000 7fd31e800640 1 -- 172.21.15.52:0/2350346101 shutdown_connections 2024-03-12T17:12:09.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:09.217+0000 7fd31e800640 1 --2- 172.21.15.52:0/2350346101 >> [v2:172.21.15.102:6832/36622,v1:172.21.15.102:6833/36622] conn(0x7fd2fc06ac00 0x7fd2fc06d0c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:12:09.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:09.217+0000 7fd31e800640 1 --2- 172.21.15.52:0/2350346101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd318143930 0x7fd31813a900 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:12:09.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:09.217+0000 7fd31e800640 1 --2- 172.21.15.52:0/2350346101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd318143460 0x7fd318137b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:12:09.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:09.217+0000 7fd31e800640 1 --2- 172.21.15.52:0/2350346101 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd3180cde20 0x7fd3180ab920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-12T17:12:09.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:09.217+0000 7fd31e800640 1 -- 172.21.15.52:0/2350346101 >> 172.21.15.52:0/2350346101 conn(0x7fd318065df0 msgr2=0x7fd31815bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-12T17:12:09.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:09.218+0000 7fd31e800640 1 -- 172.21.15.52:0/2350346101 shutdown_connections 2024-03-12T17:12:09.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-12T17:12:09.218+0000 7fd31e800640 1 -- 172.21.15.52:0/2350346101 wait complete. 2024-03-12T17:12:09.233 DEBUG:tasks.cephfs.filesystem:already dead... 2024-03-12T17:12:09.234 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-12T17:12:09.234 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-12T17:12:09.234 INFO:teuthology.orchestra.run:waiting for 300 2024-03-12T17:12:09.255 INFO:tasks.ceph.mds.a:Stopped 2024-03-12T17:12:09.256 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-12T17:12:09.256 INFO:teuthology.orchestra.run:waiting for 300 2024-03-12T17:12:09.318 INFO:tasks.ceph.mds.c:Stopped 2024-03-12T17:12:09.318 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-12T17:12:09.318 INFO:teuthology.orchestra.run:waiting for 300 2024-03-12T17:12:09.339 INFO:tasks.ceph.mds.b:Stopped 2024-03-12T17:12:09.339 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-12T17:12:09.339 INFO:teuthology.orchestra.run:waiting for 300 2024-03-12T17:12:09.413 INFO:tasks.ceph.mds.d:Stopped 2024-03-12T17:12:09.413 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-12T17:12:09.414 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-12T17:12:09.414 INFO:teuthology.orchestra.run:waiting for 300 2024-03-12T17:12:09.483 INFO:tasks.ceph.osd.0:Stopped 2024-03-12T17:12:09.483 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-12T17:12:09.483 INFO:teuthology.orchestra.run:waiting for 300 2024-03-12T17:12:09.539 INFO:tasks.ceph.osd.1:Stopped 2024-03-12T17:12:09.539 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-12T17:12:09.539 INFO:teuthology.orchestra.run:waiting for 300 2024-03-12T17:12:09.592 INFO:tasks.ceph.osd.2:Stopped 2024-03-12T17:12:09.592 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-12T17:12:09.592 INFO:teuthology.orchestra.run:waiting for 300 2024-03-12T17:12:09.665 INFO:tasks.ceph.osd.3:Stopped 2024-03-12T17:12:09.665 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-03-12T17:12:09.665 INFO:teuthology.orchestra.run:waiting for 300 2024-03-12T17:12:09.737 INFO:tasks.ceph.osd.4:Stopped 2024-03-12T17:12:09.737 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-03-12T17:12:09.737 INFO:teuthology.orchestra.run:waiting for 300 2024-03-12T17:12:09.787 INFO:tasks.ceph.osd.5:Stopped 2024-03-12T17:12:09.787 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-03-12T17:12:09.787 INFO:teuthology.orchestra.run:waiting for 300 2024-03-12T17:12:09.833 INFO:tasks.ceph.osd.6:Stopped 2024-03-12T17:12:09.833 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-03-12T17:12:09.833 INFO:teuthology.orchestra.run:waiting for 300 2024-03-12T17:12:09.880 INFO:tasks.ceph.osd.7:Stopped 2024-03-12T17:12:09.880 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-12T17:12:09.880 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-03-12T17:12:09.880 INFO:teuthology.orchestra.run:waiting for 300 2024-03-12T17:12:09.963 INFO:tasks.ceph.mgr.y:Stopped 2024-03-12T17:12:09.963 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-12T17:12:09.963 INFO:teuthology.orchestra.run:waiting for 300 2024-03-12T17:12:10.046 INFO:tasks.ceph.mgr.x:Stopped 2024-03-12T17:12:10.047 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-12T17:12:10.047 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-12T17:12:10.047 INFO:teuthology.orchestra.run:waiting for 300 2024-03-12T17:12:10.080 INFO:tasks.ceph.mon.a:Stopped 2024-03-12T17:12:10.080 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-12T17:12:10.080 INFO:teuthology.orchestra.run:waiting for 300 2024-03-12T17:12:10.113 INFO:tasks.ceph.mon.b:Stopped 2024-03-12T17:12:10.113 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-12T17:12:10.113 INFO:teuthology.orchestra.run:waiting for 300 2024-03-12T17:12:10.195 INFO:tasks.ceph.mon.c:Stopped 2024-03-12T17:12:10.196 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-12T17:12:10.196 DEBUG:teuthology.orchestra.run.smithi052:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-03-12T17:12:10.308 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi052.front.sepia.ceph.com 2024-03-12T17:12:10.308 DEBUG:teuthology.orchestra.run.smithi052:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-12T17:12:10.584 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi052.front.sepia.ceph.com 2024-03-12T17:12:10.585 DEBUG:teuthology.orchestra.run.smithi052:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-12T17:12:10.749 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi052.front.sepia.ceph.com 2024-03-12T17:12:10.749 DEBUG:teuthology.orchestra.run.smithi052:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-12T17:12:10.901 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi052.front.sepia.ceph.com 2024-03-12T17:12:10.901 DEBUG:teuthology.orchestra.run.smithi052:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-12T17:12:11.074 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi102.front.sepia.ceph.com 2024-03-12T17:12:11.074 DEBUG:teuthology.orchestra.run.smithi102:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-03-12T17:12:11.353 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi102.front.sepia.ceph.com 2024-03-12T17:12:11.353 DEBUG:teuthology.orchestra.run.smithi102:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-03-12T17:12:11.510 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi102.front.sepia.ceph.com 2024-03-12T17:12:11.511 DEBUG:teuthology.orchestra.run.smithi102:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-03-12T17:12:11.645 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi102.front.sepia.ceph.com 2024-03-12T17:12:11.645 DEBUG:teuthology.orchestra.run.smithi102:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-03-12T17:12:11.827 INFO:tasks.ceph:Archiving mon data... 2024-03-12T17:12:11.828 DEBUG:teuthology.misc:Transferring archived files from smithi052:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2024-03-12_14:59:27-fs-wip-yuri11-testing-2024-03-11-0838-reef-distro-default-smithi/7593724/data/mon.a.tgz 2024-03-12T17:12:11.828 DEBUG:teuthology.orchestra.run.smithi052:> mktemp 2024-03-12T17:12:11.888 INFO:teuthology.orchestra.run.smithi052.stdout:/tmp/tmp.NJ4cuEA4em 2024-03-12T17:12:11.889 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.NJ4cuEA4em 2024-03-12T17:12:12.089 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 0666 /tmp/tmp.NJ4cuEA4em 2024-03-12T17:12:12.253 DEBUG:teuthology.orchestra.remote:smithi052:/tmp/tmp.NJ4cuEA4em is 337KB 2024-03-12T17:12:12.364 DEBUG:teuthology.orchestra.run.smithi052:> rm -fr /tmp/tmp.NJ4cuEA4em 2024-03-12T17:12:12.422 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2024-03-12_14:59:27-fs-wip-yuri11-testing-2024-03-11-0838-reef-distro-default-smithi/7593724/data/mon.b.tgz 2024-03-12T17:12:12.422 DEBUG:teuthology.orchestra.run.smithi102:> mktemp 2024-03-12T17:12:12.479 INFO:teuthology.orchestra.run.smithi102.stdout:/tmp/tmp.OQxI6WmJJT 2024-03-12T17:12:12.480 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.OQxI6WmJJT 2024-03-12T17:12:12.687 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 0666 /tmp/tmp.OQxI6WmJJT 2024-03-12T17:12:12.836 DEBUG:teuthology.orchestra.remote:smithi102:/tmp/tmp.OQxI6WmJJT is 450KB 2024-03-12T17:12:12.945 DEBUG:teuthology.orchestra.run.smithi102:> rm -fr /tmp/tmp.OQxI6WmJJT 2024-03-12T17:12:13.000 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2024-03-12_14:59:27-fs-wip-yuri11-testing-2024-03-11-0838-reef-distro-default-smithi/7593724/data/mon.c.tgz 2024-03-12T17:12:13.001 DEBUG:teuthology.orchestra.run.smithi102:> mktemp 2024-03-12T17:12:13.051 INFO:teuthology.orchestra.run.smithi102.stdout:/tmp/tmp.kdBERfKyAb 2024-03-12T17:12:13.052 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.kdBERfKyAb 2024-03-12T17:12:13.241 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 0666 /tmp/tmp.kdBERfKyAb 2024-03-12T17:12:13.385 DEBUG:teuthology.orchestra.remote:smithi102:/tmp/tmp.kdBERfKyAb is 450KB 2024-03-12T17:12:13.492 DEBUG:teuthology.orchestra.run.smithi102:> rm -fr /tmp/tmp.kdBERfKyAb 2024-03-12T17:12:13.547 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-12T17:12:13.547 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-12T17:12:13.555 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-12T17:12:13.687 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-03-12T17:12:13.687 DEBUG:teuthology.orchestra.run.smithi052:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-03-12T17:12:13.695 DEBUG:teuthology.orchestra.run.smithi102:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-03-12T17:12:13.794 INFO:teuthology.orchestra.run.smithi102.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-03-12T17:12:13.798 INFO:teuthology.orchestra.run.smithi052.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-03-12T17:12:13.811 INFO:tasks.ceph:Archiving crash dumps... 2024-03-12T17:12:13.813 DEBUG:teuthology.misc:Transferring archived files from smithi052:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-03-12_14:59:27-fs-wip-yuri11-testing-2024-03-11-0838-reef-distro-default-smithi/7593724/remote/smithi052/crash 2024-03-12T17:12:13.813 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-12T17:12:13.890 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-03-12_14:59:27-fs-wip-yuri11-testing-2024-03-11-0838-reef-distro-default-smithi/7593724/remote/smithi102/crash 2024-03-12T17:12:13.891 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-12T17:12:13.989 INFO:tasks.ceph:Shutting down logrotate 2024-03-12T17:12:13.989 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-12T17:12:13.997 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-12T17:12:14.139 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-12T17:12:14.244 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-12T17:12:14.336 INFO:tasks.ceph:Compressing logs... 2024-03-12T17:12:14.337 DEBUG:teuthology.orchestra.run.smithi052:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-12T17:12:14.345 DEBUG:teuthology.orchestra.run.smithi102:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-12T17:12:19.155 INFO:tasks.ceph:Archiving logs... 2024-03-12T17:12:19.155 DEBUG:teuthology.misc:Transferring archived files from smithi052:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-03-12_14:59:27-fs-wip-yuri11-testing-2024-03-11-0838-reef-distro-default-smithi/7593724/remote/smithi052/log 2024-03-12T17:12:19.156 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-12T17:12:19.964 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-03-12_14:59:27-fs-wip-yuri11-testing-2024-03-11-0838-reef-distro-default-smithi/7593724/remote/smithi102/log 2024-03-12T17:12:19.965 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-12T17:12:20.527 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-12T17:12:20.537 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-12T17:12:20.537 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-12T17:12:20.545 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-12T17:12:20.687 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-12T17:12:20.697 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-12T17:12:20.697 DEBUG:teuthology.orchestra.run.smithi052:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-12T17:12:20.705 DEBUG:teuthology.orchestra.run.smithi102:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-12T17:12:20.753 INFO:teuthology.orchestra.run.smithi052.stderr:bash: line 1: ntpq: command not found 2024-03-12T17:12:20.758 INFO:teuthology.orchestra.run.smithi102.stderr:bash: line 1: ntpq: command not found 2024-03-12T17:12:20.764 INFO:teuthology.orchestra.run.smithi052.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-12T17:12:20.764 INFO:teuthology.orchestra.run.smithi052.stdout:=============================================================================== 2024-03-12T17:12:20.764 INFO:teuthology.orchestra.run.smithi052.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 46 +3565us[+3565us] +/- 101ms 2024-03-12T17:12:20.764 INFO:teuthology.orchestra.run.smithi052.stdout:^* hv02.front.sepia.ceph.com 3 6 377 47 -1495us[-1492us] +/- 64ms 2024-03-12T17:12:20.764 INFO:teuthology.orchestra.run.smithi052.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 -752us[ -752us] +/- 71ms 2024-03-12T17:12:20.764 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T17:12:20.767 INFO:teuthology.orchestra.run.smithi102.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-12T17:12:20.767 INFO:teuthology.orchestra.run.smithi102.stdout:=============================================================================== 2024-03-12T17:12:20.767 INFO:teuthology.orchestra.run.smithi102.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 45 +3603us[+3603us] +/- 101ms 2024-03-12T17:12:20.768 INFO:teuthology.orchestra.run.smithi102.stdout:^* hv02.front.sepia.ceph.com 3 6 377 46 -1469us[-1461us] +/- 64ms 2024-03-12T17:12:20.768 INFO:teuthology.orchestra.run.smithi102.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 48 -724us[ -715us] +/- 71ms 2024-03-12T17:12:20.768 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-12T17:12:20.770 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-12T17:12:20.780 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-12T17:12:20.781 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-12T17:12:20.791 DEBUG:teuthology.orchestra.run.smithi052:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-12T17:12:21.033 DEBUG:teuthology.orchestra.run.smithi102:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-12T17:12:21.246 DEBUG:teuthology.orchestra.run.smithi052:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-12T17:12:21.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T17:12:21.356 DEBUG:teuthology.orchestra.run.smithi102:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-12T17:12:21.457 INFO:teuthology.orchestra.run.smithi102.stdout:type=AVC msg=audit(1710262603.804:199): avc: denied { checkpoint_restore } for pid=1156 comm="agetty" capability=40 scontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tcontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tclass=capability2 permissive=1 2024-03-12T17:12:21.459 DEBUG:teuthology.task.selinux:ubuntu@smithi102.front.sepia.ceph.com has 1 denials 2024-03-12T17:12:21.459 ERROR:teuthology.run_tasks:Manager failed: selinux Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/__init__.py", line 136, in __exit__ self.teardown() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/selinux.py", line 166, in teardown self.get_new_denials() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/selinux.py", line 215, in get_new_denials raise SELinuxError(node=remote, teuthology.exceptions.SELinuxError: SELinux denials found on ubuntu@smithi102.front.sepia.ceph.com: ['type=AVC msg=audit(1710262603.804:199): avc: denied { checkpoint_restore } for pid=1156 comm="agetty" capability=40 scontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tcontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tclass=capability2 permissive=1'] 2024-03-12T17:12:21.460 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-12T17:12:21.470 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-12T17:12:21.482 INFO:teuthology.task.internal:Duration was 938.729268 seconds 2024-03-12T17:12:21.482 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-12T17:12:21.491 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-12T17:12:21.492 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-12T17:12:21.497 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-12T17:12:21.659 INFO:teuthology.orchestra.run.smithi052.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-12T17:12:21.659 INFO:teuthology.orchestra.run.smithi102.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-12T17:12:21.959 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-12T17:12:21.959 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi052.front.sepia.ceph.com 2024-03-12T17:12:21.959 DEBUG:teuthology.orchestra.run.smithi052:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-12T17:12:22.021 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi102.front.sepia.ceph.com 2024-03-12T17:12:22.022 DEBUG:teuthology.orchestra.run.smithi102:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-12T17:12:22.095 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-12T17:12:22.098 DEBUG:teuthology.orchestra.run.smithi052:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-12T17:12:22.106 DEBUG:teuthology.orchestra.run.smithi102:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-12T17:12:22.206 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-12T17:12:22.206 DEBUG:teuthology.orchestra.run.smithi052:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-12T17:12:22.212 DEBUG:teuthology.orchestra.run.smithi102:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-12T17:12:22.407 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-12T17:12:22.417 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-12T17:12:22.417 DEBUG:teuthology.orchestra.run.smithi052:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-12T17:12:22.518 DEBUG:teuthology.orchestra.run.smithi102:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-12T17:12:22.616 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-12T17:12:22.627 DEBUG:teuthology.orchestra.run.smithi052:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-12T17:12:22.634 DEBUG:teuthology.orchestra.run.smithi102:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-12T17:12:22.737 INFO:teuthology.orchestra.run.smithi052.stdout:kernel.core_pattern = core 2024-03-12T17:12:22.739 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern = core 2024-03-12T17:12:22.793 DEBUG:teuthology.orchestra.run.smithi052:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-12T17:12:22.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T17:12:22.845 DEBUG:teuthology.orchestra.run.smithi102:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-12T17:12:22.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T17:12:22.899 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-12T17:12:22.910 INFO:teuthology.task.internal:Transferring archived files... 2024-03-12T17:12:22.910 DEBUG:teuthology.misc:Transferring archived files from smithi052:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-03-12_14:59:27-fs-wip-yuri11-testing-2024-03-11-0838-reef-distro-default-smithi/7593724/remote/smithi052 2024-03-12T17:12:22.911 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-12T17:12:23.026 DEBUG:teuthology.misc:Transferring archived files from smithi102:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-03-12_14:59:27-fs-wip-yuri11-testing-2024-03-11-0838-reef-distro-default-smithi/7593724/remote/smithi102 2024-03-12T17:12:23.026 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-12T17:12:23.135 INFO:teuthology.task.internal:Removing archive directory... 2024-03-12T17:12:23.135 DEBUG:teuthology.orchestra.run.smithi052:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-12T17:12:23.144 DEBUG:teuthology.orchestra.run.smithi102:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-12T17:12:23.198 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-12T17:12:23.212 INFO:teuthology.task.internal:Not uploading archives. 2024-03-12T17:12:23.213 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-12T17:12:23.224 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-12T17:12:23.224 DEBUG:teuthology.orchestra.run.smithi052:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-12T17:12:23.234 DEBUG:teuthology.orchestra.run.smithi102:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-12T17:12:23.284 INFO:teuthology.orchestra.run.smithi052.stdout: 393493 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 12 17:12 /home/ubuntu/cephtest 2024-03-12T17:12:23.288 INFO:teuthology.orchestra.run.smithi102.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 12 17:12 /home/ubuntu/cephtest 2024-03-12T17:12:23.290 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-12T17:12:23.302 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-12T17:12:23.313 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: SELinuxError: SELinux denials found on ubuntu@smithi102.front.sepia.ceph.com: ['type=AVC msg=audit(1710262603.804:199): avc: denied { checkpoint_restore } for pid=1156 comm="agetty" capability=40 scontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tcontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tclass=capability2 permissive=1'] 2024-03-12T17:12:23.313 INFO:teuthology.nuke:Checking targets against current locks 2024-03-12T17:12:23.339 DEBUG:teuthology.nuke:shortname: smithi052 2024-03-12T17:12:23.340 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-12T17:12:23.363 DEBUG:teuthology.nuke:shortname: smithi102 2024-03-12T17:12:23.363 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-12T17:12:23.367 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi052.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-12_14:59:27-fs-wip-yuri11-testing-2024-03-11-0838-reef-distro-default-smithi/7593724', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-12 16:47:30.869855', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNP9kBxoQIrVoXMCHmLIBeBNFhyJHWPOQVv9rSZGWE09a9SJHaMmnq62ps8uuT4MnufX+/MMAvqYSiGfz0/5UbM='} 2024-03-12T17:12:23.388 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi102.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-12_14:59:27-fs-wip-yuri11-testing-2024-03-11-0838-reef-distro-default-smithi/7593724', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-12 16:47:30.871359', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEiIR1hJQrQZGb6VRJkvAr1V6lA0j7iFz+k9F4XpdzY2y1WZkNwMFGYXVspkew3cAc+Ow+rLYDlJyPFQcugo9NA='} 2024-03-12T17:12:23.417 INFO:teuthology.orchestra.console.smithi052:Power off 2024-03-12T17:12:23.418 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-12T17:12:23.444 DEBUG:teuthology.orchestra.console.smithi052:power off output: Chassis Power Control: Down/Off 2024-03-12T17:12:23.445 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T17:12:23.461 INFO:teuthology.orchestra.console.smithi102:Power off 2024-03-12T17:12:23.461 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-12T17:12:23.472 DEBUG:teuthology.orchestra.console.smithi052:check power output: Chassis Power is on 2024-03-12T17:12:23.485 DEBUG:teuthology.orchestra.console.smithi102:power off output: Chassis Power Control: Down/Off 2024-03-12T17:12:23.485 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T17:12:23.507 DEBUG:teuthology.orchestra.console.smithi102:check power output: Chassis Power is on 2024-03-12T17:12:27.474 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T17:12:27.508 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T17:12:27.587 DEBUG:teuthology.orchestra.console.smithi052:check power output: Chassis Power is on 2024-03-12T17:12:27.618 DEBUG:teuthology.orchestra.console.smithi102:check power output: Chassis Power is on 2024-03-12T17:12:31.588 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T17:12:31.619 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T17:12:31.698 DEBUG:teuthology.orchestra.console.smithi052:check power output: Chassis Power is off 2024-03-12T17:12:31.729 DEBUG:teuthology.orchestra.console.smithi102:check power output: Chassis Power is off 2024-03-12T17:12:31.799 INFO:teuthology.orchestra.console.smithi052:Power off completed 2024-03-12T17:12:31.830 INFO:teuthology.orchestra.console.smithi102:Power off completed 2024-03-12T17:12:31.936 INFO:teuthology.run:Summary data: client.0-kernel-sha1: bf761b6efc7e0ff8090671a5bd7700e429806836 client.1-kernel-sha1: bf761b6efc7e0ff8090671a5bd7700e429806836 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/acls} duration: 938.7292678356171 failure_reason: 'SELinux denials found on ubuntu@smithi102.front.sepia.ceph.com: [''type=AVC msg=audit(1710262603.804:199): avc: denied { checkpoint_restore } for pid=1156 comm="agetty" capability=40 scontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tcontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tclass=capability2 permissive=1'']' flavor: default owner: scheduled_yuriw@teuthology status: fail success: false 2024-03-12T17:12:31.936 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-12T17:12:32.021 INFO:teuthology.run:FAIL